Thread

Run Qwen 3.8 Flash Next (125B) on consumer hardware (RTX 4090) at 100T/s

741 points · 330 comments · snehesht

panny · 1791121753 · focus · HN ↗
I'm far less interested in how good a big expensive model is on hardware 99% of people can't afford and would rather see what runs best on a chromebook or mobile phone with 8GB of RAM.
1. somenameforme · 1791123106 · focus · HN ↗
  The card in question here had an initial MSRP of $1600. It's been bumped up by the market, probably because it turned out it's nice for things like this, but it's hardly in the 99% can't afford domain, especially if you're using it to replace a never-ending rent at which point it will pay for itself very rapidly, especially for heavy LLM users.
  In any case, we've gone from requiring supercomputers, to requiring very high end computers, to requiring $1600 video cards. It's tracking the exact same path that image rendering systems took (which if you haven't been keeping up there, now run excellently on pretty much any plain old computer), and we'll probably be there within a couple of years if not much sooner.
  1. the__alchemist · 1791128395 · focus · HN ↗
    It was available at this MSRP 3 years ago, direct from Nvidia. It now goes for 3-4k USD, as you point out. MSRP stopped being a useful value for graphics card around that time. You realize the price hike, but still mentioned that 1.6k figure after.
    No normal person is spending 3-4k on a GPU from 3 years ago. The availability is also of questionable provenance.
    somenameforme · 1791130909 · focus · HN ↗
    The MSRP is a good proxy for the 'level' of a card. It's not like the 4090 was a freak outlier. Cards of a comparable price offer comparable performance. So the 'level' for running a frontier level model at high performance is now at $1600 and continuing to trend sharply downwards.
    Another nuance is that the computer hardware market is currently extremely inefficient in a way I don't understand. You can pick these cards up locally at places throughout Asia for around $2k new. That's retail single unit prices. No idea what's stopping somebody from closing the gap and making a ton of money - perhaps tariffs and data centers purchasing in a price insensitive fashion. Whatever the exact reason may be, what people pay for hardware is increasingly just radically different depending on where you buy it at.
    the__alchemist · 1791133545 · focus · HN ↗
    That arbitrage opportunity is remarkable. Maybe duty/taxes, as you say?
    I'm the guy who (Who plays games and runs molecular dynamics simulations and other CUDA stuff) said 3 years ago "$1600 for a graphics card? That is excessive. I'll upgrade in a few years when ready" And bought a 4080 for $1200 from Nvidia instead of the 4090. Oops! Now there is no reasonable upgrade path.
  2. panny · 1791135802 · focus · HN ↗
    >The card in question here had an initial MSRP of $1600.
    63% of Americans can't come up with $400 in an emergency.
    <a href="https://www.investopedia.com/here-s-how-many-americans-can-t-afford-a-usd400-emergency-the-numbers-may-shock-you-11814788" rel="nofollow">https://www.investopedia.com/here-s-how-many-americans-can-t...
    The richest country in the world. Where all 50 states consume more than any other country in the world.
    <a href="https://x.com/cremieuxrecueil/status/2102889196000256219" rel="nofollow">https://x.com/cremieuxrecueil/status/2102889196000256219
    Can't come up with %25 of that in an emergency. (Even though the real price is something like 2-3x more than MSRP)
    It must be nice, up there where you are so incredibly disconnected from reality.
    somenameforme · 1791174093 · focus · HN ↗
    You misread your article, which really shouldn't pass a sniff test. 77% could cover the $400, with 14% of that being from people selling/credit. It also mentioned that 70% have enough saved away to cover all expenses for at least 3 months, including 15% that could do so by selling/credit. Also the US is far from the richest country in the world in terms of the population. We're 28th in terms of median wealth. [1] And that's nominal - measure it PPP and we'd be much lower. For the times they are a-changin'.
    [1] - <a href="https://en.wikipedia.org/wiki/List_of_countries_by_wealth_per_adult" rel="nofollow">https://en.wikipedia.org/wiki/List_of_countries_by_wealth_pe...

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.