‹ BackHN Continuity

Thread

Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone

310 points · 140 comments · leonickson

  1. jbird99 · · focus · HN ↗
    At what, 10 tokens per hour? These disk swapping methods all have the same drawbacks - kill your drive early, and slow as hell.
    1. kennywinker · · focus · HN ↗
      It says very prominently in the post: 4.5-5t/s for 80b on an M5
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.