‹ BackHN Continuity

Thread

Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone

310 points · 140 comments · leonickson

  1. jbird99 · · focus · HN ↗
    At what, 10 tokens per hour? These disk swapping methods all have the same drawbacks - kill your drive early, and slow as hell.
    1. wat10000 · · focus · HN ↗
      Isn’t it only writes that kill drives?
      1. Alpha3031 · · focus · HN ↗
        Yes for NAND, and I suppose nobody is using mechanical hard drives for this.
        1. wat10000 · · focus · HN ↗
          I'd like to see someone try it, just to see how incredibly slow and noisy it is.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.