‹ BackHN Continuity

Thread

Show HN: Run an 80B Qwen in 4.3 GB of RAM on a Mac, and a 35B on an iPhone

310 points · 140 comments · leonickson

  1. nc55g3g · · focus · HN ↗
    Running a 35B on iPhone at 1 tok/s with 2.5 GB RAM… this is the future of on-device inference. Insane work.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.