‹ BackHN Continuity

Thread

Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

589 points · 200 comments · JonSchneider

  1. Dwedit · · focus · HN ↗
    I tried it on my 6GB GPU and got 0.67 tokens per second. Need more than 6GB to run it well.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.