‹ BackHN Continuity

Thread

Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

589 points · 200 comments · JonSchneider

  1. Aurornis · · focus · HN ↗
    These are small enough that you can run them entirely in the browser <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;webml-community&#x2F;ternary-bonsai-2-webgpu-kernels" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;spaces&#x2F;webml-community&#x2F;ternary-bonsai...

    Remember to clear the downloaded weights afterward.

    Like the last model, it&#x27;s amazing they work as well as they do. Use it for any longer task and they fall apart spectacularly and in interesting ways.

    1. aschobel · · focus · HN ↗
      40 tokens a second on my M4 Max (Safari). Wild times.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.