Thread

Bonsai 2 27B: Near-Lossless Compression in a 9x Smaller Footprint

589 points · 200 comments · JonSchneider

danbrooks · 1789684929 · focus · HN ↗
Nice! Does anyone know how this compares to the Unsloth quantizations of this model? <a href="https://unsloth.ai/docs/models/qwen3.8#run-qwen3.8-guide">https://unsloth.ai/docs/models/qwen3.8#run-qwen3.8-guide
1. WithinReason · 1789713854 · focus · HN ↗
  Unsloth has been dethroned by ISTA:
  <a href="https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF" rel="nofollow">https://huggingface.co/ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF
  The 3-bit quant is lossless based on benchmarks.
  1. raylad · 1789725475 · focus · HN ↗
    I just checked that model (ISTA-DASLab/Qwen3.8-27B-GSQ-RCO-GGUF:IQ3_S) and it does much much worse on the "Please recite Jabberwocky" test than the original bf16 does.
    The bf16 only misses "snicker-snack" and this quantization becomes confused after the first stanza.
    WithinReason · 1789737626 · focus · HN ↗
    I tried and ended up with:
    I'm going to stop here and be direct: I'm having trouble recalling the exact text, and every attempt above is me guessing. Rather than present a mangled version as the real poem, I'd recommend you look it up — it's very short and in the public domain, so any text of Through the Looking-Glass will have it verbatim. If you'd like, I can help with the moral of the poem ("'twas the blessing of the Bird..."), the famous Humpty Dumpty word interpretations ("slithy" = lithe + sinister, "mimsy" = miserable + mys... etc.), or Carroll's original annotations for the coined words — that part I can do reliably.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.