Thread

Shapelearn Qwen 3.8 27B (13.1 GB VRAM)

104 points · 39 comments · syntaxing

sheo · 1789709466 · focus · HN ↗
Aged like milk
<a href="https://news.ycombinator.com/item?id=49746618">https://news.ycombinator.com/item?id=49746618
1. Mashimo · 1789711655 · focus · HN ↗
  In the comments it reads like bonsei falls apart on longer running tasks.
2. txrx0000 · 1789711974 · focus · HN ↗
  Not really. The largest IQ4_XS quant here is still worth it because Bonsai doesn't offer larger quants. They could beat it if they made a quaternary variant though, I don't know why they're stopping at ternary.
  1. rguiscard · 1789716075 · focus · HN ↗
    I wonder the same thing for Bonsai 2. ByteShape offers 5 models from IQ2_XXS-2.56bpw (8.8GB), IQ3_XXS-2.88bpw (9.9GB), IQ3_XS-3.01bpw (10.4GB), IQ3_S-3.23bpw (11.0GB) to IQ4_XS-3.84bpw (13.1GB). Their benchmarks show gradual improvement with size and users can pick one to fit theirs need. Bonsai-2-27B now is about 8.6GB. It might be good to have a quaternary version around 10-11GB to fit a computer with 16-24GB RAM.
  2. [deleted] · 1789721548 · focus · HN ↗
    [deleted]
  3. dingdingdang · 1789744394 · focus · HN ↗
    Agree here, as it is ByteShape wins practicality wise if the goal is doing actual work with these quants!
3. electroglyph · 1789732799 · focus · HN ↗
  prismml's title is very misleading. in their own paper the model is at 75% of coding scores.

Unofficial Hacker News client; not affiliated with Y Combinator.