Thread

1. kamranjon · 1789680375 · focus · HN ↗

   Love this for the folks with 16gb graphics cards - 3.8 27b has been incredible but not quite runnable on anything less than 32gb - will try loading this up on my 16gb intel b50 and see how it goes - not sure these quants can be accelerated by the XPU cores yet but maybe in time!

   1. kennywinker · 1789708968 · focus · HN ↗

      I run qwen 27b on an old-ass 16gb gpu. It’s very possible using unsloth 2bit and 3bit quants, tho there are a bunch of interesting quants that let you run closer to 4bit on 16gb. This article that’s currently also on the front page mentions a bunch of them while discussing their own quant <a href="https:&#x2F;&#x2F;byteshape.com&#x2F;blogs&#x2F;Qwen3.8-27B&#x2F;" rel="nofollow">https:&#x2F;&#x2F;byteshape.com&#x2F;blogs&#x2F;Qwen3.8-27B&#x2F;

Unofficial Hacker News client; not affiliated with Y Combinator.