‹ BackHN Continuity

Thread

From the creator of Redis; run LLM locally with ds4

359 points · 103 comments · fibo

  1. simoiacos · · focus · HN ↗
    Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet.

    I'm also looking into expanding the protocol and the engine to support various steering techniques.

    <a href="https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith

    1. ilaksh · · focus · HN ↗
      I wish someone would add Intel support to ds4. And also improve AMD support.

      Maybe Intel and AMD should help them with that.

      1. ABS · · focus · HN ↗
        AMD sent antirez a Strix Halo back in June for this purpose
        1. neomantra · · focus · HN ↗
          There were ROCM commits to ds4 in late summer, so that&#x27;s probably related. We include a ROCm build with ds4go (links elsewhere in this comment section), but it is absolutely untested by us whereas the Mac and DGX Spark are very tested.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.