‹ BackHN Continuity

Thread

From the creator of Redis; run LLM locally with ds4

359 points · 103 comments · fibo

  1. simoiacos · · focus · HN ↗
    Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet.

    I'm also looking into expanding the protocol and the engine to support various steering techniques.

    <a href="https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith

    1. ilaksh · · focus · HN ↗
      I wish someone would add Intel support to ds4. And also improve AMD support.

      Maybe Intel and AMD should help them with that.

      1. simoiacos · · focus · HN ↗
        Yeah I see the value but I built Xenolith to target smaller models.

        I heard antirez saying that he designed DwarfStar also to be forked and tuned to everyone&#x27;s specific needs. Do you have a specific machine&#x2F;spec in mind?

        1. ilaksh · · focus · HN ↗
          The recent Intel GPU&#x2F;AI cards. Really the same type of models as ds4
      2. ABS · · focus · HN ↗
        AMD sent antirez a Strix Halo back in June for this purpose
        1. neomantra · · focus · HN ↗
          There were ROCM commits to ds4 in late summer, so that&#x27;s probably related. We include a ROCm build with ds4go (links elsewhere in this comment section), but it is absolutely untested by us whereas the Mac and DGX Spark are very tested.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.