‹ BackHN Continuity

Thread

From the creator of Redis; run LLM locally with ds4

359 points · 103 comments · fibo

  1. simoiacos · · focus · HN ↗
    Nothing comparable but inspired from DwarfStar I wrote a little inference engine for Intel Xe-LP (no XMX) 32GB laptops. The only model supported right now is a quantized Gemma-4, but I don't exclude in the future to support other MoE of similar size. Too bad we have no Qwen 3.8 35B-A3B yet.

    I'm also looking into expanding the protocol and the engine to support various steering techniques.

    <a href="https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;simoneiacomino&#x2F;xenolith

    1. jacquesm · · focus · HN ↗
      I can&#x27;t run it because I don&#x27;t have that hardware but that looks pretty neat, kudos!
      1. simoiacos · · focus · HN ↗
        Thank you, appreciate it!
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.