‹ BackHN Continuity

Thread

Show HN: Janus – Go binary that runs GGUF models via Vulkan on AMD/Intel/Nvidia

104 points · 19 comments · Maverick617

  1. peddling-brink · · focus · HN ↗
    > llama.cpp via Vulkan (AMD / Intel / NVIDIA) or CPU fallback

    I got excited about someone paying attention to intel. Oh well.

    1. kamranjon · · focus · HN ↗
      llama.cpp sycl and vllm xmx work is pretty incredible right now - you just gotta build it with some extra flags
      1. peddling-brink · · focus · HN ↗
        Llama would be nice for the ggufs. Any specific flags or tutorials I should look at?
        1. gunalx · · focus · HN ↗
          The docs are a great start.

          <a href="https:&#x2F;&#x2F;github.com&#x2F;ggml-org&#x2F;llama.cpp&#x2F;blob&#x2F;master&#x2F;docs&#x2F;backend&#x2F;SYCL.md" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;ggml-org&#x2F;llama.cpp&#x2F;blob&#x2F;master&#x2F;docs&#x2F;backe...

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.