Thread

1. hypercube33 · 1790793332 · focus · HN ↗

   From your description looks like this isn't for AMD or Strix Halo at all? Also one of the things I'm not sure of but definitely plays a huge factor is the variant of the model you download - how does this help select the fastest version for your specific hardware / context size?

   1. anerli · 1790793997 · focus · HN ↗

      We support Vulkan as well, we just didn't mention it in the benchmark. When AMD or Strix Halo is detected the engine will use Vulkan.

      Regarding model variants - our catalog includes different quantizations, and automatically assesses these against your hardware to determine which ones will fit in your memory and how fast they will run. This lets you pick a model to download based on your desired speed/intelligence tradeoff.

      1. hypercube33 · 1791147584 · focus · HN ↗

         Interesting, I'll give it a spin. Thanks for the info - the post header here illuded that you dont support it, but the page does indeed say AMD and Vulkan.

         1. hypercube33 · 1791148897 · focus · HN ↗

            Update -

            Install is easy. I like the ability to find models for my system with estimated performance.

            Cons: I cant re-use models for other systems like LM Studio I have saved to a folder on my system; it wants to re-download 40GB models.

Unofficial Hacker News client; not affiliated with Y Combinator.