Thread

1. kamranjon · 1789680375 · focus · HN ↗

   Love this for the folks with 16gb graphics cards - 3.8 27b has been incredible but not quite runnable on anything less than 32gb - will try loading this up on my 16gb intel b50 and see how it goes - not sure these quants can be accelerated by the XPU cores yet but maybe in time!

   1. djkoolaide · 1789695087 · focus · HN ↗

      Tried it today on a B70 and couldn't get anything usable out of it.

      Prism's llama.cpp fork only has the kernels for CUDA, CPU and Vulkan. No SYCL at all :(

Unofficial Hacker News client; not affiliated with Y Combinator.