Thread

1. _ache_ · 1789708917 · focus · HN ↗

   From my own test. It's not faster than the unsloth model.

   Disclarer: I'm unsing Vulkan on an AMD GC.

   1. DiabloD3 · 1789719037 · focus · HN ↗

      Surprised its not meaningfully slower.

      Vulkan and ROCm paths are missing a few optimized versions of the quants they're using.

      1. GrayShade · 1789748032 · focus · HN ↗

         I see around 1 tk/s after a while, so perhaps it is.

Unofficial Hacker News client; not affiliated with Y Combinator.