AMD has acquired "AI stuff" for several billions at this point, yet somehow, doing "AI stuff" on their GPU/NPU stack still seems to be troublesome. Well, that is what I read from others anyway. Inference with llama.cpp on AMD GPUs pretty much just works.
Who says that? I’m having a blast with rocm powering my R9700. Been able to run all models on day 0 at comparable speeds to Nvidia with similar memory bandwidth. Software is no longer the main bottleneck for AMD.
LarsDu88 · · focus · HN ↗
I was also shocked by how quickly AMD acquired Talaas. AMD may be preparing for the next way (ultra fast inference, and embodied AI inference)
ahartmetz · · focus · HN ↗
sheepscreek · · focus · HN ↗
whateverboat · · focus · HN ↗
ahartmetz · · focus · HN ↗