AMD has acquired "AI stuff" for several billions at this point, yet somehow, doing "AI stuff" on their GPU/NPU stack still seems to be troublesome. Well, that is what I read from others anyway. Inference with llama.cpp on AMD GPUs pretty much just works.
Who says that? I’m having a blast with rocm powering my R9700. Been able to run all models on day 0 at comparable speeds to Nvidia with similar memory bandwidth. Software is no longer the main bottleneck for AMD.
just use torch/vllm/sglang/llama.cpp and the rest of the ecosystem, most have decent support for AMD, it's not that different from CUDA once you get it working (and it is much easier now than it used to)
LarsDu88 · · focus · HN ↗
I was also shocked by how quickly AMD acquired Talaas. AMD may be preparing for the next way (ultra fast inference, and embodied AI inference)
ahartmetz · · focus · HN ↗
sheepscreek · · focus · HN ↗
__rito__ · · focus · HN ↗
I don't know anything.
Things like training a Vision/RL/LM, downloading and running LMs, etc.
imjonse · · focus · HN ↗