AMD has acquired "AI stuff" for several billions at this point, yet somehow, doing "AI stuff" on their GPU/NPU stack still seems to be troublesome. Well, that is what I read from others anyway. Inference with llama.cpp on AMD GPUs pretty much just works.
Really, it's just the Python frameworks on AMD consumer GPUs that suck. AMD datacenter GPUs seem to work with them well enough, and llama.cpp crushes it for the rest of us.
LarsDu88 · · focus · HN ↗
I was also shocked by how quickly AMD acquired Talaas. AMD may be preparing for the next way (ultra fast inference, and embodied AI inference)
ahartmetz · · focus · HN ↗
MrDrMcCoy · · focus · HN ↗