US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips
> Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
you can also derive some stats from the ~10T tokens a day on 100k devices, 100M / device / day, but then one has to account for the multi-gpu model size, and I need coffee before I go there
zicohacks · · focus · HN ↗
menaerus · · focus · HN ↗
> Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.
verdverm · · focus · HN ↗
a34729t · · focus · HN ↗