‹ BackHN Continuity

Thread

How GLM built its own inference infrastructure

411 points · 285 comments · whiteros_e

  1. zicohacks · · focus · HN ↗
    US chip export restrictions may actually be an advantage for China's AI Infrastructure. Chinese companies are forced to speed up developing their own AI chips
    1. menaerus · · focus · HN ↗
      It was evident that this will happen.

      > Compared with our initial baseline on the same hardware, we achieved a 3× improvement in end-to-end serving performance, reaching hardware efficiency and per-token cost comparable to mainstream NVIDIA GPUs. This demonstrates that Chinese chips can support frontier-model inference efficiently and economically at scale.

      1. HSO · · focus · HN ↗

        [dead]

        1. goodmythical · · focus · HN ↗
          Isn't it common knowledge that every better mouse trap breeds smarter mice?
          1. conmod278 · · focus · HN ↗
            Future AI systems will eke out every last blood drop of performance from any kind of hardware. Not even a single bit flip will go to waste.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.