Thread

1. sdan · 1790237943 · focus · HN ↗

   I tried running this on a H100 and got 190ms compared to Jev's 170ms. Maybe I set it up wrong?

   1. kevmo314 · 1790263613 · focus · HN ↗

      It seems like caching has a huge contribution towards the low latency numbers they report. With a single request and no cache it appears to be quite slow, roughly in line with what you observed.

Unofficial Hacker News client; not affiliated with Y Combinator.