Thread

1. vblanco · 1790455179 · focus · HN ↗

   380.000 concurrent sandboxes on 160 Epyc based server nodes. Crazy stuff

   1. dangoodmanUT · 1790460812 · focus · HN ↗

      that's only 12 sandboxes per core

      1. aabhay · 1790461339 · focus · HN ↗

         Um, how is that not impressive

         1. r_lee · 1790465528 · focus · HN ↗

            if it's agentic stuff, they likely aren't hammering a core constantly and they will maybe sit idle quite often between model requests, so it makes sense. I do wonder how much memory they allocate to each one though.

            it's just very efficient use of shared cores that is required to make these kinds of workloads cost efficient

            1. eru · 1790484633 · focus · HN ↗

               If they sit mostly idle, you can swap out a lot of the memory to SSD, I guess.

Unofficial Hacker News client; not affiliated with Y Combinator.