12 sandboxes per code is insane, I wonder how many of these sandboxes are idle at a time. Depending on the tasks assigned the resource requirements are different. Compare an agent doing pdf conversion and one responding to a simple question. One is cpu bound the other is mostly network wait.
This is an interesting problem from infra perspective since you cannot predict the workload. On a bigger scale you may get away with forecasts.
Im waiting for tech that elastically allocates cpu/mem without restarting a container.
piterrro · · focus · HN ↗
This is an interesting problem from infra perspective since you cannot predict the workload. On a bigger scale you may get away with forecasts.
Im waiting for tech that elastically allocates cpu/mem without restarting a container.