Thread

1. api · 1789553947 · focus · HN ↗

   Unless I misread it, are they saying local GPUs use less energy?

   That’s surprising, almost unbelievable, due to batching. Local is usually not batched.

   1. stymaar · 1789555171 · focus · HN ↗

      Small models are much smaller than frontier models though, which is how they end up consuming less energy despite low batch count. (Though with local models growing strong agentic capabilities, batching becomes a reality with local models as well).

Unofficial Hacker News client; not affiliated with Y Combinator.