‹ BackHN Continuity

Thread

Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

169 points · 65 comments · pythonic_hell

  1. jmiskovic · · focus · HN ↗
    Incredibly important research. We've reached the point where local LLMs are good enough! It takes less time for local model to take the first action on your task than it does for Claude to validate your login, put you into queue and start issuing the commands. Local models are persistent and 100% predictable unlike any cloud offering. It's better for the power system for the demand to be distributed. During the winter time the GPU also doubles as a 300W in-house heater. Not to mention avoiding personal data collection and re-selling.
    1. rckoepke · · focus · HN ↗
      > During the winter time the GPU also doubles as a 300W in-house heater.

      Just a reminder that heat pumps can consume 300W of electricity to provide 1200W of heat.

      1. stymaar · · focus · HN ↗
        Of course, but the power dissipated in a data center provide zero heat for your house.

        (Tangent: I wonder why we haven't seen deployment of organic Rankin cycle generators in AI data centers, the exhaust temperature should be compatible and that could yield a 10-20% energy bill saving).

        1. MisterTea · · focus · HN ↗
          > I wonder why we haven't seen deployment of organic Rankin cycle generators in AI data centers

          Because there was never a long term plan for AI data centers. It's an AI market capture and cash grab scheme that ends when local models eat their lunch.

          1. drivebyhooting · · focus · HN ↗
            That makes no sense. Do you think their plan is to spend billions of dollars and build infrastructure mega projects just to let it sit idle?

            Did the rise personal computing make data centers and super computers obsolete?

            Any advance in inference that allows local models to do the job will also benefit hyperscalers. Imagine the sheer amount of compute they could throw at problems if each 32GB of VRAM was enough for frontier reasoning.

            1. AngryData · · focus · HN ↗
              They could also be betting that there will only be 1 or 2 big winners. And if they don't win they are saving on expenses, if they do win then they will have all the money they need to upgrade later if they want.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.