‹ BackHN Continuity

Thread

Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

169 points · 65 comments · pythonic_hell

  1. jmiskovic · · focus · HN ↗
    Incredibly important research. We've reached the point where local LLMs are good enough! It takes less time for local model to take the first action on your task than it does for Claude to validate your login, put you into queue and start issuing the commands. Local models are persistent and 100% predictable unlike any cloud offering. It's better for the power system for the demand to be distributed. During the winter time the GPU also doubles as a 300W in-house heater. Not to mention avoiding personal data collection and re-selling.
    1. rckoepke · · focus · HN ↗
      > During the winter time the GPU also doubles as a 300W in-house heater.

      Just a reminder that heat pumps can consume 300W of electricity to provide 1200W of heat.

      1. stymaar · · focus · HN ↗
        Of course, but the power dissipated in a data center provide zero heat for your house.

        (Tangent: I wonder why we haven't seen deployment of organic Rankin cycle generators in AI data centers, the exhaust temperature should be compatible and that could yield a 10-20% energy bill saving).

        1. MisterTea · · focus · HN ↗
          > I wonder why we haven't seen deployment of organic Rankin cycle generators in AI data centers

          Because there was never a long term plan for AI data centers. It's an AI market capture and cash grab scheme that ends when local models eat their lunch.

          1. drivebyhooting · · focus · HN ↗
            That makes no sense. Do you think their plan is to spend billions of dollars and build infrastructure mega projects just to let it sit idle?

            Did the rise personal computing make data centers and super computers obsolete?

            Any advance in inference that allows local models to do the job will also benefit hyperscalers. Imagine the sheer amount of compute they could throw at problems if each 32GB of VRAM was enough for frontier reasoning.

            1. perfectbeeing · · focus · HN ↗
              CCC was building the Cray-3/SSS when it went into Chapter 11 bankruptcy in March 1995.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.