Thread

Intelligence per Watt: Measuring Intelligence Efficiency of Local AI

169 points · 65 comments · pythonic_hell

jmiskovic · 1789557527 · focus · HN ↗
Incredibly important research. We've reached the point where local LLMs are good enough! It takes less time for local model to take the first action on your task than it does for Claude to validate your login, put you into queue and start issuing the commands. Local models are persistent and 100% predictable unlike any cloud offering. It's better for the power system for the demand to be distributed. During the winter time the GPU also doubles as a 300W in-house heater. Not to mention avoiding personal data collection and re-selling.
1. rckoepke · 1789566685 · focus · HN ↗
  > During the winter time the GPU also doubles as a 300W in-house heater.
  Just a reminder that heat pumps can consume 300W of electricity to provide 1200W of heat.
  1. stymaar · 1789572063 · focus · HN ↗
    Of course, but the power dissipated in a data center provide zero heat for your house.
    (Tangent: I wonder why we haven't seen deployment of organic Rankin cycle generators in AI data centers, the exhaust temperature should be compatible and that could yield a 10-20% energy bill saving).
    mattmanser · 1789572525 · focus · HN ↗
    I have no idea what they use, but Stockholm is already using the waste heat from data centres:
    <a href="https://eu-mayors.ec.europa.eu/en/news/stockholm-sweden-heat-recovery-data-centres" rel="nofollow">https://eu-mayors.ec.europa.eu/en/news/stockholm-sweden-heat...
    stymaar · 1789573077 · focus · HN ↗
    They use it in the city heating system, which is obviously the optimal way of reusing the heat but most data centers aren't put next to such a heating system.
    I'm talking about making electricity back from the heat (using a low-temp thermodynamic cycle). It has a low yield (due to the low input temperature) but it's usually economically viable when using heat that would end up in the heavens anyway.
    adgjlsfhk1 · 1789580481 · focus · HN ↗
    the heat is way too low grade. typical water cooling setups only have ~50C water. that's barely anything

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.