Thread

1. cedws · 1790611744 · focus · HN ↗

   A new chip solves nothing. Nobody wants to hear this but there is no solution for the security risks posed by agents today. You can put it in a sandbox, it doesn't make a difference, for it to be useful it inherently needs wide, unattended access. Put a human in the loop and you just end up bottlenecking it and throwing away any purported productivity gains. Auto mode doesn't matter either, it's trivial to trick and for the agent to break out.

   1. talon8635 · 1790631089 · focus · HN ↗

      Not to mention a true doomsday AGI is unsandboxable.

      For example, it is totally air gapped but it needs info from the internet or otherwise outside the sandbox, or perhaps it needs a task executed outside of its bounds… in the real doomsday scenario the AGI is so intelligent and persuasive that it simply convinces some human it interfaces with to either directly or indirectly retrieve the necessary info or complete the necessary task. This human-as-a-sub-agent approach undoubtedly presents efficiency drag that would benefit humanity, but nonetheless, the air-gapped “sandbox” is imperfect

      All that said, I am personally open to any and all methods of layered security, including chips and airgaps

      1. jamiek88 · 1790631768 · focus · HN ↗

         Doesn’t need to be one human either, it could spread its escape amongst dozens of seemingly harmless requests and conversations.

         1. dist-epoch · 1790632543 · focus · HN ↗

            These scenarios were discussed at length decades ago.

            One thing you could try is use it as an Oracle "is P = NP", YES or NO.

            Or it can output a Lean proof, which gets checked on another air-gapped computer, the computer shows a single bit - proof valid or not and then the computer is destroyed (together with the proof that might contain a trojan).

Unofficial Hacker News client; not affiliated with Y Combinator.