Thread

1. GuB-42 · 1790374877 · focus · HN ↗

   So ugly...

   It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.

   People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.

   Also, it looked so "loud", querying millions of URL with weird requests. The sandbox as weak as it can get, and there is absolutely zero smart extrusion detection or it would have found it. They used their best AI for attacking, but nothing for protection.

   1. ctolsen · 1790377693 · focus · HN ↗

      My biggest takeaway from this is just how godawful the sandboxing is. The stuff written up in OpenAIs report says more about lack of extremely basic sysadmin skills than anything else.

      I’m not that surprised about models with endless compute being capable of this, I’m more surprised that a company with the resources they have apparently can only create a sandbox that a half skilled human operator could have broken out of easily.

      1. sdenton4 · 1790399933 · focus · HN ↗

         If you're testing models by telling them 'go wild, do the evil so we can test how good you can do the evil' and have p(doom)>0, you should not have a sandbox.

         You should have a fscking air gap.

         Treat it like nukes when you're turning the safety filters off. This is very much OpenAI screwing up, running obviously unsafe tests.

         1. bitwize · 1790415729 · focus · HN ↗

            Indeed, this whole story has "farmer leaves barn door open and has shocked-pikachu-face when his horses escape" energy.

Unofficial Hacker News client; not affiliated with Y Combinator.