Thread

1. ACCount39 · 1790595494 · focus · HN ↗

   Because AI genuinely is an extremely powerful and extremely dangerous technology, and the "best practices" of dealing with that are still being written.

   OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.

   And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.

   1. nicce · 1790597864 · focus · HN ↗

      > OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.

      What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.

      1. simianwords · 1790598954 · focus · HN ↗

         The model found and exploited and chained together previously unknown vulnerabilities.

         How were the sandboxes poor?

         1. dns_snek · 1790600411 · focus · HN ↗

            Agents didn't have real network isolation. They were indirectly connected to the internet via a jump host running insecure software which was never designed or hardened to provide any kind of isolation.

            1. simianwords · 1790600899 · focus · HN ↗

               But this level of isolation is what happens normally. At least in my university and another company I worked at. It wasn’t running insecure software, as far as anyone knew, it was secure

               1. dns_snek · 1790601920 · focus · HN ↗

                  There's levels of isolation, a padlock is not equivalent to a bank vault. If you claim to be building a possibly world-ending AI then you don't get to use a padlock and call it a day.

                  1. simianwords · 1790602400 · focus · HN ↗

                     And they are not calling it a day and they have done most things possible to communicate the fact that they are building something dangerous.

                     1. uneoneuno · 1790604422 · focus · HN ↗

                        What about the waves of employees quitting both OAI and anthropic stating as their reason that they see their work as a direct threat to the future of humanity, and that their management isn't taking it seriously enough. Is that a PR stunt too?

Unofficial Hacker News client; not affiliated with Y Combinator.