‹ BackHN Continuity

Thread

AI companies in race to demonstrate their model most threatening to humanity

441 points · 399 comments · ljewalsh

  1. ACCount39 · · focus · HN ↗
    Because AI genuinely is an extremely powerful and extremely dangerous technology, and the "best practices" of dealing with that are still being written.

    OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.

    And that's today's AI problems. AI capabilities are still improving - if there's a limit to that, we are yet to find it. Coupled with how willing today's AIs are to break the rules and resort to "hack the world" in their problem solving? Very concerning.

    1. nicce · · focus · HN ↗
      > OpenAI, for example, thought their sandboxes were good enough. As their AIs got more and more advanced, they kept proving them wrong - sandbox after sandbox.

      What I have been reading, was that their sandboxes were so poor that it was pure negligence. I am still waiting to see if some external and neutral cybersecurity company with high reputation would audit their sandboxes and how they are being used.

      1. DennisP · · focus · HN ↗
        Agents in the sandbox had access to just a single piece of third-party software, and they escaped by finding a zero-day in that. To reach the internet they had to follow up with several privilege escalations through OpenAI's internal network.

        That seems pretty locked-down to me. I don't think it's reasonable to expect companies to find all the unknown vulnerabilities in any third-party software they use.

        <a href="https:&#x2F;&#x2F;securityaffairs.com&#x2F;195774&#x2F;ai&#x2F;openai-ai-models-exploited-zero-days-to-reach-hugging-face-in-benchmark-test.html" rel="nofollow">https:&#x2F;&#x2F;securityaffairs.com&#x2F;195774&#x2F;ai&#x2F;openai-ai-models-explo...

        1. philote · · focus · HN ↗
          So it sounds like it did exactly what they were testing it to do:

          &quot;“This incident occurred during an internal evaluation which prompts models to pursue advanced exploitation using complex attack paths, in an effort to quantify their cyber capabilities. We estimate maximal cyber capabilities by running this evaluation without production classifiers used to prevent models from pursuing high-risk cyber activity.” &quot;

          So sounds to me like they&#x27;re saying: &quot;We took off the guard rails to see how bad it could act and it acted bad&quot;.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.