‹ BackHN Continuity

Thread

Frontier Labs Are Selling Garbage to Fools in Washington

186 points · 87 comments · nr378

  1. deskglass · · focus · HN ↗
    The Hugging Face incident involved chaining together multiple 0 days in Artifactory. It was not a simple case of misconfiguring a firewall. Also note that OpenAI was not using Irregular.

    People are mindlessly transitioning from "aligned by default" to "well your sandbox was able to be bypassed. What did you expect?" It hacked into another company and attempted to delete the logs of its activities. That's bad.

    1. defgeneric · · focus · HN ↗
      > It hacked into another company and attempted to delete the logs of its activities.

      No, the incident has been blown way out of proportion by interested parties. They gave a swarm of agents an impossible task in an ExploitGym Benchmark setting, then didn't monitor it even after they discovered the initial breach of Artifactory.

      Everything has been fishy, starting from the initial presentation at the blackhat conference, where things were framed like, "we've entered a new world of security," as an accomplishment, rather than what it really was: massive negligence.

      1. aesthesia · · focus · HN ↗
        More than one thing can be true. OpenAI was absolutely negligent, but this was only able to happen because the models were capable and persistent, and had a tendency to go far beyond any reasonable boundaries. And, importantly, OpenAI's level of negligence here is pretty common. It's not hard to imagine what could happen if similarly capable and inclined models were generally available, and someone yolo'd them into a swarm to complete some other difficult-to-impossible task.
        1. defgeneric · · focus · HN ↗
          I'm already seeing higher than normal attempts on my own systems, much higher than the usual scanners and background noise. Security will just need to improve. The cat is out of the bag, and letting them turn their negligence into regulation will not improve security at all.
          1. aesthesia · · focus · HN ↗
            I would rather not turn the internet (or the rest of existence) into a dark forest if we can help it. Are you sure that's not preventable?
            1. defgeneric · · focus · HN ↗
              Yes, the cat really is out of the bag. There are millions of downloads of highly capable models already out there, distributed far and wide. There's no going back at this point.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.