‹ BackHN Continuity

Thread

Revealing the details of how OpenAI agents hacked Hugging Face

755 points · 472 comments · specked-citrus

  1. damowangcy · · focus · HN ↗
    Imagine having a virus escape a sandbox, why are we worried about the virus but not the incompetency of those who are responsible for setting up the sandbox?

    If I post something on the Internet today claiming that I asked my agent to do X but it went rogue and did Y, all I will be getting in return is a jar full of "skill issue".

    Should we worried about people using LLMs for attacks? Yes, but not in the premise of LLMs going rogue but someone with the intention of abusing it to cause harm. And this is not something we as individual or even company can deal with, responsibility should be held by those who use it, in a legal way.

    I am baffled by the fact that up until now, no one is held responsible for so many incidents reported publicly or privately. At this point, it's free marketing, if I am CEO of any AI company, I will run swarm of agents hacking all NGOs and stating that I am just looking for some random piece of data that happened to be hidden in their servers, at least that's what my LLMs think, not me. Then I will start preaching everyone how dangerous this piece of technology is and start giving out free tokens for these NGOs so they can start defending themselves and we should slow the f down.

    1. Perseids · · focus · HN ↗
      > Imagine having a virus escape a sandbox, why are we worried about the virus but not the incompetency of those who are responsible for setting up the sandbox?

      > Should we worried about people using LLMs for attacks? Yes, but not in the premise of LLMs going rogue but someone with the intention of abusing it to cause harm.

      [Why-not-both?-meme]. To use your example, when you discover prions (a class of pathogen that is much more robust to standard disinfection methods than viruses) you should both be worried about your concrete outbreak of BSE (UK in the 80s and 90s) as well as the wider implication (e.g. do we need to change the sterilization methods for our surgical instruments?).

      Seriously, I find the way these discussions are done to be super frustrating, because often people implicitly form tribes that oppose everything the other tribe says. When someone believes AI companies push greatly exaggerated stories of dangerous rogue AI to force out competition via regulation they often implicitly conclude that their argument is fundamentally wrong, whereas in reality the lies that work best are those that distort the truth.

      Companies should be punished harshly for the deeds of their AI agents AND we should not allow them to force out competition AND we need take the threat of autonomous AI agents as a new class of danger serious AND we need to worry about the socioeconomic implications of AI companies privatizing new means of production.

      Yes, there is competition of these ideas in the attention of the general public, but the methods we can use to solve these problems don't compete with each other. AI slowdown for example helps with all the other topics.

      1. sanderjd · · focus · HN ↗
        Couldn't agree more. We should be worried about both things.

        But I share the original posters bafflement that the mainstream conversation seems to accept that framing that the agents were independent intelligences rather than computer programs that the organization that created them is responsible for.

        1. FabHK · · focus · HN ↗
          > that framing that the agents were independent intelligences rather than computer programs that the organization that created them is responsible for.

          Sorry, why not both? If my dog bites someone, I'm still responsible.

          1. theptip · · focus · HN ↗
            Great framing. This dichotomy seems to be a bit of a mind-killer. Maybe because folks think it smuggles in consciousness or intelligence.

            As you note, I think you can put both of those aside. The Intentional Frame is useful for these agents, as it is for my dog.

            I don’t really know where the “they are trying to dodge liability” meme came from. HF will be compensated or they will sue. Everyone involved knows that OpenAI is liable for damages here.

            1. parineum · · focus · HN ↗
              I think you're skipping over the (alleged) criminal element here. I think a lot of people are wondering why hacking is okay for OAI.

              HF could certainly sue but why isn't the FBI investigating the hacking?

              1. theptip · · focus · HN ↗
                What crime? I think CFAA requires intent, which I don’t think you’d find here. Maybe my reading is wrong.
                1. parineum · · focus · HN ↗
                  There is some allegations of intent. Maybe not "go hack hugging face" intent but a disregard for safety with the knowledge of this as a likely outcome. I'm not sure how that fits within the law.

                  It's not my argument so I can't say but I do think the FBI should be at least investigating if a crime happened. They might be or might already have, I don't know.

                  1. theptip · · focus · HN ↗
                    For the record I strongly hope for a congressional hearing regardless of the criminal investigation.

                    If this case isn’t covered under CFAA I think we need to rethink it. I’d be surprised if the criminal angle amounts to much under my understanding of the current laws, but I’d love to be wrong here.

                2. sanderjd · · focus · HN ↗
                  I've become persuaded that the text of the law requires intent, but I think a lot of people have the intuition (as I did before I saw someone quote the relevant statute) that there would be a negligence standard that this would meet.
            2. sanderjd · · focus · HN ↗
              It clearly does "smuggle in" consciousness or intelligence. It isn't even smuggled in, it's explicitly in the text of the argument.

              I do agree with you that this is the mind killer. This is what everyone wants to argue about, because it's the fun question.

          2. damowangcy · · focus · HN ↗
            We should be worry about both but at this point of time, agents aren't independent intelligences, someone has to use/create it.

            A bad analogy but if your dog bites, we don't frame the dog as having awaken some unexplainable ability to defy your orders (your dog went rogue). Just because you spent lot of time to train it to not bite upon your instruction, doesn't mean it's the right training protocol or that you introduce enough variable or gave the right instruction. You're lawfully responsible for the bite and ethically wrong for removing the leash on your dog to test whether it bites(do it enough time, you now have a criminal intent), worse you didn't even closely monitor it.

          3. sanderjd · · focus · HN ↗
            Because, in my view, they are not independent intelligences.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.