Thread

Revealing the details of how OpenAI agents hacked Hugging Face

755 points · 472 comments · specked-citrus

GuB-42 · 1790374877 · focus · HN ↗
So ugly...
It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.
People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.
Also, it looked so "loud", querying millions of URL with weird requests. The sandbox as weak as it can get, and there is absolutely zero smart extrusion detection or it would have found it. They used their best AI for attacking, but nothing for protection.
1. soundworlds · 1790383252 · focus · HN ↗
  As people keep repeating, this is simply a product negligence issue. There is a mythic quality given to "AI" in the narratives that Dario and Sam are pushing which somehow lets AI companies offload responsibility for things that would be an extreme liability in every other industry.
  I'm a big fan of Jensen Huang's interviews at the moment - these companies should be validating their software is safe before releasing it to the public. And if the current CEOs can't ensure that internal testing is done safely, they should step aside for CEOs that can.
  1. no-name-here · 1790399677 · focus · HN ↗
    > these companies should be validating their software is safe before releasing it to the public
    1. That's what they are calling for: <a href="https://darioamodei.com/post/we-must-pace-the-frontier" rel="nofollow">https://darioamodei.com/post/we-must-pace-the-frontier
    2. The incident we're discussing occurred while the company was validating their software is safe before releasing it to the public.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.