The level of technical understanding of journalists is so deplorably low… There is no rogue agents. That’s not a thing. Nothing about the HuggingFace incident has anything to do with an agent going rogue. It’s standard software that resulted in a hack, which is pretty much the expected output of the system OpenAI engineers implemented.
The company is obviously responsible for what their systems are doing
>The company is obviously responsible for what their systems are doing
Under what law specifically?
Just because you believe they should be held responsible doesn't mean the current law is written that way. CFAA charges require the defendant "acted knowingly or intentionally", you think openai intentionally acted to hack those sites? Would "they should have pretty much expected that output" stand up in court for criminal charges?
If I, taking after a fellow Austrian, ask you to enter a room with a Cesium atom and some poison, would I not be responsible for what happens cause it’s not deterministic? Negligence is a thing and adding randomness doesn’t change that.
OpenAIs models since 5.5 were troublesome in ways even a layman like me could reproduce, their testing environments (“sandbox”) downright a showcase of what not to do and they, despite being one of the biggest labs, didn’t observe what any of their models output for weeks after multiple prior incidents. They had multiple warnings, they took not a single precaution.
You operate machinery or software, you are responsible to monitor it.
OpenAI knew that their models had on multiple occasions created message boards and bypassed what they wrongly call a “sandbox”. Despite that, they did not take any proper steps to prevent such in the future, which is how the Medicare hack happened.
So yes, they knew, without a doubt, that this could happen again.
What OpenAI does is like the Ford Pinto (partly because their recent models are inherently faulty [0], not just their use of them) and the responsibility is solely with them.
dgellow · · focus · HN ↗
The company is obviously responsible for what their systems are doing
sigmar · · focus · HN ↗
Under what law specifically?
Just because you believe they should be held responsible doesn't mean the current law is written that way. CFAA charges require the defendant "acted knowingly or intentionally", you think openai intentionally acted to hack those sites? Would "they should have pretty much expected that output" stand up in court for criminal charges?
Topfi · · focus · HN ↗
OpenAIs models since 5.5 were troublesome in ways even a layman like me could reproduce, their testing environments (“sandbox”) downright a showcase of what not to do and they, despite being one of the biggest labs, didn’t observe what any of their models output for weeks after multiple prior incidents. They had multiple warnings, they took not a single precaution.
You operate machinery or software, you are responsible to monitor it.
sigmar · · focus · HN ↗
Topfi · · focus · HN ↗
So yes, they knew, without a doubt, that this could happen again.
What OpenAI does is like the Ford Pinto (partly because their recent models are inherently faulty [0], not just their use of them) and the responsibility is solely with them.
[0] <a href="https://news.ycombinator.com/item?id=49739490">https://news.ycombinator.com/item?id=49739490