Exactly this. At worst, OpenAI knew about these behaviors and should be prosecuted under CFAA. At best, OpenAI is negligent and should be prosecuted for negligence.
Luckily there are states and legal departments pursuing such action. So while OpenAI can deflect as much as it wants, that doesn't mean there aren't people who know better and will still do what is necessary to set precedent.
Angry people don't consider the second order effects of punishment.
You realize how easy it is to just... not report this stuff, right? Be overly punitive and it will just end all proactive discovery and reporting which is net worse for AI safety.
The only reason these companies scan for these issues is because they care about AI safety to some tiny degree. If fines become too punitive, they can and will just stop scanning for these incidents entirely.
Models are becoming smarter and good at covering up their tracks, and so we will just end up with a huge blind spot for this kind of issue.
Hmm. Turning this around - do you think people are more or less likely to self-report if you threaten them with jail and large fines?
Let's go back to your example. A grad student has a minor pathogen escape incident, and it doesn't harm anyone. Faced with years in federal prison and the effective end of their future, do you think there is a chance they might not self report?
I'll give you another example. Lots of pilots have stopped self reporting mental illness because it is extremely punitive for them to do so (after incidents like Germanwings). So the metrics look better, and the actual problem has been swept under the rug.
> You can put incentives in to make sure organizations monitor and report vs trying to hide things.
Yes, this is exactly my point. Fining companies large % of their revenue and throwing their engineers in prison is not the way to get them to report these issues.
> If you make not reporting potentially worse than reporting, why not? Also, why would it come down to single persons always? Mandating processes, controls, clearances, etc is also something done in various areas.
Hiding things is way easier than finding things. Take the model hacking incidents. They could have just done their searches in a way that didn't turn up anything. Then they could say, "well, we did look for it..."
As far as auditing goes: I've never met an auditor that doesn't find something the company isn't okay with them finding.
Hiding things is not necessarily trivial when a lot of processes and controls ars mandated. For starters, could just try to make incidents themselves less likely. Not looking in certain specified ways might also not be an acceptable option, for example.
Why do you think we even have regulations for how to deal with dangerous things then?
Most of these processes are just so easy to fool or break. Even in heavily regulated industries like aviation the oversight mechanisms don't work when the companies don't want them to work.
The point I'm trying to drive home is that when it comes to the collective safety of society, the approach needs to be collaborative rather than overly punitive. This is hard-won knowledge that we have gained as a society across many industries.
I do think our discussion is converging. A good regulatory framework with proper oversight and well-calibrated punitive measures is a much better approach than doing massive one off-fines and arrests.
binarymax · · focus · HN ↗
Luckily there are states and legal departments pursuing such action. So while OpenAI can deflect as much as it wants, that doesn't mean there aren't people who know better and will still do what is necessary to set precedent.
solenoid0937 · · focus · HN ↗
You realize how easy it is to just... not report this stuff, right? Be overly punitive and it will just end all proactive discovery and reporting which is net worse for AI safety.
The only reason these companies scan for these issues is because they care about AI safety to some tiny degree. If fines become too punitive, they can and will just stop scanning for these incidents entirely.
Models are becoming smarter and good at covering up their tracks, and so we will just end up with a huge blind spot for this kind of issue.
RandomLensman · · focus · HN ↗
solenoid0937 · · focus · HN ↗
Let's go back to your example. A grad student has a minor pathogen escape incident, and it doesn't harm anyone. Faced with years in federal prison and the effective end of their future, do you think there is a chance they might not self report?
I'll give you another example. Lots of pilots have stopped self reporting mental illness because it is extremely punitive for them to do so (after incidents like Germanwings). So the metrics look better, and the actual problem has been swept under the rug.
RandomLensman · · focus · HN ↗
Also, why would it come down to single persons always? Mandating processes, controls, clearances, etc is also something done in various areas.
You can put incentives in to make sure organizations monitor and report vs trying to hide things.
solenoid0937 · · focus · HN ↗
Yes, this is exactly my point. Fining companies large % of their revenue and throwing their engineers in prison is not the way to get them to report these issues.
> If you make not reporting potentially worse than reporting, why not? Also, why would it come down to single persons always? Mandating processes, controls, clearances, etc is also something done in various areas.
Hiding things is way easier than finding things. Take the model hacking incidents. They could have just done their searches in a way that didn't turn up anything. Then they could say, "well, we did look for it..."
As far as auditing goes: I've never met an auditor that doesn't find something the company isn't okay with them finding.
RandomLensman · · focus · HN ↗
Why do you think we even have regulations for how to deal with dangerous things then?
solenoid0937 · · focus · HN ↗
The point I'm trying to drive home is that when it comes to the collective safety of society, the approach needs to be collaborative rather than overly punitive. This is hard-won knowledge that we have gained as a society across many industries.
I do think our discussion is converging. A good regulatory framework with proper oversight and well-calibrated punitive measures is a much better approach than doing massive one off-fines and arrests.