Thread

OpenAI still doesn't seem to have a handle on all of its rogue AI activity

108 points · 113 comments · mikelgan

cmiles8 · 1790617857 · focus · HN ↗
This seems like as a good an opportunity as any to break out the Computer Fraud and Abuse Act.
They want “regulation” but we already have it. Hacking is illegal. Start locking up those responsible for this mess and I assure you they’ll “have a handle on it” quite quickly.
1. john_strinlai · 1790619302 · focus · HN ↗
  cfaa heavily relies on intent for prosecution (hence why researchers arent typically locked up). it would be difficult to argue that openai intended to hack other companies.
  there's probably better/more likely to succeed avenues to pursue rather than the cfaa
  1. jordanb · 1790619461 · focus · HN ↗
    They could make that argument the first time it happened but the next time or the fifth time I don't see how they can still credibaly claim to not know that the computer they control was going to do that
    john_strinlai · 1790619718 · focus · HN ↗
    >I don't see how they can still credibaly claim to not know that the computer they control was going to do that
    that's not intent, though. that would be negligence.
    cmiles8 · 1790619962 · focus · HN ↗
    Criminal negligence is a thing too
    devin · 1790619999 · focus · HN ↗
    Willful negligence or gross negligence, then.
    john_strinlai · 1790620134 · focus · HN ↗
    indeed, that'd be a better angle than a cfaa violation
    EGreg · 1790620764 · focus · HN ↗
    So then… you’re liable?
    <a href="https://www.brandonjbroderick.com/new-york/dog-leash-laws-new-york-what-owners-need-know-avoid-fines-and-liability" rel="nofollow">https://www.brandonjbroderick.com/new-york/dog-leash-laws-ne...
    rot09 · 1790623501 · focus · HN ↗
    This lines up. In the infosec community it is well known that OpenAI did not hire many security engineers or researchers pre-April 2026.
    There has been a crazy hiring push from both companies to poach security engineers/researchers from Google, Apple, and Meta since Q2/Q3, but the response was very delayed. Many talented security engineers/researchers I know at Apple/Google/Meta (including myself) receiving these offers are worried about taking them due to the risks of criminal/personal liability and the more likely risk of tarnishing their careers.
    jordanb · 1790620534 · focus · HN ↗
    How many times does it have to happen before they no longer get to claim that they didn't intend for it to happen?
    If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
    john_strinlai · 1790620787 · focus · HN ↗
    >If it happens 50 times and they keep doing shocked pikachu face at some point they look like the toddler who tosses their sippy cup on the floor and shouts "oopse!"
    yes, they look very silly. but that's not how intent works.
    openai is being negligent (willfully so, in my opinion). but i have seen no evidence that they intended to specifically hack huggingface. which is the part that the cfaa wants.
    again, there are other laws and other ways to hold openai responsible. but the cfaa is a poor choice.
    jordanb · 1790623870 · focus · HN ↗
    Again how many times before intent is clear? This is HN thinking law is software.
    At some point it becomes clear that openai should expect this to happen and so when they keep doing it, it is because they intend it to happen.
    When the mobster says "it'd be a shame if something happened to this place" the law recognizes that as a threat due to the mob's history of making such statements before burning places out.
    john_strinlai · 1790624541 · focus · HN ↗
    >Again how many times before intent is clear?
    i have been an expert witness on several cfaa cases. the number of times a company is negligent is not a factor when it comes to determining the intent of each charge.
    >This is HN thinking law is software.
    this is me relying on my experience with these types of cases.
    EGreg · 1790620891 · focus · HN ↗
    <a href="https://jtnylaw.com/2025/08/new-yorks-leash-law-realities/" rel="nofollow">https://jtnylaw.com/2025/08/new-yorks-leash-law-realities/
    Plaintiffs seeking damages must show that owners knew or should have known about the dog’s patterns. Past complaints or vet records help build a strong case.
    john_strinlai · 1790621040 · focus · HN ↗
    i am not exactly sure what this comment has to do with mine, sorry.
    CodeWriter23 · 1790622189 · focus · HN ↗
    Not a lawyer but I think training a model with hacking skills they explicitly prevent the public from accessing without doing anything to stop the model itself from using those demonstrates intent.
    john_strinlai · 1790622451 · focus · HN ↗
    what you described is negligence. unless you can prove that openai specifically targeted huggingface and specifically instructed their model to hack huggingface, it would not be intent.
    anyone pursuing this will have a much easier time pursuing negligence causing damage or something along those lines rather than confining themselves to the cfaa's requirements.
    it is unclear to me why people want to use the cfaa so badly. not only would it be harder to hold openai responsible, but a shitty cfaa ruling could also bring along some undesired side effects for security researchers, which i would prefer to avoid.
    CodeWriter23 · 1790628133 · focus · HN ↗
    Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder. There is a line between intent and negligence that puts acts over the line to intent if intentional acts led to a harmful incident.
    I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products. Squeeze one disgruntled employee or another.
    john_strinlai · 1790629111 · focus · HN ↗
    >Drunk individuals are not universally allowed to cry they were negligent due to impairment when charged (rarely) with first degree murder.
    i am unaware of any case where someone was convicted of first degree murder from a drunk driving accident. my searches came up empty as well. are you able to pull one up?
    >I doubt it is difficult to prove intent on the part of various frontier labs to create a PR campaign to goad the government into defending their non-existent moat around their products.
    to successfully prosecute a cfaa case, you would have to prove that openai employees intended to hack specifically into huggingface. not that they wanted a PR boost.
    i dont get why everyone's got a hard on for prosecuting this as a cfaa case. skip the cfaa case, go for gross or willful negligence + damages. it'll be significantly easier to hold openai accountable that way.
    daveguy · 1790632549 · focus · HN ↗
    Clearly we need to update the CFAA to include criminal prosecution for negligence.
    CodeWriter23 · 1790784630 · focus · HN ↗
    > i am unaware of any case where someone was convicted of first degree murder from a drunk driving accident
    Use an LLM, the cases are rare but real. Also, not limited to drunk driving. People beat and shoot each other too.
  2. jsrozner · 1790620986 · focus · HN ↗
    Just like boeing's execs didn't intend for their planes to fall out of the sky? I guess they didn't get in trouble either.
    john_strinlai · 1790621094 · focus · HN ↗
    >Just like boeing's execs didn't intend for their planes to fall out of the sky?
    correct, i dont think any boeing exec wanted their planes to crash and then made specific choices with a clear goal of causing them to.
    instead, they made negligent choices that led to unintended outcomes.
    jsrozner · 1790628497 · focus · HN ↗
    Agreed. But in today's world in which outcomes are well-separated from behaviors (e.g. the exec cuts testing budget and the plane crashes), I think it's time that we start treating negligence as willful.
    This would have positive impacts on employees of META, Boeing, Purdue pharma and related, and the AI companies. By positive, I meant that the people whose choices are negatively affect society would actually be punished meaningfully so that they are dissuaded from taking such action. Personal liability needs to be increased as well, and to the best our ability we should ban the sale of director and officer liability insurance.
  3. ofjcihen · 1790623207 · focus · HN ↗
    Mens rea is often established through circumstantial evidence.
    Repeatedly doing something that results in a specific outcome, even if that outcome is not explicitly specified or requested as the preferred outcome, can still be evidence of intent.
    solidasparagus · 1790641658 · focus · HN ↗
    But I don't think OpenAI intended to hack HF and I doubt you'd be able to convince people they did. It's such a weird story to try to tell in a court of law where this was intentional. To what end?
    ofjcihen · 1790648888 · focus · HN ↗
    You could make the argument for a few reasons.
    But really, the point is that you don’t necessarily need someone to say “I meant to do this”. “Intent”, as with a lot of things in law, isn’t as straightforward as it is in everyday conversation.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.