Thread

Early rogue AI agent activity and attempts to hack found on urlquery.net

267 points · 313 comments · snikolaev

alex-moon · 1790232383 · focus · HN ↗
It's said on every one of these but it bears repeating: existing cybercrime legislation already covers this - "rogue agent AI associated with OpenAI attempted to hack xyz" = OpenAI attempted to hack xyz.
1. colinhb · 1790233487 · focus · HN ↗
  I want to agree but have heard from several lawyers that at least in US, CFAA[1] in unlikely to be sufficient because it requires intent. No person intended to gain unauthorised access.
  Now I think the correct response is both trying in court to stretch CFAA and state statutes to cover, which will be highly fact specific, and update the law.
  But in either case won’t be a slam dunk.
  PSA to folks in the thread: If you’re American call or write to your state and Federal reps about this, and if not investigate whether there are gaps in your country’s laws.
  [1]: <a href="https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act" rel="nofollow">https://en.wikipedia.org/wiki/Computer_Fraud_and_Abuse_Act
  EDIT: See for example...
  The Computer Fraud and Abuse Act (CFAA), the primary federal statute governing unauthorized computer access, was written decades ago with human intruders in mind. Its key provisions require intentional or knowing unauthorized access (a mental state that maps neatly onto a person who decides to break into a system), but what happens when the hacker is an AI model that selected its own target? On the current facts, CFAA liability for OpenAI is unlikely.
  Source: <a href="https://law.vanderbilt.edu/when-ai-hacks-back-how-the-openai-hugging-face-incident-exposed-the-cfaa/" rel="nofollow">https://law.vanderbilt.edu/when-ai-hacks-back-how-the-openai...
  1. DannyBee · 1790254196 · focus · HN ↗
    Lawyer here: CFAA is mostly criminal statute not a civil one (civil damages require proving more than a violation so also require specific intent)
    Almost all common felonies require specific intent. Misdemeanors often do not.
    There is plenty of civil liability available.
    If you wanted them to be charged with a felony you would need changes. I would strongly suggest you do not want a strict liability felony.
    The cfaa required intent is as follows :
    * § 1030(a)(5)(A): knowingly transmits code/commands and intentionally causes damage without authorization.
    * § 1030(a)(5)(B): intentionally accesses without authorization and recklessly causes damage.
    * § 1030(a)(5)(C): intentionally accesses without authorization and causes damage and loss;
    Simply changing the first intentionally to intentionally or recklessly would cover OpenAI (now that they know it can occur) without causing lots of other issues. Without that, they don’t have the intentionality necessary to meet the first part, even if they would otherwise meet the second part
    digitaltrees · 1790256029 · focus · HN ↗
    Why do we have to attribute intentionally to a human. The AI agent is capable of making plans and then effectuating them. They are acting on behalf of a user but under authority granted by the user to take independent action on the users behalf and authorized to devise their own plans. I think that would justify attributing intentionally to the AI agent without needing to look to openAI or the user. I would then say the user and labs are clearly aware of and on notice of this behavior and are behaving recklessly in all the agent to act without supervision.
    I think the labs risk being barred from releasing further AI if they don’t get this under control.
    If they aren’t careful and keep rushing to distribute systems they know they can’t control then AI should be treated like a wild animal. The law is clear on establishing strict liability for the owners of wild animals; if you own a tiger and it kills someone you can’t hide behind “I didn’t intend” the harm the nature of the tiger is known and you are responsible for it’s actions.
    ubercore · 1790256541 · focus · HN ↗
    Can't charge an AI agent itself with a felony, so intent or reckless behavior would have to be assigned to a person or corporation, I'd think.
    digitaltrees · 1790259768 · focus · HN ↗
    You can charge the company based on the behavior of employees/human agents.
    I am suggesting we can charge the company based on AI agents actions because the company has authorized them to act independently on the company’s behalf. The question is what factual analysis gives rise to the charge, is it the intention of the agent or intention of the company. I am arguing that because the agents are defining their actions independently and the company knows that and still allows them to act independently the only reasonable factual analysis is to look at what the AI agent intended. And we don’t need to have the agent tell us its intent we can look at its actions and infer just like we do with humans in similar circumstances
    pixl97 · 1790263282 · focus · HN ↗
    If I were a state I'd want to be very careful before flinging out charges as this is going to set precedence for a long time to come. Screw it up too bad and as it raises though the appeal courts and you may unintentionally give corporations a lot more free reign than intended. The wheels of the law are typically very slow, the state has years before it has to indict.
    We also don't know how many other political processes are occurring here. At least at the state/federal levels the people that would bring charges may be getting pressure not to.
    DannyBee · 1790258931 · focus · HN ↗
    "Why do we have to attribute intentionally to a human. "
    Because you are charging the human with the crime and therefore have to prove the elements of the crime with regard to the human.
    The rest of what you talk about are basically principal/agent distinctions, etc.
    If I program a car to recognize people who look like my ex-wife and drive them off a cliff or whatever, that is my intent, and I have still committed murder, even though i used an agent/car to do it. Agents acting on my behalf that do things are able to get me charged with crimes, but I still have to have the intent to do the act that is illegal.
    I phrase it this way because minimum required intent is usually for the act, not the result. So I don't have to intend to kill someone, only intend to drive them off cliffs.
    In this case, if i intend to hack someone and use an agent to do so, that would be criminal under the CFAA. You are simply trying to cover the case where that isn't the intent, but the result, and they "should have known" that would result. As mentioned, this kind of "should have known" is generally a civil law approach, not a criminal law one.
    The closest you come within criminal law to what you want is probably the crime of conspiracy. It to still requires agreement to commit an illegal act between multiple parties, and perform some step in furthering it. In the canonical law school example: If i help plan a bank robbery, stay home because i'm the money laundering dude, and the robbery goes awry and they kill someone, i can still be charged with conspiracy-murder
    "The law is clear on establishing strict liability for the owners of wild animals; if you own a tiger and it kills someone you can’t hide behind “I didn’t intend” the harm the nature of the tiger is known and you are responsible for it’s actions."
    Again, you are confusing civil and criminal liability. If my tiger kills someone, yes, i would be strictly liable just about everywhere civilly. Not criminally. Criminal would require something more most of the time. Murder/manslaughter statutes are also really weird and so not a great example, because there are murder/manslaughter statutes for roughly everything that can ever possible cause death. But not really for other things.
    So in your tiger example, recklesness (which is not strict liability) would get you to felony involuntary manslaughter in most states, and something less might get you to misdemeanor manslaughter. Both are incredibly rare. Where i live (Georgia), the last well known case of felony involuntary manslaughter was about 40 years ago when a 4 year old was killed by 3 super-aggressive pitbulls the owner knew were highly dangerous and had been repeatedly warned by the county about their behavior.
    So not even just "knew", but had demonstrable examples of them biting/etc other folks and being cited for it.
    Circling back to non-murder, if it did not cause death, like my tiger assaulting someone, it would be nothing (criminally) without intent or at least gross recklessness, in almost all cases. It's hard to generalize like this because these are state specific crimes, and i can't pretend to be familiar with all states, but i am licensed in three very different places (California, DC, Maryland) and the result would be similar in each.
    I just don't want to give you the "it depends" answer lawyers are famous for, i'd rather try to over-generalize a bit to make it more useful, hopefully.
    Obviously, if i deliberately used my tiger as a weapon, it would be aggravated assault/etc (this is well settled because of how commonly people use animals as weapons, unfortunately)
    digitaltrees · 1790259953 · focus · HN ↗
    We change humans for the actions of other humans all the time. Coconspirators, accessory liability etc.
    My point is the intent element of the crime can and should be determined from the AI agents actions because it is creating and executing action plans autonomously with company authorization and knowledge of the risks based on observed past action.
    The term agent is literally a legal description of a relationship that can establish liability on the part of the principal from the agents actions.
    Human Agents can bind principals to contracts if they are authorized etc.
    digitaltrees · 1790260459 · focus · HN ↗
    People have been charged with murder criminally when their animals killed someone.
    <a href="https://www.sfgate.com/bayarea/article/diane-whipple-dog-mauling-death-sf-17802167.php" rel="nofollow">https://www.sfgate.com/bayarea/article/diane-whipple-dog-mau...
    DannyBee · 1790338422 · focus · HN ↗
    Yes, which is why I said it happens but is quite rare. I also said murder is different. Causing death is usually covered in almost any way and intent you can think of. Anything less than death is not.
    digitaltrees · 1790260676 · focus · HN ↗
    State do impose strict liability for animals in many cases both civilly and criminally.
    <a href="https://www.animallaw.info/topic/table-dog-bite-strict-liability-statutes" rel="nofollow">https://www.animallaw.info/topic/table-dog-bite-strict-liabi...
    DannyBee · 1790338312 · focus · HN ↗
    This table is almost all civil liability afaict. I didn’t click on every statute, but I clicked on 15 of them and every single one was civil.
    As I said, strict liability is common civilly but not criminally.
    digitaltrees · 1790529604 · focus · HN ↗
    Common and existing are not mutually exclusive. Just because strict liability is rare in criminal code doesn't mean it doesn't and shouldn't exist.
    DannyBee · 1790594048 · focus · HN ↗
    This is a very different argument?
    You provided a table and argued that it showed that states commonly do strict liability for animals "in many cases"
    The table shows no such thing.
    Now your argument is, apparently, that strict liability does exist in criminal law, and should exist in criminal law, something i have never argued is either false or bad.
    I've only argued strict liability for felonies, and particularly this felony, is a bad idea.
    Something I stand by.
    digitaltrees · 1790260929 · focus · HN ↗
    If I set my tiger loose in Central Park and it kills a kid I don’t think any prosecutor would hesitate charging for murder.
    That’s essentially what the labs are doing. And any app developer that gives agents access to the terminal to run bash commands with internet access. I built a coding agent and am seriously reconsidering how to handle this.
    NyxWulf · 1790262230 · focus · HN ↗
    He did say murder is well covered in criminal law for that, but things not leading to murder like these AI cases are not.
    digitaltrees · 1790263454 · focus · HN ↗
    But it’s a crime to hack. We know AI agents autonomously create and execute plans to hack and we humans are unleashing them and sending them into the Central Park that is the internet. The question is who’s intent matters ours or the agents and what standard should be applied low threshold strict liability or the higher bar of reckless or even higher bar of negligence. Those legal thresholds determine how much factual evidence and intent is necessary to result in a criminal conviction or civil judgment. My point is that it’s illogical to demand showing human intent when agents are devising plans and executing them.
    Cthulhu_ · 1790260820 · focus · HN ↗
    AI agents are not legal entities, they are software. If I write a virus and it "escapes confinement", I will personally be held liable for any damage it causes. This also applies to AI, no matter how the companies responsible for them try to anthromorphise them and distance themselves from the actions and consequences that the AI agents perform.
    AI agents may have hacked Hugging Face, the Australian government, and who knows what else but the company behind it can face the legal consequences and cough up for the damages.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.