‹ BackHN Continuity

Thread

Early rogue AI agent activity and attempts to hack found on urlquery.net

267 points · 313 comments · snikolaev

  1. alex-moon · · focus · HN ↗
    It's said on every one of these but it bears repeating: existing cybercrime legislation already covers this - "rogue agent AI associated with OpenAI attempted to hack xyz" = OpenAI attempted to hack xyz.
    1. colinhb · · focus · HN ↗
      I want to agree but have heard from several lawyers that at least in US, CFAA[1] in unlikely to be sufficient because it requires intent. No person intended to gain unauthorised access.

      Now I think the correct response is both trying in court to stretch CFAA and state statutes to cover, which will be highly fact specific, and update the law.

      But in either case won’t be a slam dunk.

      PSA to folks in the thread: If you’re American call or write to your state and Federal reps about this, and if not investigate whether there are gaps in your country’s laws.

      [1]: <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Computer_Fraud_and_Abuse_Act" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Computer_Fraud_and_Abuse_Act

      EDIT: See for example...

        The Computer Fraud and Abuse Act (CFAA), the primary federal statute governing unauthorized computer access, was written decades ago with human intruders in mind. Its key provisions require intentional or knowing unauthorized access (a mental state that maps neatly onto a person who decides to break into a system), but what happens when the hacker is an AI model that selected its own target?
  On the current facts, CFAA liability for OpenAI is unlikely.
      Source: <a href="https:&#x2F;&#x2F;law.vanderbilt.edu&#x2F;when-ai-hacks-back-how-the-openai-hugging-face-incident-exposed-the-cfaa&#x2F;" rel="nofollow">https:&#x2F;&#x2F;law.vanderbilt.edu&#x2F;when-ai-hacks-back-how-the-openai...
      1. lelanthran · · focus · HN ↗
        &gt; I want to agree but have heard from several lawyers that at least in US, CFAA[1] in unlikely to be sufficient because it requires intent. No person intended to gain unauthorised access.

        Only in terms of CFAA, not in terms of damages. Culpability does not require intent.

        You may not have intended to attack $CORP, but you can still made to pay the cleanup costs of that attack.

        So, yeah, you won&#x27;t be convicted, but current laws still allow for you to be billed.

        1. gpt5 · · focus · HN ↗
          Which is the correct way to handle this.

          With that said, there is also criminal negligence. Now that OpenAI is made aware of the risks, it&#x27;s also expected to take additional precautions in the future, otherwise there could be criminal liability as well.

          1. Sharlin · · focus · HN ↗
            Just paying some pocket money for cleanup costs is absolutely not enough. And they should’ve know better the whole time, they were absolutely negligent and incompetent, and their stepping up precautions may well turn out to lag behind the models getting even smarter and actually capable of covering their tracks.
            1. lelanthran · · focus · HN ↗
              &gt; Just paying some pocket money for cleanup costs is absolutely not enough.

              It&#x27;s not my first prize, but I won&#x27;t mind it. And millions like me won&#x27;t mind it. Easy way to make money - setup a site with all the default server software installed and patched at a reasonable frequency. Then just wait for bots to attack it, and claim a few hundred (or single-digit thousand) dollars from OpenAI or Anthropic, etc.

              Sure, it&#x27;s pocket change for them, but just the admin of dealing with millions of cases will, even if they win half the time, will bankrupt them. Thus, they have incentive to make sure that their bots are not performing attacks.

              First prize is, of course, holding them liable with punitive fines, not theatrical fines.

              1. Sharlin · · focus · HN ↗
                I’m all for LLM honeypots, but I don’t think there’s nearly enough LLM hacking activity going on for some random honeypot to be found and targeted unless it’s somehow very visible and appears as a high-reward target (&quot;reward&quot; in the sense of RL).
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.