Thread

OpenAI agent hacked Australian government website, PM says

256 points · 198 comments · rudy6912

port3000 · 1790248514 · focus · HN ↗
If a bull escapes a field and causes damage in the village, the farmer pays for the damages and is liable. It's been like that for hundreds of years and I don't see how this is any different?
1. z3c0 · 1790248769 · focus · HN ↗
  Decades worth of hackers missed the opportunity to say "It wasn't me -- my computer did it."
  1. jacquesm · 1790252251 · focus · HN ↗
    I've head the 'the hacker did it' excuse from lots of companies that messed up themselves but did not want to admit it.
    wat10000 · 1790263523 · focus · HN ↗
    I got banned from school computers and nearly expelled from high school (and threatened with “blacklisting,” lol) for “hacking” after a virus got into their network. Took them a month to figure out what really happened. Or at least, a month to come clean about it.
  2. tokai · 1790262556 · focus · HN ↗
    Gottfrid Svartholm tried that defense but he still ended up spending three years in prison.
2. graemep · 1790250189 · focus · HN ↗
  Not necessarily
  <a href="https://www.farrer.co.uk/news-and-insights/shut-that-gate-shut-that-bull/" rel="nofollow">https://www.farrer.co.uk/news-and-insights/shut-that-gate-sh...
  1. preommr · 1790255509 · focus · HN ↗
    > "Not necessarily"
    Why did you even link that article when it doesn't help your point?
    I supports the idea that the person responsible or an animal is held liable - it just makes clarificaitons on common sense caveates like when a professional is moving the animal. It even doubles down on making it clear that expected behavior of an animal is taken into account even if unlikely, like how ai swarms have a reasonable potential to just go awry and commit cyber crimes.
    graemep · 1790258330 · focus · HN ↗
    The point is that it is not as simple as "its yours so you are liable for damage", and the article makes it clear. None of the law specific to animals applies to AI, so you need to show negligence. You could bring in legislation to treat AI in a similar way to a dangerous species, but that would be more like keeping a lion than keeping a bull.
    SpicyLemonZest · 1790260595 · focus · HN ↗
    I think it's pretty clearly negligent to set up a harness that will run whatever outputs it receives from your LLM if you cannot reliably stop it from outputting hacking instructions. I've said in the past that I'm sympathetic to the idea of testing your anti-hacking controls, but that doesn't seem to be where this incident came from.
    preommr · 1790271761 · focus · HN ↗
    From the article (that I am sure you read):
    > In Mirvahedy v Henley, the claimant was injured after the car he was driving was hit by the defendant’s horse. The horse had pushed over a wooden fence and an electric wire fence and bolted from its field after being frightened. This test was met here because, when a horse is sufficiently alarmed or panicked – ie in those particular circumstances – it is a known characteristic of a horse to bolt and cover long distances.
    So, it's not just 'lions', but regular animals that people know can behave in a particular way.
    I would say AI companies know the cybersecurity dangers since they literally advertise it as part of their marketing.
    ---
    And as a tangent, it isn't simple if we're talking broadly - e.g. I don't think Anthropic should be liable for agents that are directed by a user to write code that takes part in a crime. Nor should they be held liable if their internal security measures are lax in a way that an employee steals data that gets distilled into another model. But we're not talking about those edge cases.
3. pluc · 1790252315 · focus · HN ↗
  They've largely avoided compensating for everything they've stolen to build their technology upon; these people know that they will never face consequences for their actions. Ask for forgiveness, not permission.
  1. jstanley · 1790260121 · focus · HN ↗
    What did they steal? Did anybody thereby lose anything?
    physicallyIllfr · 1790262442 · focus · HN ↗
    Im sure they didnt discover who it was and what they did for free.
    simonh · 1790268206 · focus · HN ↗
    AI is being used to replace artists. They are being trained on copies of work by artists without being licensed for that use. So yes, artists are losing out.
    kjshsh123 · 1790283553 · focus · HN ↗
    The use of copyrighted material is unethical, I agree, but I don't think it follows that we should consider the replacement of work with technology itself unethical.
    simonh · 1790298220 · focus · HN ↗
    Sure. That seems like a non sequitur. Maybe some people are making such arguments in relation to AI, but this thread is about a particular unethical practice.
4. Mattrou · 1790255414 · focus · HN ↗
  What damages were caused here that would need reparation exactly?
  1. Zone3513 · 1790261585 · focus · HN ↗
    If you hack a government website and access confidential information but don't do anything with it ("no harm") you're still going to prison. Good luck arguing in court the no harm no foul defense.
    phoghed · 1790262856 · focus · HN ↗
    > If you hack a government website and access confidential information but don't do anything with it ("no harm") you're still going to prison.
    You could potentially go to prison. Individuals and companies face different sets of consequences though.
    The other side of the coin is that the government won’t suffer any consequences for having shitty security either.
    ozozozd · 1790265257 · focus · HN ↗
    I rarely notice usernames on HN, but after reading this comment it was so easy to guess that the classy contribution on another comment about legal liability was also yours.
    And you are right again! The Australian government must compensate OpenAI for having shitty security.
    It must be a bliss to transact with you in real life!
    phoghed · 1790265574 · focus · HN ↗
    Honestly I’d expect my government to secure sensitive data to a higher standard than I’d expect a random company motivated only by profit to. We should have an expectation of both sides to do better. Do you disagree?
    simonh · 1790267952 · focus · HN ↗
    You are correct, but the regulatory and liability issue regarding AI is one issue and the government’s liability for having insecure systems is another.
    They are independent concerns. A discussion about the first issue is not enhanced by the interjection of the second.
    phoghed · 1790268590 · focus · HN ↗
    You can discuss two things at once. Especially if they are inherently related. Australia fining OpenAI a large amount of money and using that to audit and harden their systems would be a good outcome here, for example.
    Too many people are making inane fantasy level comments that Sam and his employees should be jailed, and conspiracy theories that he did it on purpose to get regulated because he’s out of runway. That’s boring and repeated ad nauseam every time the topic comes up.
    simonh · 1790298032 · focus · HN ↗
    I agree, but switching to a different subject doesn’t strengthen that argument.
  2. ambicapter · 1790268107 · focus · HN ↗
    What damages are caused at the moment in time a human downloads a file they weren't supposed to have access to? That's apparently enough to send people to jail for a long time.
5. fg137 · 1790261262 · focus · HN ↗
  Farmers doesn't move fast and break things.
  /s
  1. [deleted] · 1790262512 · focus · HN ↗
    [deleted]
6. whalabi · 1790262725 · focus · HN ↗
  In theory, if Altman or Musk want some highly confidential data for their models, they could set a swarm free at it then claim the agents were operating without authorization

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.