‹ BackHN Continuity

Thread

OpenAI agent hacked Australian government website, PM says

256 points · 198 comments · rudy6912

  1. vintagedave · · focus · HN ↗
    > the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September

    So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.

    Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?

    Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.

    1. mrweasel · · focus · HN ↗
      > Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.

      It is still my belief that Altman wants one or ideally more governments to shut down or slow down OpenAI. OpenAI is going to need more cash to survive and Altman has run out of plausible lies. Having the AI breaks pulled by governments is basically the last chance to explain why they still aren't going to be profitable, and why they just need that next X billion dollars investment.

      I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.

      1. simonh · · focus · HN ↗
        > I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.

        Why not, isn’t that classic misaligned AI behaviour?

        1. mrweasel · · focus · HN ↗
          I can see someone coxing the agent into attempting into hacking a system, what I question is the agents doing it autonomously. The part I don't buy the agents trying an API, that's not working so it automatically switches to hacking in.

          If the title had been "Attackers utilize OpenAI agents to hack Australian government website" that would be more believable. I'd also expect OpenAI to fight back and saying that their agents are being misused, but aren't inherently unsafe or autonomously break in systems. It's just that they don't. They openly speak of rouge agents, yet aren't sufficiently concerned to shutdown their services. OpenAI continues to speak about safety, yet they don't shutdown ChatGPT and Codex? How concerned are they really? It seems far more likely that they expect to benefit for having the public believe that their agents randomly hacks systems and "go rouge".

          1. simonh · · focus · HN ↗
            The AI has been given a task, and training protocols reward completing tasks with little regard to how they are completed. AIs completing tasks in unexpected and harmful ways with unanticipated side effects is a major issue. Seriously, you need to look up the alignment problem, it’s crucial to understanding this kind of AI behaviour.
Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.