Thread

1. vintagedave · 1790241637 · focus · HN ↗

   > the breach took place on 18 June - Open AI informed the government with an email to a general address on 10 September

   So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.

   Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?

   Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.

   1. ben_w · 1790242664 · focus · HN ↗

      > And, in terms of ethics, they take almost three months to notify;

      Kinda worse than that. It took between 10 and 40 days, not 3 months, between the organisation knowing and the reporting.

        August (precise date unknown) – OpenAI said it became aware of a potential breach during a broader review of "misaligned model activity"
      
        10 September – An email from OpenAI lands in the public inbox of Services Australia, the general services hub of the federal government, informing of the incident
      

      - <a href="https:&#x2F;&#x2F;www.bbc.com&#x2F;news&#x2F;live&#x2F;cvgl73pxgndwt?post=asset%3A69668a7f-8199-4515-acdd-a9de6a2e6b7c#post" rel="nofollow">https:&#x2F;&#x2F;www.bbc.com&#x2F;news&#x2F;live&#x2F;cvgl73pxgndwt?post=asset%3A696...

      &gt; open weights, open training

      Given it was the AI agents which did the hacking, doing this will result in basically every organisation at least as rich as the government of Tuvalu being able to hack anyone at any time.

      &gt; Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.

      Him having control would be an improvement on the reality.

      1. ozgung · 1790246565 · focus · HN ↗

         This was a just case of: (owner of the agents detected the hack) &amp;&amp; !(hacked party didn’t detect the hack) &amp;&amp; (owner of the agents decided to notice the other party) &amp;&amp; (they decided to went public with what happened so we know it)

         One can find many other logical combinations that we can’t possibly know about such incidents.

         1. jacquesm · 1790252202 · focus · HN ↗

            So, you&#x27;re telling me they didn&#x27;t have any monitoring in place around their AI to notify them of an attempt at breaching a system they have no business visiting in the first place? OpenAI should be blackholed on this basis until they clean up their act.

            1. ben_w · 1790253171 · focus · HN ↗

               They must have had monitoring in order to be able to detect this retrospectively.

               Any automated alarms for detecting things in real-time were not sufficient.

               Given a previous generation of agents discovered a zero-day and used it to get around attempts to sandbox them into one specific test, this is not hugely surprising, but it is a reason to force them (and everyone else) to stop until security catches up with capabilities.

               I&#x27;m thinking of the Jurassic Park novel: they had sensors to count the dinosaurs, but the test was made under the assumption escapes were possible and breeding was not, i.e. something like &quot;if (dinosaurs_found &lt; n) then escape_alert();&quot;. They didn&#x27;t know dinosaurs_found &gt;&gt; n until everything was already going wrong.

               1. jacquesm · 1790327739 · focus · HN ↗

                  No, they must have had logging, not monitoring. That&#x27;s a world of a difference, the one is useful after the fact to figure out what is going on the other would give you a reason to pull the emergency brake while it was happening.

Unofficial Hacker News client; not affiliated with Y Combinator.