OpenAI agent hacked Australian government website, PM says
Thread
Unofficial Hacker News client; not affiliated with Y Combinator.
OpenAI agent hacked Australian government website, PM says
Unofficial Hacker News client; not affiliated with Y Combinator.
vintagedave · · focus · HN ↗
So we have a company hacking a foreign government's websites and data. And, in terms of ethics, they take almost three months to notify; and in terms of competence, appear to have no formal contacts nor to have found one in that time.
Once an American business starts hacking allied governments, it's time for strict responses, yes? Replace the governance (board and C-level)? Remove financial incentives and open the company - open weights, open training, per its original 'open' ethos?
Altman is busy saying there needs to be regulation, but in terms of what OpenAI does, he can control that already.
mrweasel · · focus · HN ↗
It is still my belief that Altman wants one or ideally more governments to shut down or slow down OpenAI. OpenAI is going to need more cash to survive and Altman has run out of plausible lies. Having the AI breaks pulled by governments is basically the last chance to explain why they still aren't going to be profitable, and why they just need that next X billion dollars investment.
I don't for a second believe that an agent starts trying to hack backend system, when the form or API it has been asked to use isn't working.
macNchz · · focus · HN ↗
I have seen coding agents on my own machine (in sandboxed VMs) start doing things while trying to accomplish what I've asked that I felt sort of exceeded my mandate (changing database passwords, poking at the egress proxy that's preventing them from accessing some domains). Not to the point of causing any real issues, but I don't have much trouble envisioning scenarios like this when using stronger instructions around pursuing the goal + a running in a misconfigured sandbox envrionment.
That said, there's a lot of potential upside for American AI labs if they're able to get people scared about AI, they can:
- To your point, claim the regulations slowed them down and paper over near/mid term financial concerns
- Get the government to create stupid regulations that don't actually slow them down at all, but do effectively lock out any future competition (and current global competition)
- Position themselves as the only organizations blessed by the government with the ability to make safe AI, therefore eventually allowing them to claim to be some flavor of "too big to fail" and worthy of a bailout, should the financials not work out.
- Effectively create a distraction that avoids further public conversation/accountability/regulation/liability re the more tangible sorts of problems their products cause right now.
disgruntledphd2 · · focus · HN ↗
Yeah, when I was using Claude Code some months back, I was building something that needed LLM calls in the frontend. Sonnet decided that the best way to accomplish its goal (get the tests passing) was to start grepping around my filesystem looking for my API keys.
One of the few cases where I've ever used the memory system, I told it to never do that under any circumstances. And then it did it again, a few days later.
RL is definitely an issue here, the models are getting trained based on task completion which leads to madness like this.
hishboy · · focus · HN ↗
[dead]