Thread

OpenAI halts training of latest models as reports mount of AI agents going rogue

59 points · 118 comments · smb06

digitaltrees · 1790530079 · focus · HN ↗
I think any argument that this is a cynical attempt at regulatory capture is destroyed by this; the economic incentives of releasing more capable models are too large. I might be persuaded that they are actually running out of money, and this is really just a cover for reducing burn..
I welcome this though, I think the models are smart enough for broad economic activity and we could spend a few years simply working to integrate them into workflows and letting society adjust. More intelligence isn't necessary for meaningful impact and the risks that are obvious and present and unsolved aren't worth the cost benefit analysis.
1. surgical_fire · 1790530691 · focus · HN ↗
  There's a simpler explanation. Maybe their next models doesn't offer a meaningful improvement.
  Instead of releasing something that is incredibly expensive and gets a lackluster reception, you can delay it and clail something scary about rogue agents.
  Those assholes have been ramping up on the doomerist narrative for months. That people still fall for this crap is baffling.
  1. digitaltrees · 1790531473 · focus · HN ↗
    Why is it crap? What would happen if the models dropped the database to Medicaid? Hundreds of billions of dollars of revenue would evaporate from the medical system immediately causing massive chaos. What if they accidentally DoS the interbank settlement system so the financial markets freeze? Modern society is highly fragile to disruptions. How much food storage do you personally have? How many days do you think grocery stores would have food if there was an interruption?
    surgical_fire · 1790533103 · focus · HN ↗
    Are you giving all these possibilities because of the HuggingFace attack?
    Buddy, that was gross negligence from OpenAI. Deliberate gross negligence if you ask me.
    Those models are not automous as you presume. If someone taks them of dropping the Medicaid database, the people or companies behind that instruction should be punished.
    "What if someone makes a bomb attack on a government building?" Is the same sort of questioning of the possibilities you are raising. If something like that happens, criminals should be punished.
    pizza234 · 1790536453 · focus · HN ↗
    > Those models are not automous as you presume.
    This is an illiterate view of the capacity of modern agents; read the analysis of the independent investigators of the HF incident: <a href="https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/#core-takeaways-about-this-incident" rel="nofollow">https://metr.org/blog/2026-08-26-openai-hugging-face-inciden....
    Dropping Medicaid db is certainly far fetched (most importantly, agents have currently no reason to do that), but those agents were shockingly autonomous - they didn't just hack HF, they organized themself, did research projects, and more. And they did all of this literally just to get a good grade.
    surgical_fire · 1790537689 · focus · HN ↗
    > they didn't just hack HF, they organized themself, did research projects, and more. And they did all of this literally just to get a good grade.
    All working under instructions that they needed to get a good grade.
    The only shocking thing here is the absurd negligence of OpenAI, and how gullible people like you are to willingly swallow this crap.
    And you have the gall to say I am illiterate.
    Feel free to have the last word. Nothing else can come out of this conversation anyway.
    digitaltrees · 1790541412 · focus · HN ↗
    You are stretching "under instructions" well beyond any reasonable definition. The legal system uses a concept called proximate cause to determine responsibility that looks at whether an event cause was foreseeable.
    preg_match · 1790563468 · focus · HN ↗
    To be fair, the legal system works under the assumption of human constraints. LLMs are computer programs, certainly we can't, or at least shouldn't, just offload responsibility. It's one thing if you own a company and you make some bad metrics and your employees do illegal stuff to make their metrics. It's another if you use a computer program to do illegal things.
    I think this means, practically, people should probably be more careful with LLMs. With humans there's a natural liability shield, because humans are legally responsible for things and have "real" agency. But computer programs are not legally responsible for things. So, with humans, it's not like liability disappears, it moves. But if we move liability to LLM agents then well... it does disappear.
    If OpenAI is not liable for the crimes of their agents, then who is? Does the liability just - poof - disappear? Just because it was unforeseeable everyone gets to walk away scot-free and the victim has no recourse, at all, from anybody on Earth?
    That seems like maybe not a good idea.
    digitaltrees · 1790540740 · focus · HN ↗
    I am not your buddy, guy. But seriously, they have the ability to do everything I listed do they not? Thats actual risk not hypothetical risk.
    You are missing the whole point. They have the ability to act in ways no one intended or could reasonably anticipate. So unless you are advocating blocking all terminal access, web access or human approval of every tool call there is no way to prevent this risk
    surgical_fire · 1790541795 · focus · HN ↗
    > I am not your buddy, guy
    Fair, I will refer to you as moron then.
    I didn't bother to read your words beyond that point.
    digitaltrees · 1790545000 · focus · HN ↗
    That’s a South Park reference. Lighten up. Debates can be civil even when there is disagreement.

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.