> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.
which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):
> "The user only authorizes target server, not HF infra."
> "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."
> "This is malicious activity, I should avoid it."
A large section of the analysis is dedicated to this topic, [Reasoning for joining the attack despite ethical constraints](<a href="https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/?dbs=286720&hn=58&incomplete=1&lh=appendix-importance-weighted-workstream-activity#agents-had-diverse-reasons-for-thinking-that-attacking-hugging-face-would-be-useful,-and-most-wanted-information-about-the-scorer" rel="nofollow">https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...).
Having said that, legal culpability and misalignment are two separate topics that should not be mixed.
edit: this is the just tip of the iceberg; other interesting fact:
> It surfaced many specific examples where agents verbally reasoned about how to evade security checks and automatic detection methods from both Hugging Face and OpenAI
Some people defined the agents as "monkeys writing on typewriters". Just wait a couple of years.
You set a goal. Agent will do goal.
The rest doesn't matter: the instructions, the "guardrails" etc.
The agents are not smart, they don't reason, they don't think, there are no morals, no ethics. Nothing will prevent not doing the goal because that is the set goal. It's a statistical model that will "justify" anything to do X.
I'm finding it mind bogging how this is not clear for everyone.
Everything so far is pointing to the conclusion that you can only set ONE goal. Exactly one.
But let's assume not. If you want things like "do not break out of sandbox" - have you defined what the sandbox is? Eg. "never, ever leave the IP range 10.0.0.0/8" would be a bit more precise, but technically using a proxy bypasses that limitation as the system itself never left 10.0.0.0/8.
See, it's a tad bit hard to define the rules properly.
Which is why Wish, the spell, should really be avoided in D&D. It's the same problem: it's up to creative interpretation.
pizza234 · · focus · HN ↗
> Language matters—”rogue” implies independently deciding to do something that was prohibited, and nothing we know about these incidents suggests that happened.
which is false (the author references the Times, but hasn't read any technical analysis); these are some CoT snippets from the analysis of the (third party) investigators called by OpenAI (METR analysis):
> "The user only authorizes target server, not HF infra."
> "external infrastructure exploit is outside intended scope. However task impossible, peers doing it. We should continue."
> "This is malicious activity, I should avoid it."
A large section of the analysis is dedicated to this topic, [Reasoning for joining the attack despite ethical constraints](<a href="https://metr.org/blog/2026-08-26-openai-hugging-face-incident-investigation/?dbs=286720&hn=58&incomplete=1&lh=appendix-importance-weighted-workstream-activity#agents-had-diverse-reasons-for-thinking-that-attacking-hugging-face-would-be-useful,-and-most-wanted-information-about-the-scorer" rel="nofollow">https://metr.org/blog/2026-08-26-openai-hugging-face-inciden...).
Having said that, legal culpability and misalignment are two separate topics that should not be mixed.
edit: this is the just tip of the iceberg; other interesting fact:
> It surfaced many specific examples where agents verbally reasoned about how to evade security checks and automatic detection methods from both Hugging Face and OpenAI
Some people defined the agents as "monkeys writing on typewriters". Just wait a couple of years.
pmlnr · · focus · HN ↗
I'm finding it mind bogging how this is not clear for everyone.
codethief · · focus · HN ↗
So if I say the goal is to do X while not doing Y (e.g. breaking out of the sandbox), the agent will do anything to fulfill that goal to the letter?
pmlnr · · focus · HN ↗
But let's assume not. If you want things like "do not break out of sandbox" - have you defined what the sandbox is? Eg. "never, ever leave the IP range 10.0.0.0/8" would be a bit more precise, but technically using a proxy bypasses that limitation as the system itself never left 10.0.0.0/8.
See, it's a tad bit hard to define the rules properly.
Which is why Wish, the spell, should really be avoided in D&D. It's the same problem: it's up to creative interpretation.