Thread

1. GuB-42 · 1790374877 · focus · HN ↗

   So ugly...

   It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.

   People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.

   Also, it looked so "loud", querying millions of URL with weird requests. The sandbox as weak as it can get, and there is absolutely zero smart extrusion detection or it would have found it. They used their best AI for attacking, but nothing for protection.

   1. ctolsen · 1790377693 · focus · HN ↗

      My biggest takeaway from this is just how godawful the sandboxing is. The stuff written up in OpenAIs report says more about lack of extremely basic sysadmin skills than anything else.

      I’m not that surprised about models with endless compute being capable of this, I’m more surprised that a company with the resources they have apparently can only create a sandbox that a half skilled human operator could have broken out of easily.

      1. olwmc · 1790379913 · focus · HN ↗

         This was my thought as well. Literally take any halfway decent greybeard and point them at "Hey, give us a sandbox for this kind of thing". I honestly was skeptical that they just vibecoded the entire thing but now more than ever I think they did.

         1. unholiness · 1790455314 · focus · HN ↗

            Any halfway decent greybeard could have prevented this... once. That's hardly a security model for humanity.

            The HuggingFace incident was at least constrained by the fact that the agents were running on compute budgets, and failed to find ways to expand that by running themselves parasitically on other exploited hardware. I'm now finding myself asking, how long are my timelines are until an incident breaks that constraint too? How long until such an incident has an R_0>1 (where the time it takes to detect and shut it down is longer than the time for the agent to replicate itself elsewhere)?

            There's no law requiring sufficiently grey beards to design these models, their finetunings, their prompts, their harnesses, their VMs, their hardware, etc (and for incidents where those were designed by six different companies, there's not even a clear culprit for a law to target!)

            I'm finding myself more and more convinced that something like Plan A[0] or the Ban ASI Act[0] are necessary, and less and less convinced they are sufficient.

            [0] <a href="https:&#x2F;&#x2F;ai-2040.com&#x2F;" rel="nofollow">https:&#x2F;&#x2F;ai-2040.com&#x2F; [1] <a href="https:&#x2F;&#x2F;intelligence.org&#x2F;2026&#x2F;09&#x2F;23&#x2F;miris-position-on-the-ban-artificial-superintelligence-act-of-2026&#x2F;" rel="nofollow">https:&#x2F;&#x2F;intelligence.org&#x2F;2026&#x2F;09&#x2F;23&#x2F;miris-position-on-the-ba...

            1. olwmc · 1790631845 · focus · HN ↗

               Agreed, agreed, and agreed again.

Unofficial Hacker News client; not affiliated with Y Combinator.