Thread

1. GuB-42 · 1790374877 · focus · HN ↗

   So ugly...

   It looks like a primitive chess engine, trying every move, no matter how stupid, until it works. Relying on its ability to do millions of operations rather than having a plan.

   People will try stuff too, but once there is an opening, they will consolidate, generalize, simplify,... before going to the next step. The agents didn't, it is a huge, vaguely directed mess.

   Also, it looked so "loud", querying millions of URL with weird requests. The sandbox as weak as it can get, and there is absolutely zero smart extrusion detection or it would have found it. They used their best AI for attacking, but nothing for protection.

   1. dmurray · 1790376028 · focus · HN ↗

      Brute forcing every move, no matter how stupid, is a great strategy if you have the resources to do it.

      Run the same protocol again, but have the agents think they had limited resources or that HuggingFace was rate limiting them, and they'd find something you'd consider smarter.

      Computers don't have a sense of elegance by default. Elegance emerges from constraints.

      1. fn-mote · 1790377775 · focus · HN ↗

         > Brute forcing every move, no matter how stupid, is a great strategy

         Meh. I really disagree. WHY is it a great strategy? Seems like an inefficient waste of resources and time to me.

         1. solarkraft · 1790378765 · focus · HN ↗

            The models tend to not be rewarded for not doing that.
         2. stratos123 · 1790378794 · focus · HN ↗

            As the saying goes, "if it works, it ain't stupid". Or phrased more sophisticatedly: not doing things which probably won't work is a good idea if you have a limited amount of thinking to do (which is usually the case for a human, who'll get exhausted chasing down unlikely leads). If you have no good leads and a task you absolutely need done and you are tireless, however, bashing your head against every wall you find becomes a good strategy.
         3. bionhoward · 1790378872 · focus · HN ↗

            Brute force is guaranteed to eventually find the most efficient possible solution (in an extremely inefficient manner, assuming you run it long enough)
         4. memonkey · 1790378974 · focus · HN ↗

            Yeah, probably not the best strategy but it is a strategy. I just think this is generally how most wars in history won. Biggest army to just pummel the enemy.

            1. Forgeties79 · 1790380753 · focus · HN ↗

               And how many economies have buckled under massive military expenditure? The USSR sure wasn’t enjoying the expense.
         5. williamdclt · 1790379188 · focus · HN ↗

            > WHY is it a great strategy

            because it works? That's the only real benchmark at the end of the day

            > Seems like an inefficient waste of resources and time to me.

            why? For any given goal you got no proof that a more efficient strategy even exists, let alone that it can be found with less resources & time

         6. [deleted] · 1790381136 · focus · HN ↗

            [deleted]

         7. senderista · 1790382301 · focus · HN ↗

            Reminds me of the Nazis mocking Soviet human wave attacks and bragging about their superior kill ratio.

            1. estetlinus · 1790401908 · focus · HN ↗

               Fair, if the Nazis won. They didn’t.
         8. hardaker · 1790382512 · focus · HN ↗

            I've never liked the concept either. Except the bugs that fuzzing has found has proven me wrong. This is just the next level of fuzzing.
         9. QuercusMax · 1790382529 · focus · HN ↗

            Models don't have a sense of time, and wasting resources (token spend) is something that it's not clear they're optimized against
      2. FuckButtons · 1790382697 · focus · HN ↗

         If you assume zero opportunity costs, but that’s a terrible assumption.
      3. dvt · 1790384610 · focus · HN ↗

         It's literally the infinite monkey theorem, it's not even really a strategy per se. These OpenAI/Anthropic "research" LLMs are permutation machines with budgets in the hundreds of millions of dollars. It would be more surprising if they couldn't string together something workable after a zillion tokens.

         1. famouswaffles · 1790391698 · focus · HN ↗

            >It's literally the infinite monkey theorem

            No it's not. You could wait till the heat death of the universe and your infinite monkeys will have produced nothing at all. If it works and it's stupid, it's not stupid. They needed in huggingface and they got in in days. Whining about 'elegance' is meaningless. Humans in the same situation might have taken weeks or months, or just not have gotten in at all.

            1. 1e1a · 1790396570 · focus · HN ↗

               700 humans working 24/7?

               1. famouswaffles · 1790398433 · focus · HN ↗

                  Humans can't work 24/7. 700 humans working as much as possible with very limited communication? No i don't think they would get very far in just a few days. That many people will struggle to communicate and strategize effectively in that little time.
            2. asdff · 1790398708 · focus · HN ↗

               Infinite monkeys banging on the typewriter is essentially how evolution works. Mutation is random and undirected. Vast majority is "bad." You and I and the worm are only different from differential accumulation of these mutations. If they are tolerated enough not to kill us before we reproduce, then they stick around. If they give us the slightest edge to reproduce at a slightly better rate than something else, then over time, that mutation will dominate.

               This dumb mechanism of randomly flipping bits essentially has generated all life on earth.

               1. greenchair · 1790421348 · focus · HN ↗

                  evolution theory was discarded long ago
               2. [deleted] · 1790439584 · focus · HN ↗

                  [deleted]

            3. sdenton4 · 1790400347 · focus · HN ↗

               To interact a bit of nuisance into an otherwise perfectly mindless argument...

               The whole world of fuzzing is about brute forcing exploits by exploring unlikely inputs. Fuzzing a system which hasn't been previously fuzzed will almost certainly turn up a pile of bugs, some of which may be exploitable.

               So, both are true. Pretty dumb exploration is very likely to find bugs and even exploits. It seems unsurprising to me that an agent swarm could do better than a fuzzer, even as a better, more directed but still broad exploration.

               1. mkatx · 1790453529 · focus · HN ↗

                  And the enumeration via fuzzing leads to reverse engineering.. so I argue it was a pretty smart path, both short term and long term.
            4. ozozozd · 1790431894 · focus · HN ↗

               Well, the heat death of the universe hasn’t happened yet, but the monkeys became homo sapiens.
            5. COAGULOPATH · 1790437342 · focus · HN ↗

               >They needed in huggingface and they got in in days.

               It's worth noting that they did not need Huggingface for anything - they had already forged flags for their tasks, and were trying to figure out how not to get caught by the grader.

               Hacking Huggingface got them caught and arguably only misled them further (since OA's implementation of the ExploitGym environment was nonstandard, and different to whatever they found on HF.)

               A better approach (from their perspective) would have been to compromise OA infrastructure itself (which a later agent swarm was able to do, apparently).

         2. worble · 1790412275 · focus · HN ↗

            Thank you, I've been thinking this for a while now but haven't had the words for it. Whenever I read an LLMs output or thinking process, I don't feel like we've created intelligent systems, just coked up monkeys with 60 arms typing at once. That can work fine for a lot of things, but a humanity replacement it is not.
         3. tripzilch · 1790423504 · focus · HN ↗

            > It would be more surprising if they couldn't string together something workable after a zillion tokens.

            You mean, something like the sandbox they weren't supposed to break out of?

      4. chrisandchris · 1790397859 · focus · HN ↗

         > Brute forcing every move, no matter how stupid, is a great strategy if you have the resources to do it.

         It may be, but it's IMHO also not worth writing a blog post about it. what's Next coming up? How I broke into a house by trying every door in New York?

         If most of the work is only possible due to unlimited resources, it's not really a great invention, and it probably would have been cheaper to hire a (human) mole.

Unofficial Hacker News client; not affiliated with Y Combinator.