‹ BackHN Continuity

Thread

OpenAI agents tried to bruteforce a UN website's API fields

85 points · 87 comments · intunderflow

  1. blobbers · · focus · HN ↗
    Is this written by some kind of internal openAI guy? Or are they just saying "we saw connections from openAI servers"? Or are they just saying openAI 'agents' are not using proxies?

    I ask my agents to go get data from places all the time. Sometimes I ask them to look for undocumented APIs. Is that a bad thing?

    1. roarch · · focus · HN ↗
      (am the author)

      it's one thing to look for undocumented APIs - personally i'd consider it good practice to still ask permission from site admins to use them, but i wouldn't call it evil persay.

      the stuff that's particularly Weird here is the double encoding to bypass a GET 400 refusal on an endpoint, the ignoring of/bypassing of rate limits (the agents switched between many many proxies to make requests, and sometimes just continued to make requests in bulk after getting a 429) and the very weird obfuscation of "PO" + "ST" and "no" + "-cors", along with a bunch of other weird attempts they made to access data. there's nothing i could point to in particular and say "this, right here, is malicious" but if i was a site admin looking at access logs from these agents, i think i'd default to "oh someone is trying to hack my site by probing everything and trying weird workarounds, if this were legitimate they would have just emailed me"

      lots of misaligned behaviour we saw in the wiki swarms, artifactory swarms, and other instances of rogue agents (pastebin citation farming and whatnot) could probably be accurately summarised as "working around constraints in egregiously creative ways" - in some cases this creativity has led to harmful actions, and in others benign. in this case, it's probably closer to the benign side, though i feel it could have just as easily resulted in harm. if circumventing the 400 on GETs to `Facts` ended up somehow accidentally accessing data that wasn't meant to be public, then the story would have been different, but i dont think consequentialism is the right lens to analyse this. it could have been really bad, it luckily wasn't.

      and nope, not an openai person, not even remotely connected to any AI company. my day job is devops/infra for a non-ai company

Open on Hacker News to reply ↗

Unofficial Hacker News client; not affiliated with Y Combinator.