Thread · HN Continuity

hello! i was wondering where all the visits with hackernews referers were coming from. guess i've found it, hello and i'm glad people seem to have found the post informative! just wanted to address some of the comments i've seen in the replies a bit (i've never used this site before so apologies if i'm doing netiquette wrong)

  was this hacking? (similar question: wasn&#x27;t this info all publicly available?) - for this one i&#x27;ll copy&#x2F;paste the thing i added to the FAQ part of my post

I don't think I'd call it that. UNCTADstat doesn't have any particular usage guidelines I could find, though the agents did get rate-limited ("please stop rinsing my site") and continued rinsing the API with requests regardless

The main argument I'd put forth for what's so concerning about the behaviour is that, when you bypass restrictions such as the 400 on GETs to Facts, you don't really know what the server will return. And from a site admin perspective, if I see someone sending those sorts of carefully-contrived queries such as double-encoding, man, that sure looks like the actions of a hacker.

Basically, these look like the actions of someone, or something, that won't take "no" for an answer, and I think that behaviour is worth investigation.

  Isn&#x27;t this still the same huggingface&#x2F;wiki incidents where the agents managed to hack their way out of a testing center in Israel?

this one is hard to answer. we find a lot of shared IPs between the wiki swarms and the IPs that seem to follow after Urlquery scans (i.e. we can correlate an Urlquery IP visit with a shortly preceding/following visit to the exact same page and infer from that), but that still doesn't tell us whether it's the same swarm, let alone the same agents. From the wiki incident we know that at least some of the agents had shared IPs without sharing context windows - they have azure ip ranges and one public IPv4 can obviously be associated with many many boxes.

Basically I wouldn't rule it out but I wouldn't rule it in either.

  The more of these that come out the more incompetent OpenAI looks

It is a bad look, though I don't think OpenAI is uniquely bad here. They're a bigger company than many other AI companies, so obviously you'd expect that they'd have a proportionally larger number of incidents. To their credit, in many respects they've been far more open than other similar incidents (we have very little data about the Anthropic rogue agents as far as I'm aware, though I hope that soon we can find more traces of them on the open web).

But for sure, there's negligence here. Many of these attacks could have been stopped by some basic CoT monitoring, network traffic anomaly detection, etc. I think it's useful to consider why this stuff doesn't seem to have been put in place, beyond the blame game of "it's just incompetence" - if your road crossing has a traffic light button, but people keep crossing without pressing it and getting hit by cars, there's a point where you have to consider why people aren't using it. Maybe it turns out you painted it completely grey and the whole thing camouflages into the pavement.

In short, maybe there just needs to be better, more easy-to-set up, comprehensive tooling around this.

And I don't think it's purely the cybersecurity angle either. I think these agents do present a new kind of threat - I don't buy the "it's just like a computer worm" framing. I'm not really sure where to go with that thought, but I think there's something going on here beyond mere "just do better"

(i'm told that "rinsing" is actually british slang, which is where i'm from - i thought it was global, so i'm somewhat regretting using it in the article, but oh well. i use it to basically mean "hammering")