I built an AI "web harness" running on a sandboxed Chromium (using a custom side-loaded plugin that talks over websockets to a "driver") to basically do anything a normal user could do in a browser. It totally bypasses any and all bot measures and only gets the ones you yourself would get as well (and passes those successfully, e.g. Cloudflare checkbox or those annoying OCR puzzles).
Not sure if I should release it, but I'm sure more people are catching onto the power of agentic browsing.
Thanks for letting us know that we need a new layer of detection systems.
Also it’s great(!) to see that we’re going from “but ethics” to “I got mine, who cares”.
Humans are interesting creatures.
Edit: Please before assuming that I'm assuming things, this is an observation I'm making over time. It's possible that I'm in a bubble, but it's not a sample size of 1 (i.e. The comment I replied only).
There is no detection method that will prevent AI from accessing systems without also blocking humans. The only thing we can do at this point is throttling.
Agreed on inevitable collateral blockage of real people using real "headed" browser. I'm getting a ton of that already, personally.
Throttling is poor help though. Mass scrapers are using "residential proxy" loophole + rotating UA and other attributes. You can't throttle somebody without identifying them. Unless you're talking about a global rate-limit.
throttling based on sessions kind of works; we're headed in the direction that sites like Reddit will probably prevent logged-out users from viewing threads (as they already do with mobile devices)
Once the LLMs create sockpuppets to get around that, the web services will need to resort to profiling users more aggressively so that they know which actual human an account corresponds to.
If someone has a malicious browser extension that uses their session to scrape Reddit then, they're probably going to see significant usage obstacles.
dvt · · focus · HN ↗
Not sure if I should release it, but I'm sure more people are catching onto the power of agentic browsing.
bayindirh · · focus · HN ↗
Also it’s great(!) to see that we’re going from “but ethics” to “I got mine, who cares”.
Humans are interesting creatures.
Edit: Please before assuming that I'm assuming things, this is an observation I'm making over time. It's possible that I'm in a bubble, but it's not a sample size of 1 (i.e. The comment I replied only).
pcthrowaway · · focus · HN ↗
kees99 · · focus · HN ↗
Throttling is poor help though. Mass scrapers are using "residential proxy" loophole + rotating UA and other attributes. You can't throttle somebody without identifying them. Unless you're talking about a global rate-limit.
pcthrowaway · · focus · HN ↗
Once the LLMs create sockpuppets to get around that, the web services will need to resort to profiling users more aggressively so that they know which actual human an account corresponds to.
If someone has a malicious browser extension that uses their session to scrape Reddit then, they're probably going to see significant usage obstacles.
We are headed to a very user-hostile place.
mvt67 · · focus · HN ↗
rennf93 · · focus · HN ↗
[dead]
cyanydeez · · focus · HN ↗
This predates AI as a _capitalism_ problem.