Thread

1. itintheory · 1789663596 · focus · HN ↗

   We implemented CrowdSec for bot/scraping mitigation. The architecture is sound, but it ended up having an unacceptable false positive rate for us. This may be an issue with any kind of IP reputation approach. After a couple of months of work getting it ready to go I had to turn it off after a couple of days.

   1. mazzma · 1789664910 · focus · HN ↗

      Interesting, did you implement only IP reputation (via blocklist) or did you deploy the WAF as well? Regarding bot scrapping, you would probably want to try the new bot detection feature recently released

      1. itintheory · 1789667581 · focus · HN ↗

         This was just blocklist based. We had the main community list and a handful of the curated paid lists enabled.

         wrt bot detection - this sounds very much like Anubis which we're also using with some success.

Unofficial Hacker News client; not affiliated with Y Combinator.