Comment by tortillasauce

Comment by tortillasauce 3 days ago

0 replies

Anubis works because AI crawlers do very little requests from an ip address to bypass rate-limiting. Last year they could still be blocked by ip range, but now the requests are from so many different networks that doesn't work anymore.

Doing the proof-of-work for every request is apparently too much work for them.

Crawlers using a single ip, or multiple ips from a single range are easily identifiable and rate-limited.