Comment by binaryturtle

Comment by binaryturtle 3 days ago

I'm on an older system here, and both Cloudflare and Anubis entirely block me out of sites. Once you start blocking actual users out of your sites, it simply has gone too far. At least provide an alternative method to enter your site (e.g. via login) that's not hampered by erroneous human checks. Same for the captchas where you help train AIs by choosing out of a set of tiny/ noisy pictures. I often struggle for 5 to 10 minutes to get past that nonsense. I heard bots have less trouble.

Basically we're already past the point where the web is made for actual humans, now it's made for bots.

inejge 3 days ago

> Once you start blocking actual users out of your sites, it simply has gone too far.

It has, scrapers are out of control. Anubis and its ilk are a desperate measure, and some fallout is expected. And you don't get to dictate how a non-commercial site tries to avoid throttling and/or bandwidth overage bills.

Reply View 5 replies

account42 3 days ago

No, they are a lazy measure. Most websites that slap on these kinds of checks don't even bother with more human-friendly measures first.

Reply View | 4 replies
- mschuster91 3 days ago
  
  Because I don't have the fucking time to deal with AI scraper bots. I went harder - anything even looking suspiciously close to a scraper that's not on Google's index [1] or has wget in its user agent gets their entire /24 hard banned for a month, with an email address to contact for unbanning.
  That seems to be a pretty effective way for now to keep scrapers, spammers and other abusive behavior away. Normal users don't do certain site actions at the speed that scraper bots do, there's no other practically relevant search engine than Google, I've never ever seen an abusive bot hide as wget (they all try to emulate looking like a human operated web browser), and no AI agent yet is smart enough to figure out how to interpret the message "Your ISP's network appears to have been used by bot activity. Please write an email to xxx@yyy.zzz with <ABC> as the subject line (or click on this pre-filled link) and you will automatically get unblocked".
  [1] https://developers.google.com/search/docs/crawling-indexing/...
  
  Reply View | 3 replies
  
  account42 3 days ago
  
  > Normal users don't do certain site actions at the speed that scraper bots do
  How would you know when you have already banned them.
  
  Reply View | 2 replies

alperakgun 3 days ago

I gave up on a lot of websites because of the aggressive blocking.

Reply View 0 replies

johnklos 3 days ago

FYI - you can communicate with the author of Anubis, who has already said she's working on ways to make sure that all browsers - links, lynx, dillo, midori, et cetera, work.

Unless you're paying Cloudflare a LOT of money, you won't get to talk with anyone who can or will do anything about issues. They know about their issues and simply don't care.

If you don't mind taking a few minutes, perhaps put some details about your setup in a bug report?

Reply View 0 replies