valicord 2 days ago

The parent comment was "The author of that site assumes that scrapers will keep track of the access tokens for a week, but most internet-wide scrapers don't do so.". There's no technical reason why they wouldn't reuse those tokens, they don't do that today because they don't care. If anubis gets enough adoption to cause meaningful inconvenience, the scrapers would just start caching the tokens to amortize the cost.

The point of the article is that if the scraper is sufficiently motivated, Anubis is not going to do much anyway, and if the scraper doesn't care, same result can be achieved without annoying your actual users.

tecoholic 2 days ago

Hmm… by setting the verified=1 cookie on every request to the website?

Am I missing something here? All this does is set an unencrypted cookie and reload the page right?

  • notpushkin 2 days ago

    They could, but if this is slightly different from site to site, they’ll have to either do this for every site (annoying but possible if your site is important enough), or go ahead and run JS (which... I thought they do already, with plenty of sites still being SPAs?)

    • rezonant 2 days ago

      I would be highly surprised if most of these bots are already running JavaScript, I'm confused by this unquestioned notion that they don't.