jsheard 13 hours ago

Bold to assume that an AI scraper won't come back to download everything again, just in case there's any new scraps of data to extract. OP mentioned in the other thread that this bot had pulled 3TB so far, and I doubt their git server actually has 3TB of unique data, so the bot is probably pulling the same data over and over again.

  • xena 13 hours ago

    FWIW that includes other scrapers, Amazon's is just the one that showed up the most in the logs.

_heimdall 12 hours ago

If they only needed a one-time scrape we really wouldn't be seeing noticeable not traffic today.