Comment by jsheard
> Whatever happened to courtesy in scraping?
Money happened. AI companies are financially incentivized to take as much data as possible, as quickly as possible, from anywhere they can get it, and for now they have so much cash to burn that they don't really need to be efficient about it.
not only money, but also a culture of "all your data belong to us" because our ai going to save you and the world.
the hubris reminds me of dot-com era. that bust left a huge wreckage. not sure how this one is going to land.