Comment by fluoridation

Comment by fluoridation 4 days ago

>No. A human sees a 10x slowdown.

For the actual request, yes. For the complete experience of using the website not so much, since a human will take at least several seconds to process the information returned.

>And the scraper paid one 1/1000000th of a dollar. (The scraper does not care about latency.)

The point need not be to punish the client, but to throttle it. The scraper may not care about taking longer, but the website's operator may very well care about not being hammered by requests.

avhon1 4 days ago

But now I have to wait several seconds before I can even start to process the webpage! It's like the internet suddenly became slow again overnight.

Reply View 1 reply

fluoridation 4 days ago

Yeah, well, bad actors harm everyone. Such is the nature of things.

Reply View | 0 replies

jsnell 4 days ago

A proof of work challenge does not throttle the scrapers at steady state. All it does is add latency and cost to the first request.

Reply View 2 replies

fluoridation 4 days ago

Hypothetically, the cookie could be used to track the client and increase the difficulty if its usage becomes abusive.

Reply View | 1 reply
- soulofmischief 4 days ago
  
  Yes, and then we can avoid the entire issue. It's patronizing for people to assume users wouldn't notice a 10x or 50x slowdown. You can tell those who think that way are not web developers, as we know that every millisecond has a real, nonlinear fiscal cost.
  Of course, then the issue becomes "what is the latency and cost incurred by a scraper to maintain and load balance across a large list of IPs". If it turns out that this is easily addressed by scrapers then we need another solution. Perhaps, the user's browser computes tokens in the background and then serves them to sites alongside a certificate or hash (to prevent people from just buying and selling these tokens).
  We solve the latency issue by moving it off-line, and just accept the tradeoff that a user is going to have to spend compute periodically in order to identify themselves in an increasingly automated world.
  
  Reply View | 0 replies