Comment by jsnell

Comment by jsnell 4 days ago

No, the economics will never work out for a Proof of Work-based counter-abuse challenge. CPU is just too cheap in comparison to the cost of human latency. An hour of a server CPU costs $0.01. How much is an hour of your time worth?

That's all the asymmetry you need to make it unviable. Even if the attacker is no better at solving the challenge than your browser is, there's no way to tune the monetary cost to be even in the ballpark to the cost imposed to the legitimate users. So there's no point in theorizing about an attacker solving the challenges cheaper than a real user's computer, and thus no point in trying to design a different proof of work that's more resistant to whatever trick the attackers are using to solve it for cheap. Because there's no trick.

pavon 4 days ago

But for a scraper to be effective it has to load orders of magnitude more pages than a human browses, so a fixed delay causes a human to take 1.1x as long, but it will slow down scraper by 100x. Requiring 100x more hardware to do the same job is absolutely a significant economic impediment.

Reply View 1 reply

jsnell 4 days ago

The entire problem is that proof of work does not increase the cost of scraping by 100x. It does not even increase it by 100%. If you run the numbers, a reasonable estimate is that it increases the cost by maybe 0.1%. It is pure snakeoil.

Reply View | 0 replies

fluoridation 4 days ago

>An hour of a server CPU costs $0.01. How much is an hour of your time worth?

That's irrelevant. A human is not going to be solving the challenge by hand, nor is the computer of a legitimate user going to be solving the challenge continuously for one hour. The real question is, does the challenge slow down clients enough that the server does not expend outsized resources serving requests of only a few users?

>Even if the attacker is no better at solving the challenge than your browser is, there's no way to tune the monetary cost to be even in the ballpark to the cost imposed to the legitimate users.

No, I disagree. If the challenge takes, say, 250 ms on the absolute best hardware, and serving a request takes 25 ms, a normal user won't even see a difference, while a scraper will see a tenfold slowdown while scraping that website.

Reply View 8 replies

michaelt 4 days ago

The problem with proof-of-work is many legitimate users are on battery-powered, 5-year-old smartphones. While the scraping servers are huge, 96-core, quadruple-power-supply beasts.

Reply View | 0 replies
jsnell 4 days ago

The human needs to wait for their computer to solve the challenge.
You are trading something dirt-cheap (CPU time) for something incredibly expensive (human latency).
Case in point:
> If the challenge takes, say, 250 ms on the absolute best hardware, and serving a request takes 25 ms, a normal user won't even see a difference, while a scraper will see a tenfold slowdown while scraping that website.
No. A human sees a 10x slowdown. A human on a low end phone sees a 50x slowdown.
And the scraper paid one 1/1000000th of a dollar. (The scraper does not care about latency.)
That is not an effective deterrent. And there is no difficulty factor for the challenge that will work. Either you are adding too much latency to real users, or passing the challenge is too cheap to deter scrapers.

Reply View | 6 replies
- fluoridation 4 days ago
  
  >No. A human sees a 10x slowdown.
  For the actual request, yes. For the complete experience of using the website not so much, since a human will take at least several seconds to process the information returned.
  >And the scraper paid one 1/1000000th of a dollar. (The scraper does not care about latency.)
  The point need not be to punish the client, but to throttle it. The scraper may not care about taking longer, but the website's operator may very well care about not being hammered by requests.
  
  Reply View | 5 replies
  
  avhon1 4 days ago
  
  But now I have to wait several seconds before I can even start to process the webpage! It's like the internet suddenly became slow again overnight.
  
  Reply View | 1 reply
  
  fluoridation 4 days ago
  
  Yeah, well, bad actors harm everyone. Such is the nature of things.
  
  Reply View | 0 replies
  
  jsnell 4 days ago
  
  A proof of work challenge does not throttle the scrapers at steady state. All it does is add latency and cost to the first request.
  
  Reply View | 2 replies