Comment by nikisweeting
Comment by nikisweeting 3 days ago
Are you using a browser provider or building your own CUA scraping pipelines in-house? What's your strategy around CAPTCHAs, paywalls, rate-limits, etc.?
Comment by nikisweeting 3 days ago
Are you using a browser provider or building your own CUA scraping pipelines in-house? What's your strategy around CAPTCHAs, paywalls, rate-limits, etc.?
All in-house! We don't visit sites with paywalls, but we do take lots of measures to avoid rate-limits.