Comment by cjonas
How do you get by when every major sites starts blocking headless browsers? A good example right now is Zillow, but I foresee a world where big chunks of the internet are behind captcha and bot detection
How do you get by when every major sites starts blocking headless browsers? A good example right now is Zillow, but I foresee a world where big chunks of the internet are behind captcha and bot detection
Playwright does have some docs on scraping, which makes it seem like they do want to support that as a use case https://playwright.dev/docs/docker#crawling-and-scraping. I'm unfamiliar with that though and wouldn't be able to speak to addressing issues with scraping
Right, but I mean it's a plugin issue outside of Stagehand. That's currently how the space treats the issue.
There are plugins that extend the major browser automation libraries to do this sort of thing.
And undetectable browser automation doesn't use Playwright, Puppeteer, or Selenium at all.
That's not really a problem for Stagehand. It's a problem for Selenium, Playwright, Puppeteer and others at the browser automation library level.