Comment by jsnell

Comment by jsnell 9 hours ago

34 replies

View on Hacker News

I don't think it's a cloud. It's more likely a residential proxy network, which are typically created by installing malware on users' machines.

The operators of these proxy networks want to avoid detection by both the users whose bandwidth they're stealing, and by the companies whose data is being scraped. So they want to make the bandwidth very expensive. And that expensive bandwidth in turn means that their only clients are dodgy as well. Either people looking to scrape data without consent and monetize it, or outright criminals.

iforgotpassword 9 hours ago

I use one. I run a bot on IRC that extracts the <title> of every link posted (or downloads the image/whatever and extracts Metadata) and announces that to the channel. It has become more and more pointless to run this on a vps. Google/YouTube block the IP range, a lot of websites return the cloudflare security check, Amazon works on some days and doesn't on others... Ever since I proxy via residential proxies it just works. I'm a smooth criminal. :>

Reply View 5 replies
  • morkalork 8 hours ago

    So much for the open internet.

    Reply View 3 replies
    • nolist_policy 6 hours ago

      You can thank the spammers.

      Reply View 2 replies
      • Nextgrid 4 hours ago

        I’m not sure how much of this is due to spammers and how much is due to “growth & engagement” that wants to make sure a human’s time is being wasted.

        Reply View 0 replies
      • wruza 3 hours ago

        To stop spammers, you implement measures before posting, not before viewing. Spam is just a minor technical nuisance. It's automated interaction that really makes their executives sweat and shiver.

        Reply View 0 replies
  • derekzhouzhen 6 hours ago

    I feel your pain, but I refuse to cave. Say, 10% of the links fail to load, so what? It is their loss, not mine.

    Reply View 0 replies
dewey 6 hours ago

There's many reputable residential proxy networks too, usually there's a lot of vetting involved too as they don't want people running illegal activities though their network.

It's almost a necessity these days to have access to that due to how much datacenter ranges are blocked.

Reply View 0 replies
bscphil 9 hours ago

It's kind of surprising that a presumptively legitimate company (and YC-funded startup) would out themselves as buying black market residential proxy bandwidth, isn't it?

Reply View 16 replies
  • jsheard 9 hours ago

    Their frontpage also advertises the ability to pass CAPTCHAs, whether by automation or more likely by delegating them to third-world CAPTCHA farms. If that's a major selling point for your automation service then your target market probably ranges from dubious (e.g. data scrapers trying to get around limits) to extremely dubious (e.g. ticket scalpers, spammers, click fraud, etc).

    Reply View 11 replies
    • xp84 6 hours ago

      Just because something can be used for sketchy purposes doesn't mean that's the only purpose of it. there are thousands of situations where people are forced to interact with a shitty website 100x per day and the site won't provide an api. Imagine if your job was booking plane tickets all day. United could provide you an API key to do so via an API, but in practice they won't, only some enterprisey travel software company can get that kind of access, for a steep fee. You could build a tool which automatically puts together an itinerary based on rules and books it, through a tool like this. Perhaps a slightly contrived example but I believe things like this definitely happen.

      Reply View 10 replies
      • miki123211 3 hours ago

        A very common and pro-consumer use for residential proxies is price scraping and price comparisons.

        Most businesses don't want to compete on price and are extremely unhappy if you tell people that their competition sells the same stuff but for less, that their "best deal of the month" is actually a price raise, or that they significantly raise toilet paper prices every time there's a natural disaster.

        Reply View 0 replies
      • dontlikeyoueith 6 hours ago

        > United could provide you an API key to do so via an API, but in practice they won't, only some enterprisey travel software company can get that kind of access, for a steep fee. You could build a tool which automatically puts together an itinerary based on rules and books it, through a tool like this. Perhaps a slightly contrived example but I believe things like this definitely happen.

        And you think that's NOT sketchy?

        I'm almost afraid to ask where you think the bar is...

        Reply View 2 replies
      • suchintan 5 hours ago

        Agreed. Just for reference, one of our most popular use-cases is automating data entry into CRMs without APIs... No one wants to be doing this stuff manually, and automating it has some serious positive QoL impact

        We get a lot of requests for bad usage (ie spinning up upvote rings on Reddit) but we don't want to support things like that

        Reply View 2 replies
      • rty32 6 hours ago

        Imagine a legitimate travel agency cannot book 100 United tickets a day via methods outlined in business contracts and need to resort to shady practice.

        Dude, please provide some real solid evidence to back this up, and perhaps come up with another realistic scenario where bypassing captcha is justified.

        Reply View 2 replies
  • dewey 6 hours ago

    Residential proxies are not necessarily "black market".

    Reply View 2 replies
    • asmor 4 hours ago

      It's almost never done with the full understanding of the person providing the proxy, doesn't matter if they get promised some change, their browser addons betray them or they install bundleware/adware.

      I'd say it has about the same moral standing as a payday loan.

      Reply View 1 reply
      • dewey 4 hours ago

        There’s other ways for example through mislabeled “residential” blocks, or “residential” proxies that are sold by ISPs to vendors.

        Reply View 0 replies
  • mrguyorama 9 hours ago

    How long have you been here? It's not surprising at all. HN and YC have not demonstrated an aversion to "uh, greyhat" activity.

    If it were 2000, people would be sharing their ad clicking startups.

    YC has funded a looooooot of sketchy companies.

    Reply View 0 replies
floam 9 hours ago

It’s not necessarily malware. There are services that are pretty upfront and pay cash money for residential US bandwidth. That said, naive people might be surprised when their IP starts getting blocked.

e.g. https://www.honeygain.com/ (something like 100GB = $20).

Reply View 1 reply
  • Saris 2 hours ago

    >That said, naive people might be surprised when their IP starts getting blocked.

    Or law enforcement shows up at their door because their IP is involved in a bunch of illegal stuff.

    Reply View 0 replies
peab 9 hours ago

how does expensive bandwidth equate to dodgy clients? There are lot's of valid use cases for scraping data, and it's legal to scrape publicly available data, even if the websites hosting it try to block it (try a curl request to reddit, for example)

Reply View 2 replies