Comment by majorchord a day ago
You can't just blanket block all VPN access, that's not how the internet works... they could pick some common/well-known providers of VPN services and block their IPs/ASN/etc., but you can't just flip a switch and make all forms of VPN/proxy stop working, as there's no way to tell with certainty that someone is using one.
> Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.
Assuming your VPN identification service operates commercially, I trust that you are in full compliance with all contractual agreements and Terms of Service for the services you utilize. Many of these agreements specifically prohibit commercial use, which could encompass the harvesting of exit node IP addresses and the subsequent sale of such information.
> I trust that you are in full compliance with all contractual agreements and Terms of Service
Why? It's not like there's any real moral (or, likely, legal) reason to care beyond avoiding the service's ban hammer.
In Illinois you could, in theory, be jailed for up to three years for violating a web site ToS. (classified as "Computer Tampering")
There's a little secret that most of the business world knows but individuals do not know: You don't have to follow Terms of Service. In most cases, the maximum penalty the company can impose for a ToS violation is a termination of your account. And it's not illegal to make a new account. They can legally ban you from making a new account, and you can legally evade the ban.
Unless you're the one-in-a-million unlucky user who gets prosecuted under the CFAA's very generic "unauthorized access to a protected computer" clause, like Aaron Swartz. It seems the general consensus is this doesn't apply to breaking a website ToS, and Aaron was only in so much trouble because he broke into a network closet, as well as for copyright violation. But consult a lawyer if unsure. (That's another difference: A business will ask a lawyer if it wants to do something shady, while an individual will simply avoid doing it)
Tangent: if you hold access to all VPN providers, have you thought about also releasing benchmarks for them? I would be interested in knowing which ones offer the best bandwidth / peering (ping).
> which are surprisingly comprehensive
How does the buyer even know what the precision and recall rates might be?
Much of the internet still does not support IPv6, so most providers will give you an IPv4 address. In fact only a few providers even support IPv6 at all.
Even with IPv6 it's not a huge problem. With a few samples we can know that a provider is operating in a given /64 or /48 or even /32 space, and can assign a confidence level that the range is used for VPNs.
It won’t end up in our proxy detection database, but we track hosting provider ranges separately: https://www.iplocate.io/data/hosting-providers/
It is even easier to block hosting providers. They typically publish official lists. Here's the full list for both of those providers:
https://ip-ranges.amazonaws.com/ip-ranges.json
https://digitalocean.com/geo/google.csv
(And even if they don't publish them, you can just look up the ranges owned by any autonomous network with the appropriate registry.)
This will also cause problems with anyone that happens to (even accidentally/unknowingly) use apps that integrate services from companies such as BrightData/Luminati/HolaVPN/etc. where they sell idle time on your device/connection to their VPN/proxy customers.
The legitimate end-user will then no longer be able to use e.g. SoundCloud.
I’m with you on this one. Some of my projects are flooded with sus traffic from Brazil. I don’t believe there are a million eager Brazilian hackers targeting me in particular. It’s pretty clear from analysis that they’re all residential hosts running proxies, knowingly or otherwise.
The more concise word for this is “botnet”. Computers participating in one should be quarantined until they stop.
> unknowingly
Often times random shovelware apps will have these proxy SDKs embedded in them, and the only mention of it being part of the software is buried in some long ToS that nobody reads.
Sort of valid today.
But the more sites that require a residential VPN for normal use, the less legitimate that argument becomes.
You might want to learn how internets work today: https://en.wikipedia.org/wiki/Network_address_translation
GEOIP providers often sell a database of known VPN/Proxy endpoints. They take the approach of shoot first, ask questions later. Using one of these databases bans a lot of legitimate ip addresses that have seen been the source of known VPN or proxy traffic.
Its not perfect ofc, but its not meant to be. Its usually just used as a safety blanket for geoblocked intellectual property, like netflix.
For low-volume stuff you can always get a non-expiring 4G/5G bundle eSIM and tunnel through that. Because 4G/5G roaming always tunnels traffic through the home country, and then emerges from CGNAT so it can't be identified as foreign traffic.
But those data packages are expensive and not available with each wanted origin country. Also you need hardware on your side. But it is an option, just saying.
I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.
Maybe they mean commercial VPN providers that run on the cloud?
You know perfectly well what blocking VPN access means in common verbiage. I don't understand the motivation of these "hey look my WireGuard connection to home isn't blocked, you guys don't know the true meaning of VPN" comments that inevitably pop up in these discussions. Like come on, this is a tech forum, you're not impressing anyone for knowing the technical definition of VPN and how to set up WireGuard.
Here's me making a similar argument a month or so ago
https://news.ycombinator.com/item?id=45926849
Besides the political implications, I think we should try to find an objective taxonomy, it's clear that privacy VPNs and network security VPNs are different products semantically, commercially and legally, even if the same core tech is used.
Possibly the configuration and network topology is different even, making it a technically different product, similar to how a DNS might be either an authorative server for a TLD, an ISP proxy for an end user, a consumer blacklist like pihole, or an industrial blacklist like spamhaus. It would be a non trivial mistake to conflate any pair of those and bring one up in an argument that refers to the other.
The exhausting "well actually" masks a corrosive argument, that if you can't enforce the rules in a rigid and rigorous fashion, the rule is fiat.
It's not that he doesn't know the difference. He's making the argument that since there's no _technical_ difference there can be no legal difference.
Tailscale is really not that hard to set up. There's an Apple TV app for it, even. And who doesn't have some friend in another state or country that would like an Apple TV?
>I connect to my residential ISP in the USA via VPN all the time and have never had issues with being blocked for VPN use.
Bit of a non sequitur, you would have to outline your entire usage pattern to even submit that as N=1.
GEOIP providers dont sit on your home network. They do accept data from third parties, and are themselves (likely) subscribed to other IP addressing lists. Mostly they are a data aggregator, and its garbage in > garbage out.
If someone, say netflix, but other services participate, flag you as having an inconsistent location, they may forward those details on and you can get added to one of these lists. You might see ip bans at various content providers.
But the implementation is so slapshod that you can just as likely, poison a single ip in a CGNAT pool, and have it take over a month for anyone to act on it, where some other users on your same ISP might experience the issue.
These things can also be weighted by usage, larger amounts of traffic are more interesting because it can represent a pool of more users, or more IP infringement per user.
You can also get hit from poor IP reputation, hosting a webserver with a proxy or php reverse shell, or a hundred other things.
(Also, larger ISPs might deal with a GEOIP provider selling lists of VPN users that include their IP address space, legally, rather than just going through the process of getting the list updated normally. This means the GEOIP providers can get skittish around some ISPs and might just not include them in lists)
There is even a single company in the unique position to actually tell where exactly(-ish, considering CGNAT exists) where an IP address is located: Google. They do use the "enhanced location" data on Android devices to pinpoint where an IP is, so a single Android device can actually change fings for Google (and YouTube).
https://ipinfo.io/what-is-my-ip
Here’s one database to check.
Yes, and email is decentralized in theory...
If using a VPN for access is forbidden by the ToS, you only need to detect a VPN connection once to prove violation.
The IPv4 address space to consider is limited and it is technically absolutely feasible to exhaustively scrape and block the majority of VPN endpoints. Realistically any VPN provider will have some rather small IPv4 subnets make do, shit's expensive. More so, for the trivial case, VPN anonymization works best, when many people share one IP endpoint, naturally the spread is limited. There are VPN providers, some may even be trustworthy, which have the mission of "flying under the radar" with residential IPs and all, but they are way, waaaay more expensive. For most people that's no option.
IPv6 is a different matter, but with the very increase in tracking and access control discussed here, that may be even more of a reason, IPv6 is not going to be a thing any time soon....
Thinking about it, maybe this AI monetization FOMO and monopoly protectionism, will incidentally lead to a technological split of the web. IPv4 will become the "corpo net" and IPv6 will be the "alt net". I think there may be a chance to make IPv6 the cool internet of the people, right now!
> you only need to detect a VPN connection once to prove violation
But an IP address is not a person (legally in the US at least), and many IPv4 addresses get re-used fairly often. My home 5G internet changes IP every single day, and it's a constant struggle because other users often get my IP blocked for things I didn't do. I cannot even visit etsy.com for example. Just for fun I even checked 4chan and the IP was banned for CP, months before I ever had this particular IP (because I'm paranoid and track all that stuff).
> But an IP address is not a person (legally in the US at least)
That's a completely different matter (and still probably reasonable suspicion for a search, anyway). If an account/service ID evidently uses a service through a VPN there is no uncertainty of ToS violation. Of course someone could have hacked your account and used a VPN, it doesn't ultimately prove you did it, but nevertheless the account can be flagged/blocked correctly for VPN usage.
> many IPv4 addresses get re-used fairly often
The VPN's servers won't be using changing, "random" IPs. That's something ISPs do when assigning residential IPs. VPNs with residential IPs are not common. (I am not sure those VPNs are even really legal offerings.)
If your ISP uses NAT for its subnet space, you could argue it's technically similar to a VPN. However, same as with VPN exit scraping/discovery, those IP spaces can be determined and processed accordingly. I am also sure those ISP subnets for residential IPs are actually publicly defined and known. Eg. the Vodafon IP may get temporarily flagged for acute suspicious behavior, but won't get your account flagged for VPN violation, or even blocked permanently, since it's known to be the subnet of a mobile ISP, which uses NAT.
Additionally, I presume e.g. SoundCloud prohibits anonymizing VPNs, not everything that's technically a VPN or similar.
And also it doesn't matter what the legally provable significance of an IP address is for the purposes of violating a ToS. A ban from SoundCloud is not a court proceeding. ToS agreements are allowed to have arbitrary rules, and they routinely do.
As long there isn't a critical risk, these kind of business decisions won't aim for certainity.
They probably assume some amount of collateral damage, a small number of VPN users still flying under the radar, the bulk of VPN users being properly targeted, and the vast majority of users not noticing anything.
It is easier to block all non-residential addresses, than block VPNs. As an added "bonus" it also kills personal VPNs running on VPS. VPNs in residential space exist but are sold as "premium" product.
yes and those users that happen to have their bw sold as residential VPN will be caught in the crossfire... many times they are not even aware of it because it's something buried in a ToS they didn't read for some random app.
Hell, I remember malware (Trojans / RATs) from the 2000s that allowed you to use your victims IP as your personal proxy.
Now that you mention it, I never used those, but I always did wonder how they do those.
Someone googles "free VPN" so they can watch region locked videos and now their connection is a part of that network too. They may or may not realize that this is the arrangement.
Maybe its a trick and they are logging all the people on VPN's trying to see if they are blocked over the next 24 hr. Then they can take the data and start blocking it lol. Maybe not lol?
> but those are rare.
yeah sure, if you ignore the existence of literally every mobile isp.
looks at Japan, UK (OpenReach), and a lot of other places still using PPPoE (on fiber!) for complicated reasons
Hard disagree... there are still a vast many providers around the world doing < 1500, such as PPPoE DSL.
There are plenty of VPN and proxy detection services, either as a service (API) or downloadable database, which are surprisingly comprehensive. Disclaimer: I’ve run one since 2017. Years on, our primary data source is literally holding dozens of subscriptions to every commercial provider we can find, and enumerating the exit node IP addresses they use.
There are also other methods, like using zmap/zgrab to probe for servers that respond to VPN software handshakes, which can in theory be run against the entire IP space. (this also highlights non-commercial VPNs which are not generally the target of our detection, so we use this sparingly)
It will never cover every VPN or proxy in existence, but it gets pretty close.