Ask HN: Why does the US Visa application website do a port-scan of my network?

I had to deal with the DS-160 multiple times over the year. I don't think you give justice to how bad this website really is. I have started to notice that these "timeouts" are very random. At the worst times, the session "times out" immediately after login.

These random logouts happens more frequently during certain times of the day and seems to follow a semi-predictable pattern. It is almost certainly tied to system load in some way.

Also, the site's HTML and JavaScript are bloated beyond hope for what should be a fairly simple set of web forms. And itnhas been thisnway since at least 2018 with exactly zero improvements.

You might be making the assumption that the US wants to make the process easier.

The VISA appointment scheduling site rate limits to a ridiculous degree these days. As in refresh your page within 10seconds and get a 429 error.

That's probably because of the fact that the appointments are near impossible to get, they only allow booking a few months out and it's always completely booked. So everyone was refreshing (or if clever botting) to get an appointment slot.

As I wrote elsewhere; they subcontract the bot protection to F5, an external company that I see for some reason a lot on old/horrible banking websites.

Hey, this is actually something I have a keen interest in as I'm fighting my government (as an MP) to drop those scammers where possible. Do you have any media links to send me about them selling the "good hours" on the black market?

Even if the US has a horrible visa system – as I can attest, despite only having to do it every 5 years – the EU countries could benefit from attracting talent by being more welcoming. So that is part of my mission as an MP and tech-entrepreneur. Any help and pointers is welcome.

Whenever I'm filling a long form on an official website, I feel like I'm racing against an invisible clock because of this session time out thing that happened to me countless times.

I had this problem too last year. I found, at the time, it was the website was poorly managing the session in some browsers causing the timeout countdown to not be reset on activity. I had to find a windows computer and use microsoft edge I think (maybe it was chrome). But no browser on my mac would not have that issue.

> In the process she was also directed to a non-.gov website for something during the process, I thought she was getting scammed but no.

No clue if this specific instance if scam but such scams have indeed been done before

https://www.bbc.com/news/articles/cdr56vl410go

> According to Ablakwa, a locally recruited staff member and "collaborators" were allegedly involved in a "fraudulent" scheme whereby they extracted money from visa and passport applicants.

> It is alleged that the scheme consisted of creating an unauthorised link on the embassy's website to redirect visa and passport applicants to a private firm where they were "charged extra for multiple services" without the knowledge of the foreign ministry.

> Ablakwa added that the staff member "kept the entire proceeds" in their private account, and that the scheme had been going on for five years.

> Applicants seeking visas were charged unapproved fees ranging from almost $30 (£22) to $60 by the private firm.

The hard truth of it all is that both the US and (partially) the EU don’t want to make this easier because seeing as wanting “outside” people is now a political liability. You may want to adjust your expectations around that.

>> the system just kick her out

The "waterfall model" is a toxic way of thinking that pervades corporate management. Simplistic minds can't fathom any states other than "done" or "not done". Corporations are determined to crush the human soul. That is why it's not a progressive series of forms, saving your progress all along.

Not sure if this is the case for India, but I've experienced similar situations for other countries, but the 'scam websites' actually provided a real service - if you needed some ultra-urgent processing (like you only realized you needed a visa to this country before boarding a flight, once you were already at the airport check-in...) they were able to provide 30 minute approval, whereas the official site's accelerated processing was 24 hours.

So obviously the only way they could to this is with government contacts meaning the government themselves could already do it, but a lot of immigration stuff everywhere is full of people taking kickbacks.

The scam websites are probably owned by someone who works in the Indian govt.

I found the real website, but the application never went through, always some issue. My boss told me which service to use and everything just worked. (I could expense that service so cost didn't bother me)

[dead]

Just a guess, but maybe a typical bot has a webserver, ssh server, some other servers running on the same machine, whereas a typical Visa applicant doesn't.

I imagine it may not be a proxy in the true sense, but a headless browser that's "proxying" the application process rather than the network traffic itself.

Proxy is being used in the traditional sense here. It’s common for a business (scam or legit) to handle visa applications on behalf of customers.

That bought nginx, but yes!

I didn't know that! But apparently yes

There are certainly use cases, but whether they’re warranted is a good question.

One popular router maker offers a ‘magic URL’ (domain name) that scans your network for the gateway management page, and redirects. It’s not necessary, but it certainly helps novice users. Having worked in IT support,

I’ve also purchased hardware devices that have a web management UI; which connects directly instead of proxying through a cloud.

Ultimately this is probably one thing that should be behind a permission request (like webcam access), but it’s not a feature without value.

It's being looked at.

https://bugzilla.mozilla.org/show_bug.cgi?id=1481298

Internal apps on non-private IP addresses occasionally use this. There is a standard called Private Network Access[1] that requires these requests to have preflights like CORS requests. Only Chrome has implemented it so far.

[1]: https://wicg.github.io/private-network-access/

… or you can instead phase out those browsers who try to force blocker restrictions i.e. spyware on you (e.g. chrome and such), and use one of the browsers where you can use the full-featured (not "lite") uBlock Origin instead, e.g. Firefox.

It’s only being phased out on Chrome, by Google.

You can't change browser? Or is there something bigger happening?

Just checked, and it seems like it is. Not enabled by default for some reason.

It is not being phased out for Firefox.

> Firefox doesn't implement either approach -- I assume this is indicative of their lack of development resources.

Since ublock had this as a feature for a long time, I'm sure they are aware of it. Unlike other non funded oss projects, Firefox can't and shouldn't shield themselves with this lack of development resource excuse. They have millions.

It doesn't handle it. Anyway, there's no way to know what a website does on the server site. Even a completely static website could be sending the server logs somewhere.

There are options to not load JS, images, XMLHttpRequests, frames, cookies, for each site, but it doesn't list individual files.

But uBlockOrigin UI is so much worse...

Besides, uMatrix works fine. It's that kind of program that doesn't need any updates.

Until uBO has an even remotely usable interface for this use case people (including myself) will continue to use uMaxtrix or forks of it instead.

I reluctantly switched to only uBo because of uM bugs. But the UI/UX is just a huge step backwards to enable mobile usability.

uBO advanced settings still isn't as flexible as uMatrix was though, fwiw. (I did give in and switch in the end though.)

With uBO I can't block cookies by domain.

The requests are not made, because some operating systems prevent this.

If you're on OSX, the permission to "discover on the local network" prevents it from happening ( System Settings -> Privacy & Security -> Local Network -> yourbrowser )

Could also be 'network' permissions on firefox ( Go to Settings > Privacy & Security > Permissions ) which is on a per site level, but iirc that could be set site-wide at some point.

The other browsers likely have similar configs, but this is what I have found.

I have no ideea. Possibly that's a limitation of Chrome+Firefox developer tools (I get the feeling it's the same code)?

But I found what "burp" is: https://portswigger.net/burp/communitydownload

Chrome is already in the process of killing it https://developer.chrome.com/blog/local-network-access

Enable "Block Outsider Intrusion into LAN" filter list in uBlock Origin.

You should actually harden your browser or PC... to block any unwanted requests. Apparently some browser extensions can do that.

It would be the job of the operating system to give or take away the ability of your browser to access your local network. But you can run your browser in a container/vm and disable localhost. (And use a separate browser for localhost only if you need it.)

I looked at the website again and noticed that the request paths looked odd, one of them being `/400_random_url_with_numbers_403`. I googled that and it looks like it's part of a client-side bot detection script that's testing something, the explanation isn't very informative.

https://my.f5.com/manage/s/article/K000138794

> These requests are caused by the bot profile to test the different browser capabilities.

> 'http://127.0.0.1:xxxx' request is a call to the localhost/client machine, which is normal when trying to protect assets like end-server using ant-bot defense. It does not have any impact regarding application page load.

Why do you say that’s most likely?

This is a common pattern for connecting to smart cards / hardware security devices. Probably a service or hardware that’s run on official CBP machines that should be disabled for prod, but forgot.

How are you so sure?

Isn't CORS supposed to prevent this?

https://my.f5.com/manage/s/article/K000138794

It seems to be part of some "bot defense" product by these F5 people, to "test the different browser capabilities". I doubt it's intended to hit a real endpoint on any system.

Hopefully should soon be a thing of the past with https://developer.chrome.com/blog/local-network-access

I see. So the website would try to access private IP adresses (RFC 1918) by having elements like <iframe src="http://10.0.0.1"> in the web site and then the web site would check if the iframe was loaded successfully?

Why do you need a heavyweight extension to block sites from scanning your local network? Ridiculous.

Also I wonder if this protection is available only with old extension manifest version or new network request hooks API also supports it.

Pushing the burden of network permissions management outside the browser, to the OS? Heresy!

To be serious, this has introduced me to sandboxing on BSD via pledge [0] and comparisons against Linux seccomp [1] - thank you!

[0] https://news.ycombinator.com/item?id=17289654

[1] https://kristaps.bsd.lv/devsecflops/ (submission by same poster at https://news.ycombinator.com/item?id=44264021)

t-shirt worthy quote - "modern web design is a joke" ;)

It's also inaccurate, as this style of page (relating to layout and specific graphic style) didn't appear until 2006ish.

I wish gov.uk was even a smidgen as "outdated" looking as that page.

Not necessarily.

I might create a login for a porn site so that I can have some favorite videos bookmarked and it can figure out the type of material I like. That doesn't mean I want my history saved locally.

Not contradictory at all. These days private browsing for most people just means (1) don't save the browsing history and (2) log me out of all websites temporarily.