Consumer Reports VPN Whitepaper

[exponent 1]

Security and Privacy of VPNs Running on Windows 10

Written by Yael Grauer

48 pages in length.

AirVPN is mentioned on page 5.

Seemingly comprehensive coverage of VPN services.

Submitted for the edification of the community.

VPN-White-Paper.pdf

Edited ... by exponent
Accuracy.

[OpenSourcerer 1360]

Link to the original, please.

AirVPN is mentioned, yes, but somehow it was not tested because they found DoH was deliberately disabled. I should test that in Firefox, try to reproduce that. Edit: Oh, on page 46 they state they did test them all.
They also didn't find DNS manipulation in Mullvad, but it's there.

The focus on Windows is also wrong. The tester software is still not open source despite intentions from the past.

What this paper does well on the other hand is listing notable and not so notable controversies of all tested providers and shedding some light to the problematic ownership situation of the VPN provider market.

[Staff 9761]

Hello!

We consider bypassing DNS strictly picked by the user a bad practice, while the paper authors consider bad practice basically the opposite.

Anyway, it must be said that the paper says "doing so [disabling DoH} without reasons is considered poor practice". Of course AirVPN does it for a reason which in our opinion is strong and valid (you can infer it from the questions below), so the sentence should not be considered a negative evaluation of AirVPN software for Windows.

Please feel free to open a debate, we are very much interested in it. Some questions from us, we would be glad if you could answer freely:

• Should a browser be able to bypass your own DNS choices and hijack your queries to third-party DNS servers whose privacy, logging and neutrality policy you may or may not have read (*)?
• If you answer "yes" to the previous question, should it do it even without your knowledge by default, or should it offer this feature as "opt-in"?
• If you tell a VPN client to use the VPN DNS only, should the VPN client have the ability to disable browser's DNS bypass, or not?

(*) For example Firefox enables DoH even if you have not been informed about who are the third party DNS your queries end up to, by whom they are operated, and what their policies are.

Kind regards

 

[OpenSourcerer 1360]

20 minutes ago, Staff said:

• Should a browser be able to bypass your own DNS choices and hijack your queries to third-party DNS servers whose privacy, logging and neutrality policy you may or may not have read (*)?
• If you answer "yes" to the previous question, should it do it even without your knowledge by default, or should it offer this feature as "opt-in"?
• If you tell a VPN client to use the VPN DNS only, should the VPN client have the ability to disable browser's DNS bypass, or not?

1. Yes, because it cuts both ways: If the user chooses a global DNS and is oblivious to the fact his/her choice is worse than the previous DNS setting, the browser's DoH will save the user from a poor choice. This is happening kinda en masse here. The amount of users I've seen the sysconfigs of who set Google DNS as their global DNS provider is… significant. In such a case, blocking the default Cloudflare DoH will even harm users, despite good intentions.
   And if the user chooses to use DoH, on purpose, a network should not block that. I think that is also what the authors tried to say.
2. Without knowledge? No. The users should somehow be informed about defaults (like documentation, or "UI tutorials", one can get creative here), and the devs should always provide the means to change this default configuration (and probably also outline how to do so).
3. Analogous to 2, AirVPN could provide this by default. But the user should be given the choice to disable that. It would've satisfied the whitepaper authors, I'm sure of it.

52 minutes ago, Staff said:

(*) For example Firefox enables DoH even if you have not been informed about who are the third party DNS your queries end up to, by whom they are operated, and what are their policies

It's something I'd really call /r/assholedesign. Android does that, too, by the way. At least both projects do offer a way to disable that.

[Staff 9761]

23 hours ago, OpenSourcerer said:

1. Analogous to 2, AirVPN could provide this by default. But the user should be given the choice to disable that. It would've satisfied the whitepaper authors, I'm sure of it.

Hello!

Thank you for your answers. There's just a terrible misunderstanding we can infer from one of them though (quoted), which we would like to fix.

The user can, and have always been able to, disable this feature by not using Air VPN DNS, as usual. It takes a few seconds to configure it in Eddie Windows edition. By not using Air VPN DNS they will have no more NXDOMAIN returned by "use-application-dns.net" resolution (unless of course they force some other DNS that does ), as specified in our https://airvpn.org/specs  page.

What the authors of the paper consider a problem is probably caused by the fact that they don't like that the feature is "opt-out". But we need it otherwise we would have hundreds (thousands?) of customers complaining (and rightly so!) of alleged DNS leaks, complaining that DNS block lists don't work, complaining that geo-routing doesn't work. It's our opinion that the current implementation is good design, not poor design as claimed by the paper authors, whose consideration is frankly very questionable, again in our opinion.

Our case is exactly foreseen and described by Mozilla in the list of cases for which default DoH in Firefox must be disabled. "Risks: [...] When enabling DoH by default for users, Firefox allows users (via settings) and organizations (via enterprise policies and a canary domain lookup) to disable DoH when it interferes with a preferred policy. ". In our case the preferred policy is letting the users take advantage of geo-routing, DNS block lists, as well as defusing the extremely dangerous route hijack attacks by making the DNS server address matching the VPN gateway address, and providing users with peace of mind when they test for "DNS leaks" through web sites etc. Such strong AirVPN features should remain "opt-out" and not become "opt-in", as they are not only a fairly required part of AirVPN features but also an important security addition.

Those who still want Firefox DoH can simply disable DNS check and reject DNS push, or force any public DNS in Eddie, because they would not use VPN DNS in any case, obviously.

Kind regards
 

[OpenSourcerer 1360]

4 hours ago, Staff said:

There's just a terrible misunderstanding we can infer from one of them though (quoted), which we would like to fix.

It's actually not a misunderstanding, I simply should've been a little clearer. I didn't mean the choice of DNS servers, and people will probably know that they will pass on AirDNS features like double-hop if they choose to use custom DNS servers. I mean the choice to enable the resolution of that DNS name if the choice is to use AirDNS, purely to cater to the case, edge or not, of deliberately enabling the use of DoH.
 

4 hours ago, Staff said:

What the authors of the paper consider a problem is probably caused by the fact that they don't like that the feature is "opt-out". But we need it otherwise we would have hundreds (thousands?) of customers complaining (and rightly so!) of alleged DNS leaks, complaining that DNS block lists don't work, complaining that geo-routing doesn't work. It's our opinion that the current implementation is good design, not poor design as claimed by the paper authors, whose consideration is frankly very questionable, again in our opinion.

I share your opinion, this decision totally makes sense for what AirVPN is all about. But in this case you don't leave the users a middle ground, only the two "all or nothing" choices "use AirDNS, block DoH" and "use external DNS, where queries go is your responsibility". Let me express it like this:

• I trust AirDNS, I don't want my queries to be seen over the internet.
  • AirDNS is used. Queries are encrypted up to the VPN server and end there. DNS leak via DoH prevented. The "default" choice.
• I don't trust AirDNS, and I accept that my queries can be seen over the internet.
  • External DNS is used. Queries are encrypted up to the VPN server and unencrypted after that. The "user knows what he/she's doing" choice.
• I don't trust AirDNS, but I don't want my queries to be seen over the internet, either.
  • DoH is used. Queries are encrypted throughout to the DoH server, independent from the underlying transport.
  • AirDNS blocks it (case 1). And using external DNS exposes queries from all software not using DoH (case 2). Ergo, the "impossible" or "dilemma" choice.

Your default must remain the default, this we definitely share. But I think one could use the new DNS adblock feature with a dedicated "list" choice of blocking the resolution of that domain or not. To describe it: "Enable this list if you want to block DoH. DoH is [this] and it does [that]. We recommend turning it on because some apps like Firefox can otherwise ignore our DNS. If you trust your DoH server more than our DNS server and want your queries to be encrypted beyond the VPN server, disable it."
Really hope this makes some sense, though. Maybe it's blatant overthinking on my part…?

[Staff 9761]

@OpenSourcerer

Hello!
 

Quote

But in this case you don't leave the users a middle ground, only the two "all or nothing" choices "use AirDNS, block DoH" and "use external DNS, where queries go is your responsibility". [...] one could use the new DNS adblock feature with a dedicated "list" choice of blocking the resolution of that domain or not.

There is no middle-ground, either you use VPN DNS or you use Firefox DoH and renounce to some AirVPN features. Well, to say it all you can also use DNS over HTTPS or DNS over TLS with the Air VPN DNS, they are both supported, but it does not make much difference because the DNS queries remain in the tunnel even without DoH or DoT (anyway we support them both for the comfort of those users who set peculiar configurations).

Surely we can improve the description, but it's not that simple, i.e. it can't be limited to DNS block lists, as other considerations such as geo-blocking feature and route hijack attack immunization should be described when one renounces to VPN DNS. We enter a sticky situation, because the advanced user already knows it all, while the beginner might be unable to understand it from a synthetic description and might join that choir of donkeys (initially formed because we refused and refuse to pay ransoms for reviews) braying that AirVPN is too complex to use.
 

Quote

Your default must remain the default, this we definitely share.

Excellent.  The paper authors can't understand or accept our point of view (which pj explained in private before the paper was published when queried, but no replies came in after the explanation) but it doesn't matter, what it matters most is that our core community, advanced and/or long time users, understand why our choice makes sense and is based on good design.

Kind regards
 

[Lalofaw 0]

I don’t think the authors of the paper declined to test because of DoH. It seems like they picked VPNs for further testing based on market share, lack of leaks, and use of DNS proxy. And were impressed with open source software, third party audits, ad copy, etc.