Staff

@colorman Hello! We finally managed to understand how to reproduce the issue. We are working on it. In the meantime you should be able to circumvent the problem in any of the following ways. Please check if you have time. specify your network lock favorite method via networklock or networklockpersist directive in file /etc/airvpn/bluetit.rc - then you don't need to specify it on goldcrest command line. You can edit, with root privileges, /etc/airvpn/bluetit.rc with any text editor (example sudo nano /etc/airvpn/bluetit.rc). Enter anywhere the line networklock on, or even networklockpersist on then save the file alternatively, connect without a profile, for example goldcrest --network-lock on --air-connect -air country <your favorite country> (you may enter your AirVPN credentials on the terminal when Goldcrest asks for them, or set them in Goldcrest configuration file) Kind regards

@colorman Hello! We are struggling to reproduce the issue, openSUSE 15.2 is one of our testing systems for AirVPN Suite and we fail to understand why we can't get the same behavior you experience. We have tested both with firewalld enabled and disabled, both with iptables-legacy and nft 0.8.2 (all combinations) and everything went smoothly. In other words, network lock is engaged and disengaged correctly, either with iptables-legacy or nft, tested both with firewalld running and not running. Can you please try again network lock with: - firewalld disabled (not running at all) - by enforcing iptables-legacy (networklock or networklockpersist set to iptables) - then, by enforcing nftables (networklock or networklockpersist set to nftables) We are looking forward to hearing from you. Kind regards

@183aTr78f9o Hello! You can find an explanation and the immediate solution for you needs, which takes just a few seconds, in our previous message.. In this way you can have network lock even when you press a hard reset button or you power cycle the machine while it's working at your own risk.. The fact is that a daemon must never perform root operations if the lock file exists Ignoring this simple but important rule would be terrible, for quite obvious reasons. Never mind if you can't understand why right now: remove the lock file and you will have the behavior you want. If you read the whole documentation there should be no misunderstanding. What it matters is underlining that Bluetit is so flexible that even in this case you can achieve your purpose simply, in the way we described. Sure, if it suits your needs why not. Just remember that Hummingbird is not a daemon and keep in mind the architectural difference, to understand the risks. No reason to be sorry, you were never asked to in the first place. Those questions were rhetorical ones to clarify that in our opinion Bluetit should not take into account all the possible combinations and try to enlarge its scope, because that's strictly a matter of other system components. Given the non-uniformity in behavior of hundreds of Linux distributions, and their ever changing and evolving behavior, we believe it's a wise choice not to interfere in other processes duty. Therefore, we can close the issue, the behavior is correct and expected, no bugs here. Kind regards

@colorman Hello! We see that you get some error even when Bluetit enforces network lock through iptables-legacy. However, from the log it's unclear whether those errors are irrelevant or not. While Bluetit is still connected with network lock on (with iptables-legacy, as it is in your last log you sent us) can you send us the output of the command iptables-save ? About nft 0.8.2, it's an obsolete version. However, Bluetit now handles it correctly (tested in Debian 9 and openSUSE 15.2) - let's see first iptables-legacy anyway. Kind regards

@183aTr78f9o Hello! Behavior 1 is expected: when you enforce a savage reboot by pressing the hardware button, bluetit.lock fie in /etc/airvpn is not deleted (not even by intrusive systemd, as it happens normally), so Bluetit needs to exit at the next run, necessarily: another instance might be running - and anyway the anomalous situation must be investigated by root. If you want to force Bluetit start in any case without verifications, you just need to delete /etc/airvpn/bliuetit.lock first if it exists (re) start Bluetit finally run goldcrest --recover-network In this way Bluetit will start in any case. Then, if it finds a dirty situation for the DNS and firewall backup files, it will remain running (it would exit only if the lock files existed): then Goldcrest will ask Bluetit to recover network, Bluetit will comply (if it's in a clean situation it will simply ignore the network recovery order) and immediately after will activate network lock (if newtorklockpersist is specified) and also connect if connectatboot <mode> is specified. Remember, therefore, not to postpone Bluetit (re)start command with goldcrest --recover-network command, otherwise Golcrest will find no service to contact via D-Bus in your context. Issue 2 is related to how a system handles suspend/resume: is the network interface powered off? If so, at resume is network re-initialized? If network interface kept working, is network initialized anyway, or do we still have the routing table set by the VPN client according to the server push? What about firewall rules and tun status? We leave the behavior to each system, ideally Bluetit should not take care of all the possible combinations. Your proposed solution is fine, if we come out with something better we will let you know. A possible weakness of your solution is that the "wake up" unit you defined might start BEFORE Bluetit runs, in some anomalous situation. Thank you very much for your tests! Kind regards

@John Gow Hello! It looks like more a "picking the right server for you" problem than an overload or load balancing problem. We also don't see any overload in US or CA West Coast. Can you define a while list of your favorite servers? It should meet your needs. You can define the list in Eddie's "Preferences" > "Servers" window. Multiple selection (CTRL + click and SHIFT + click) is allowed for your comfort. Once a white list is defined, Eddie will consider only servers in that white list. Kind regards

Hello! @colorman Goldcrest invocation appears correct. The full Bluetit log might be more helpful, can you please publish it? Also, can you tell us your nft version (command "nft --version")? @183aTr78f9o The 1st reported behavior is expected. because Bluetit must not proceed when it is in a "dirty condition", but it remains to be seen why Bluetit did not exit cleanly in its last session. Was it intended as a test (i.e. you explicitly wanted a dirty startup to test), or is it unexpected? (OK, you already wrote it was a test ). Once you have issued goldcrest --recover-network you don't need to restart Bluetit, you can directly start a connection. Also, Bluetit will activate network lock immediately after a network recovery, if networklockpersist is set in bluetit.rc - another good reason to avoid a restart. The 2nd remark describes an unexpected event. What happens after you resume the system? Can you send us the full Bluetit log taken just after the problem has occurred? The expected behavior would be that Bluetit daemon does not exit at all, in the first place, when you order an hibernation. Let's see what kills it, if possible. @leori Thank you, we're very glad to know it! @air2157 We're glad to know that Hummingbird launch by Eddie is fine again. As explained, up and down scripts with script-security 2 enlarge attack surface and we think that it's very appropriate that a library and a root process (Hummingbird) do not offer them. Even in Eddie, script-security, up and down custom directives have been forbidden a long ago for critical security reasons. Furthermore Eddie does not even allow you to use an arbitrary external ovpn profile, and we're considering to stop supporting it in Goldcrest/Bluetit too. About the documentation: please note that, just like it happens with any documentation, we document implemented features, not unimplemented ones. Bluetit developer's documentation is also on its way and we plan to publish it immediately after AirVPN Suite 1.1.0 stable has been released. As you know we have forked OpenVPN3 so we don't need to wait for bug fixes from the main branch if they are too slow to come or they are urgent. The parts which were not ready for production in our tests have been fixed and/or rewritten already, as these last 3 years of releases and testing in different systems should have shown. In any case, please feel free to point us to those source code parts that you deem as not ready for production at your convenience. Thank you for your previous tests, they have been much appreciated. Kind regards

Hello! We're glad to inform you that AirVPN Suite 1.1.0 RC 4 is now available. Download URLs have been updated accordingly in this thread first message. Please check the changelog for a full list of important changes, which in this last time frame between RC3 and RC4 have been particularly numerous. The latest problems found by our internal and public testers have been reproduced and addressed. Thank you all, you brought to light various, elusive bugs which slipped unnoticed through all the previous testing phases. AirVPN Suite 1.1.0 RC 4 includes new OpenVPN AirVPN library 3.7 fixing a couple of issues coming from the main branch causing the library to crash under the conditions found by our community testers and reported in this thread: thank you all! Another issue in the library was the confusion between "proto tcp-client" and "proto tcp" in OpeNVPN 3 main branch, which caused a critical problem as reported in this thread by testers. OpenVPN AirVPN 3.7 is now 103 commits ahead of the main branch, featuring several, new important features and very many bug fixes. https://github.com/AirVPN/openvpn3-airvpn Please keep testing RC 4! Kind regards AirVPN Staff

@buy airvpn Hello! The issue you report does not depend on DNS. You would have the same outcome even if the machine could resolve names. When Bluetit is in a "dirty" status (*), a manual intervention is required, either by a an airvpn group user with goldcrest --recover-network command, or by root, who can decide for an automatic or a manual clean-up. Enabling Bluetit to perform automatically a clean up at startup may lead to potentially dangerous situations. It can't know whether the dirty condition has been caused by a power outage (in which case an autonomous clean up would be harmless) or some other cause which requires human analysis before proceeding to restore settings "blindly". For the reckless system administrators we might consider in the future to enable Bluetit ability to perform autonomously a recover-network (opt-in only) at startup. Anyway you can achieve the same purpose right now by sending automatically a recover-network command to Bluetit always (even during system bootstrap) at startup (via Goldcrest for example). If no recover-network is necessary, Bluetit will just not proceed to do it, and if it's necessary Bluetit will do it. After that, you do NOT need to restart Bluetit, you can send an air-connect command: if a connection is already established, the command will be ignored, otherwise the connection will be performed. We do not feel to recommend the above, but at the end of the day it's up to each system administrator. Kind regards (*) Bluetit is in a "dirty status" when it finds in /etc/airvpn directory one or more of his system settings backup files which would not have remained there if the previous Bluetit shut down had been successful. If you lose power while Bluetit is connected, clearly all of those files are still there.

Hello! Thank you. The problem is in the OpenVPN3 profile parser unfortunately. We have completely rewritten the option parser but not the profile parser. Maybe we'll do it in the future, as the bug is there, but not before the 1.1.0 Suite release. Maybe in the meantime the bug will be fixed in the main branch, but yes, profile parser needs a deep rewrite, which is not a trivial job. So, at the moment, you need to use persist-tun as an option, so it will be handled by our option parser and will work as expected. Right. We'll document it, in the meantime remember it. OpenVPN3 main branch in Linux fails any re-connection of any sort last time we tried, for a critical bug we fixed which messed up routing table. OpenPVN3 AirVPN handles re-connection with persist-tun only (otherwise the tun interface is brought down and never brought up again). We can't rule out that in the future we'll investigate this other failure too but it has low priority now because tun persistence during a re-connection is a feature that should be enforced for a good reason. Anyway, as you can see bug fixes from the main branch proceed at a sustained pace in our fork, but bug discoveries in the main branch proceed quickly too. We can't humanly fix bugs which accumulated in a decade, or so, all at once, but we will do it one by one according to priorities. Please keep reporting and we'll keep fixing according to priorities. About compatibility.... Compatibility with OpenVPN 2 servers is a high priority of ours and our latest OpenPVN3 AirVPN version is right now much more compatible than OpenVPN 3 main branch, but compatibility with choices on the client side we do not agree with may or may not be maintained (more on this below). Re-connection means "re-connect to the same server". It does not mean disconnect, pick a new server, if it's different than the previous one connect to it otherwise repeat the server pick. Currently you can't reliably have any "randomization" with a re-connection, instead you can expect normally quite the opposite. At the moment you don't have a way to efficiently pseudo-randomize connections from inside Hummingbird by sending it any signal, we're sorry. OpenVPN3 has never supported up / down scripts. We might suppose that, perhaps, the missing feature is due to security considerations. Up and down scripts have been historically the tool for exploits, for example privilege escalation, as they allow an attacker to run with root privileges her own script or binary pre-installed without root privileges. Support to run external scripts or binaries from our library is currently not planned. Running external scripts from Bluetit is currently not planned as well, as Bluetit is a daemon: running any binary or script external to it is a practice which should be avoided when possible, as it enlarges the attack surface. Of course nothing prevents you to control Bluetit in ways which allow execution of your scripts and binaries when certain events occur. Developer guide will be published soon after we reach 1.1.0 stable release. Hummingbird, Bluetit and Goldcrest handle signals, or any other thing, according to our logic and considerations, with no constraints coming from any OpenVPN 2 or OpenVPN 3 tradition. For example, OpenVPN 3 has maintained and maintains several incompatibilities with OpenVPN 2 servers various versions, and we have decided to resolve them. In general, you can expect that OpenVPN 3 AirVPN library aims at maximum compatibility with OpenVPN 2 servers, while AirVPN programs Suite (Hummingbird, Goldcrest and Bluetit) are not tied to OpenVPN traditions, or to OpenVPN itself. In the future they might make use of protocols, libraries or kernel modules different than OpenVPN 3 AirVPN, of course. SIGHUP to OpenVPN causes a hard re-start, while Hummingbird exits cleanly when it receives that signal.. Thank you all for your tests again, we have been working on a new Release Candidate since weeks ago, together with your directions and bug findings, and it is almost ready, stay tuned! Kind regards

@air2157 Hello! We should have spotted the problem. The firewall is not the problem, apparently, but the fact that --persist-tun works fine when used as a Hummingbird option, while persist-tun as an OpenVPN directive causes the problem. We could reproduce the problem: when we have persist-tun merely as a profile directive, the tun interface goes down at the re-connection (as if the directive were ignored), but if --persist.-tun is added as an Hummingbird line option, then tun persistence is respected. Thank you for the report, the bug is now under investigation. Can you test with the option, as a cross-check? Kind regards

@air2157 Hello! --persist-tun option is mandatory to perform a successful re-connection. Without persist-tun you should lose the tun interface (then routing table as well as Network Lock prevent leaks, that's why you note you lose connectivity). Can you please re-check? We ask because we can't reproduce the issue. Please note that without persist-tun, it's expected that re-connection fails and your network is locked. Note that Bluetit keeps persist-tun on by default, while Hummingbird keeps persist-tun off by default. Kind regards

@TheMicape Hello! Unfortunately there is no coordination between different Linux distributions, so you find different, and often mutually incompatible, configurations, package names, package managers, directory structures, ilibraries and library names, init systems, DNS handling and so on. You find distributions which in their latest version use packages and/or kernel which are 10 years old (even the clib, just to say...), distributions which run a few years old packages and kernel, and distributions on the bleeding edge. Some distributions differ from each other so deeply that in a sense they can be considered different Operating Systems (today not even built on a common kernel anymore). Out of curiosity have a look here: https://distrowatch.com/images/other/distro-family-tree.png https://distrowatch.com/images/other/periodic-table-of-distro.png That's to say that it's difficult nowadays to develop something "for Linux" with any little complexity degree that will run out of the box on every and each distribution, Not even by statically linking everything (which is anyway an extreme and not an elegant solution....) you can be sure to reach a high compatibility degree. Actually you can see that since years ago, some software houses tend not to say anymore that their software is "for Linux", but for some specific distribution ("for Ubuntu", "for Debian/Ubuntu", "for openSUSE", and so on). To mitigate the problem, we offer an AppImage for Eddie. Try it if libappindicator1 is not available in your system. AppImage should run out of the box in at least eight major distributions and most of all of their plethora of derivatives. https://appimage.org/ Eddie AppImage can be found in the usual Linux download page: https://airvpn.org/linux As a further alternative, you can consider the AirVPN Suite for Linux, but only if you don't need a GUI. https://airvpn.org/suite/readme/ A formidable effort has been put both on Eddie and on the AirPVN Suite to let them work out of the box in an exceptionally wide amount of distributions and architectures, but of course anything can be improved. Please consider that about one thousand Linux distributions were around in 2020. Can you tell us your distribution name and exact version? Kind regards

@Display Name REQUIRED Hello! Eddie desktop edition sends simultaneously a lot of ICMP packets to different destinations (all the VPN servers in your white list, or all AirVPN servers if a white list is not defined). Such a behavior is sometimes forbidden by some ISPs which will block ICMP packets after a certain threshold is reached. However Eddie should not get stuck and it should go on after each request times out, so it could be a bug which deserves investigation, thank you for the report. In the meantime, try this: disable latency tests and the problem should be gone. From Eddie's main window select "Preferences" > "Advanced", then de-tick "Enable latency tests" and finally click "Save". What Eddie calls "latency tests" are a measurement of the round trip time via ping, so by disabling them you should cut out the problem roots. When you disable latency tests Eddie is not able to pick a really good server for your node (you might find its choice of a "recommended" server bizarre, because it does not know anymore the round trip time from your node to the VPN servers), so take care to define a white list of servers according to your preferences, or pick servers or countries manually. Kind regards

@air2157 Hello! We confirm that Hummingbird does not enforce restrictions on profile name and that OpenVPN 3 enforces restriction on the suffix of the file name. We managed to reproduce the issue easily, even when a merge is not requested. The merge.hpp class has been modified very recently (25 days ago) and that might perhaps explain why the problem has never been reported in years. We will seriously consider to remove this limitation. Kind regards

Hello! Correct. Of course it's not that systemd has been fixed, but the problem has been fixed in the sense that no workaround is anymore necessary because now Bluetit verifies by itself the connectivity link, so it does need to trust anymore any init system. Only when the link is up (network interface up WITH a default gateway according to the kernel, and not according to anything else) Bluetit starts network operations, thus avoiding any error. The matter became relevant because default Raspbian and OSMC settings cause systemd to run those services which are targeted to start only after the link is up even when the link is not up, a mixture of bad systemd behavior and questionable settings which expose such behavior. Kind regards

Hello! Of course. Not only OpenVPN-AirVPN code is clean, clearly commented and can be maintained efficiently (Donald Knuth is an ideal master of the AirVPN suite developer chief ) but we also submitted pull requests initially for the master branch, before we forked.. An OVPN3 project leader and some other person reminded us that pull requests could not be accepted if AirVPN did not disclose the real identity of its developers, and if AirVPN did not accept the re-licensing clause specified in the CLA.rst contract stating that OpenVPN Inc., and only OpenVPN Inc., can re-license our code under any other license, including "non open" licenses (Part, II, https://github.com/OpenVPN/openvpn3/blob/master/CLA.rst) Both conditions were and are not accepted by AirVPN, and we were forced to fork. Therefore, upstream must be performed by third-parties or by OpenVPN Inc. itself, but remember that we do not agree to the mentioned contract, especially in the part which states that code may be re-licensed by OpenVPN Inc. under any other license. Currently, the following modifications by AirVPN can be of particular interest for the main branch: the (N_)RECONNECT bug fix in Linux the data channel cipher selection bug fix the new implementation of Data Channel cipher negotiation compatible with OpenVPN 2.5 new method as well as the older OpenVPN 2 implementation the "ncp-disable" directive implementation (it comes handy with older than OpenVPN 2.5 servers) the "data-ciphers" directive implementation the fix of the "explosion" bug triggered under apparently random circumstances, in reality due to the unbelievable lack of data structure initialization in at least five different parts of the code the cleaner rewrite of the options parser getting rid of a good amount of parsing bugs the new method to select the correct cipher for each packet with pointer to methods in place of a dumb, and slower, series of "if" in the main branch which are executed at every and each outgoing or incoming packet block (useful when it's linked against mbedTLS) many other bug fixes which would make this message too long for this thread - being 103 commits ahead, you really have a more stable, more OpenVPN 2 compatible, and faster library now Kind regards

Hello! Acknowledged. The documentation is not only online, it's included in the package too so you can consult it anytime, even if you're offline. Kind regards

@air2157 Hello! We think you're right. If confirmed, it does not seem as an Hummingbird limitation at all: we must check twice but at a first glance we have dragged such a limitation from the main branch of OpenVPN 3 library, specifically here: https://github.com/OpenVPN/openvpn3/blob/master/openvpn/options/merge.hpp when OpenVPN3 wants to merge additional profile (about which, we will have some questions for you after some tests, stay tuned ). We have already widely rewritten critical OpenVPN 3 parts in our fork, fixed dozens of bugs and conceptually wrong code, added new key features and essential directives, and we will rewrite even this part if it comes out that it's a counter-productive implementation by the main branch developers that we missed. Again, we will not hesitate to rewrite other code whenever the main branch code is inappropriate, after we have consulted with the development leader, as we have always done. Thank you for having found out the issue and reported it to us. @OpenSourcerer In this case the documentation is correct, as Hummingbird does not enforce limitation on the file name. Work on Hummingbird documentation has received excellent feedback (dozens of positive comments), readers please check here: https://airvpn.org/suite/readme/ https://airvpn.org/hummingbird/readme/ However, and obviously, anything can be improved. Is there anybody else here who can give us an additional feedback about Hummingbird & Suite documentation? OpenVPN3 main branch is undocumented while its source code is poorly commented, so it may have happened that a peculiar feature, especially when it is as "strange" as it seems in this case, has been missed. Luckily, nothing that justifies the "sloppy" adjective, in our opinion. Remember that we offer a README.md specific to HB which is much more complete than a man page (you can't of course stuff the whole readme.md in the man pages). man pages had been planned for the AirVPN Suite and not for Hummingbird, since Hummingbird had been considered a binary which can be launched by Eddie, while as a stand alone binary it was just transitional software to show what our AirVPN OpenVPN 3 library could do, before we had a real daemon, Bluetit, whose version 1.1.0 release has now the priority. Of course now that Hummingbird appears to have reached such a wide appreciation and success (which we did not expect) we may change our plans, in order to maintain Hummingbird not only as an Eddie alternative to OpenVPN (Hummingbird in Intel Mac and in M1 Mac provides a 2x throughput boost against OpenVPN 2.5), but also as a part of the Suite. Next step is publishing the developer's manual for Bluetit, our first and only real daemon, which we deem as very important. Then we can consider man pages specific to the Suite and in that occasion we can re-think about Hummingbird man pages, because at the of the day we offer a much more complete documentation than a man page. We totally agree with you. If the restriction is confirmed OpenVPN3 main branch developers might be asked about why they think it is appropriate to enforce it, anyway we have seen worse things in the original source code. On our side, we will verify the problem and we will remove it if confirmed, luckily we forked OpenVPN 3 a long ago so we have no constraints. Our fork is now 103 commits ahead the main branch, and we assure you that we are determined to wipe out any other OpenVPN3 main branch "absurdity", according to your definition, we find. Kind regards

@lostagain Hello! Trivially because when your system is in the virtual network, it is also in the real network provided by your ISP. You will not receive unsolicited packets from the VPN if you don't forward ports remotely, but you can still receive them from your ISP network. Network Lock will block those incoming packets but that will not prevent a DDoS of course . A better solution is that you do not forward packets from your router to your systems when your systems are in the VPN. Also remember that if you forward remotely ports, you will be protected by the global anti-DDoS system on the server datacenter, and you will not have specific traffic inspection for your forwarded packets to filter them against attacks. We do not inspect our customer packets by policy and our network remains agnostic. Kind regards

@OpenSourcerer @WinniethePooh Hello! Yes, that 50.x.y.z is an address of one of our micro-routing servers. Certainly Binance must have been "micro-routed" for some anti-blocking purpose after customers' requests. This information should clarify and resolve the "enigma". Kind regards

@pjnsmb Hello! gb3.ipv6.vpn.airdns.org is updated correctly, so Bluetit will connect to the correct server that domain resolves into. The failure to resolve names during Bluetit start at bootstrap but the ability to do so afterwards remains a problem which we could not reproduce so far. It was detected in Raspbian and OSMC, due to their settings which cause systemd to run Bluetit prematurely, and fixed by implementing in Bluetit its own network link detection. Kind regards

@pjnsmb Hello! It has been sorted out, please try again now (for the readers: anyway not a Bluetit issue). Kind regards

@colorman Hello! Another piece of info please. When you run Hummingbird (and it "explodes"), do you have Bluetit daemon running as well? Kind regards

@colorman Good! Now network lock works with Goldcrest/Bluetit, when you force nftables. Momentarily, do not forget to force it, otherwise network lock will fail. We will be investigating on this bad issue. Correction: it does not work with nftables as well. About Hummingbird, thank you: we see that Hummingbird crashes when it is invoked in that way, as Eddie does, so we have now a track to understand what happens. We'll investigate on it as well, of course. Kind regards