ANSWERED Ain, Sweden server overloaded?

[Stalinium 47]

Quality of Service post: Although it is a 10Gbit server it seems to be suffering. I'm from Europe and checking its ping shows average 344ms over 234 attempts. This puts it in range of JP and NZ servers at 350ms. This is not the first time I've seen it perform poorly. Last week I connected to se.vpn.airdns.org (and it still shows as the preferred choice in API and https://airvpn.org/status/ EDIT: not any longer as of finishing writing) and had the same problems downloading a file with speed jumping up and down.

To get the ping results I filtered the API JSON into an IP list to use with Nirsoft PingInfoView: https://www.nirsoft.net/utils/multiple_ping_tool.html
Current IP list: (if you read this post at a later date: it is not up to date!)

sortscript.sh | sort -k2

185.9.19.106 at, Alderamin (Austria, Vienna; 1000)
37.120.155.178 at, Beemim (Austria, Vienna; 1000)
217.64.127.194 at, Caelum (Austria, Vienna; 1000)
194.187.251.90 be, Capricornus (Belgium, Brussels; 1000)
91.207.57.114 be, Castor (Belgium, Brussels; 1000)
194.187.251.114 be, Columba (Belgium, Brussels; 1000)
194.187.251.162 be, Diadema (Belgium, Brussels; 1000)
194.187.251.154 be, Mebsuta (Belgium, Brussels; 1000)
82.102.23.130 bg, Apus (Bulgaria, Sofia; 1000)
82.102.23.138 bg, Grus (Bulgaria, Sofia; 1000)
45.162.229.146 br, Lalande (Brazil, Sao Paulo; 1000)
45.162.228.170 br, Peony (Brazil, Sao Paulo; 1000)
184.75.223.210 ca, Agena (Canada, Toronto, Ontario; 1000)
162.219.176.2 ca, Alhena (Canada, Toronto, Ontario; 1000)
184.75.221.202 ca, Alkurhah (Canada, Toronto, Ontario; 1000)
104.254.90.202 ca, Aludra (Canada, Toronto, Ontario; 1000)
184.75.221.114 ca, Alwaid (Canada, Toronto, Ontario; 1000)
184.75.221.170 ca, Alya (Canada, Toronto, Ontario; 1000)
184.75.221.162 ca, Angetenar (Canada, Toronto, Ontario; 1000)
184.75.221.210 ca, Arkab (Canada, Toronto, Ontario; 1000)
184.75.223.234 ca, Avior (Canada, Toronto, Ontario; 1000)
184.75.214.162 ca, Cephei (Canada, Toronto, Ontario; 1000)
104.254.90.234 ca, Chort (Canada, Toronto, Ontario; 1000)
104.254.90.242 ca, Enif (Canada, Toronto, Ontario; 1000)
104.254.90.250 ca, Gorgonea (Canada, Toronto, Ontario; 1000)
87.101.92.170 ca, Lacerta (Canada, Montreal; 1000)
184.75.221.2 ca, Lesath (Canada, Toronto, Ontario; 1000)
184.75.223.218 ca, Mintaka (Canada, Toronto, Ontario; 1000)
192.30.89.66 ca, Nahn (Canada, Vancouver; 1000)
192.30.89.26 ca, Pisces (Canada, Vancouver; 1000)
184.75.221.34 ca, Regulus (Canada, Toronto, Ontario; 1000)
139.28.218.234 ca, Ross (Canada, Montreal; 1000)
104.254.90.186 ca, Rotanev (Canada, Toronto, Ontario; 1000)
184.75.221.178 ca, Sadalbari (Canada, Toronto, Ontario; 1000)
184.75.223.226 ca, Saiph (Canada, Toronto, Ontario; 1000)
184.75.223.194 ca, Sargas (Canada, Toronto, Ontario; 1000)
192.30.89.74 ca, Sham (Canada, Vancouver; 1000)
104.254.90.194 ca, Sharatan (Canada, Toronto, Ontario; 1000)
184.75.221.42 ca, Sualocin (Canada, Toronto, Ontario; 1000)
137.63.71.50 ca, Tegmen (Canada, Toronto, Ontario; 1000)
184.75.221.194 ca, Tejat (Canada, Toronto, Ontario; 1000)
192.30.89.50 ca, Telescopium (Canada, Vancouver; 1000)
192.30.89.58 ca, Titawin (Canada, Vancouver; 1000)
184.75.223.202 ca, Tyl (Canada, Toronto, Ontario; 1000)
184.75.221.58 ca, Ukdah (Canada, Toronto, Ontario; 1000)
185.156.175.170 ch, Achernar (Switzerland, Zurich; 1000)
185.156.175.34 ch, Achird (Switzerland, Zurich; 1000)
185.156.175.50 ch, Baiten (Switzerland, Zurich; 1000)
195.206.105.226 ch, Dorado (Switzerland, Zurich; 1000)
185.156.175.42 ch, Hamal (Switzerland, Zurich; 1000)
91.214.169.68 ch, Kitalpha (Switzerland, Zurich; 1000)
195.206.105.202 ch, Sextans (Switzerland, Zurich; 1000)
185.156.175.58 ch, Sirrah (Switzerland, Zurich; 1000)
46.19.137.114 ch, Virginis (Switzerland, Bern; 1000)
79.142.69.159 ch, Xuange (Switzerland, Zurich; 10000)
185.156.174.114 cz, Centaurus (Czech Republic, Prague; 1000)
185.156.174.26 cz, Markab (Czech Republic, Prague; 1000)
185.156.174.154 cz, Turais (Czech Republic, Prague; 1000)
89.238.166.234 cz, Zuben (Czech Republic, Prague; 1000)
185.104.184.42 de, Adhara (Germany, Frankfurt; 1000)
141.98.102.186 de, Alsephina (Germany, Frankfurt; 1000)
185.189.112.26 de, Cervantes (Germany, Frankfurt; 1000)
37.120.217.242 de, Cujam (Germany, Berlin; 1000)
141.98.102.242 de, Dubhe (Germany, Frankfurt; 1000)
185.189.112.10 de, Errai (Germany, Frankfurt; 1000)
178.162.204.227 de, Intercrus (Germany, Frankfurt; 1000)
141.98.102.226 de, Menkalinan (Germany, Frankfurt; 1000)
79.143.191.166 de, Mesarthim (Germany, Munich; 1000)
141.98.102.234 de, Mirfak (Germany, Frankfurt; 1000)
141.98.102.178 de, Mirzam (Germany, Frankfurt; 1000)
185.189.112.18 de, Ogma (Germany, Frankfurt; 1000)
178.162.209.151 de, Serpens (Germany, Frankfurt; 1000)
178.162.204.219 de, Tucana (Germany, Frankfurt; 1000)
178.162.204.222 de, Veritate (Germany, Frankfurt; 1000)
185.195.237.202 ee, Alruba (Estonia, Tallinn; 1000)
185.183.106.2 es, Eridanus (Spain, Barcelona; 1000)
185.93.182.170 es, Mekbuda (Spain, Madrid; 1000)
194.99.104.34 es, Taurus (Spain, Madrid; 1000)
185.103.96.132 gb, Alathfar (United Kingdom, Maidenhead; 1000)
217.151.98.162 gb, Alshain (United Kingdom, London; 1000)
#89.238.150.42 gb, Arion (United Kingdom, London; 1000)
217.151.98.167 gb, Asterion (United Kingdom, London; 1000)
89.249.74.212 gb, Asterope (United Kingdom, Manchester; 1000)
185.103.96.134 gb, Betelgeuse (United Kingdom, Maidenhead; 1000)
94.229.74.90 gb, Carinae (United Kingdom, Maidenhead; 1000)
89.249.74.217 gb, Chow (United Kingdom, Manchester; 1000)
185.103.96.133 gb, Denebola (United Kingdom, Maidenhead; 1000)
2.58.47.202 gb, Geminorum (United Kingdom, London; 1000)
185.103.96.131 gb, Kitel (United Kingdom, Maidenhead; 1000)
185.103.96.130 gb, Minkar (United Kingdom, Maidenhead; 1000)
84.39.117.56 gb, Naos (United Kingdom, Manchester; 1000)
84.39.116.179 gb, Nashira (United Kingdom, Manchester; 1000)
192.145.126.114 gb, Orbitar (United Kingdom, Manchester; 1000)
141.98.101.132 gb, Westerlund (United Kingdom, Manchester; 1000)
37.120.210.210 jp, Biham (Japan, Tokyo; 1000)
82.102.28.106 jp, Iskandar (Japan, Tokyo; 1000)
37.120.210.218 jp, Okab (Japan, Tokyo; 1000)
193.148.16.210 jp, Taphao (Japan, Tokyo; 1000)
46.183.220.202 lv, Felis (Latvia, Riga; 1000)
#159.148.186.13 lv, Meissa (Latvia, Riga; 100)
159.148.186.18 lv, Phact (Latvia, Riga; 100)
159.148.186.24 lv, Schedir (Latvia, Riga; 100)
159.148.186.31 lv, Shaula (Latvia, Riga; 100)
213.152.161.180 nl, Alchiba (Netherlands, Alblasserdam; 1000)
213.152.161.116 nl, Alcyone (Netherlands, Alblasserdam; 1000)
134.19.179.170 nl, Aljanah (Netherlands, Alblasserdam; 1000)
213.152.187.199 nl, Alphard (Netherlands, Alblasserdam; 1000)
213.152.187.194 nl, Alphecca (Netherlands, Alblasserdam; 1000)
134.19.179.242 nl, Alpheratz (Netherlands, Alblasserdam; 1000)
213.152.187.214 nl, Alphirk (Netherlands, Alblasserdam; 1000)
213.152.162.78 nl, Alrai (Netherlands, Alblasserdam; 1000)
213.152.161.4 nl, Alshat (Netherlands, Alblasserdam; 1000)
213.152.161.169 nl, Alterf (Netherlands, Alblasserdam; 1000)
213.152.187.204 nl, Alzirr (Netherlands, Alblasserdam; 1000)
213.152.162.164 nl, Ancha (Netherlands, Alblasserdam; 1000)
213.152.161.228 nl, Andromeda (Netherlands, Alblasserdam; 1000)
213.152.186.18 nl, Anser (Netherlands, Alblasserdam; 1000)
213.152.187.209 nl, Asellus (Netherlands, Alblasserdam; 1000)
134.19.179.194 nl, Aspidiske (Netherlands, Alblasserdam; 1000)
213.152.161.9 nl, Atik (Netherlands, Alblasserdam; 1000)
213.152.161.218 nl, Canis (Netherlands, Alblasserdam; 1000)
134.19.179.138 nl, Capella (Netherlands, Alblasserdam; 1000)
213.152.162.169 nl, Caph (Netherlands, Alblasserdam; 1000)
213.152.161.68 nl, Celaeno (Netherlands, Alblasserdam; 1000)
213.152.187.219 nl, Chara (Netherlands, Alblasserdam; 1000)
213.152.186.162 nl, Comae (Netherlands, Alblasserdam; 1000)
213.152.162.14 nl, Crater (Netherlands, Alblasserdam; 1000)
213.152.161.243 nl, Cygnus (Netherlands, Alblasserdam; 1000)
213.152.161.164 nl, Diphda (Netherlands, Alblasserdam; 1000)
213.152.161.210 nl, Edasich (Netherlands, Alblasserdam; 1000)
213.152.186.39 nl, Elnath (Netherlands, Alblasserdam; 1000)
134.19.179.146 nl, Eltanin (Netherlands, Alblasserdam; 1000)
213.152.162.73 nl, Garnet (Netherlands, Alblasserdam; 1000)
213.152.161.100 nl, Gianfar (Netherlands, Alblasserdam; 1000)
213.152.162.93 nl, Gienah (Netherlands, Alblasserdam; 1000)
213.152.161.39 nl, Hassaleh (Netherlands, Alblasserdam; 1000)
213.152.162.4 nl, Horologium (Netherlands, Alblasserdam; 1000)
213.152.161.34 nl, Hyadum (Netherlands, Alblasserdam; 1000)
213.152.162.9 nl, Hydrus (Netherlands, Alblasserdam; 1000)
213.152.186.23 nl, Jabbah (Netherlands, Alblasserdam; 1000)
213.152.161.84 nl, Kajam (Netherlands, Alblasserdam; 1000)
213.152.162.180 nl, Kocab (Netherlands, Alblasserdam; 1000)
134.19.179.178 nl, Larawag (Netherlands, Alblasserdam; 1000)
213.152.186.167 nl, Luhman (Netherlands, Alblasserdam; 1000)
213.152.162.103 nl, Maasym (Netherlands, Alblasserdam; 1000)
213.152.187.224 nl, Matar (Netherlands, Alblasserdam; 1000)
134.19.179.162 nl, Melnick (Netherlands, Alblasserdam; 1000)
213.152.161.29 nl, Merga (Netherlands, Alblasserdam; 1000)
213.152.162.68 nl, Mirach (Netherlands, Alblasserdam; 1000)
213.152.162.88 nl, Miram (Netherlands, Alblasserdam; 1000)
134.19.179.202 nl, Muhlifain (Netherlands, Alblasserdam; 1000)
213.152.162.153 nl, Muscida (Netherlands, Alblasserdam; 1000)
213.152.161.248 nl, Musica (Netherlands, Alblasserdam; 1000)
213.152.161.24 nl, Nash (Netherlands, Alblasserdam; 1000)
213.152.161.238 nl, Orion (Netherlands, Alblasserdam; 1000)
213.152.187.229 nl, Phaet (Netherlands, Alblasserdam; 1000)
134.19.179.130 nl, Piscium (Netherlands, Alblasserdam; 1000)
213.152.162.148 nl, Pleione (Netherlands, Alblasserdam; 1000)
213.152.161.233 nl, Pyxis (Netherlands, Alblasserdam; 1000)
213.152.162.83 nl, Rukbat (Netherlands, Alblasserdam; 1000)
213.152.161.19 nl, Salm (Netherlands, Alblasserdam; 1000)
134.19.179.154 nl, Scuti (Netherlands, Alblasserdam; 1000)
213.152.186.34 nl, Sheliak (Netherlands, Alblasserdam; 1000)
213.152.161.14 nl, Situla (Netherlands, Alblasserdam; 1000)
213.152.162.98 nl, Subra (Netherlands, Alblasserdam; 1000)
134.19.179.186 nl, Suhail (Netherlands, Alblasserdam; 1000)
213.152.161.137 nl, Talitha (Netherlands, Alblasserdam; 1000)
213.152.161.132 nl, Tarazed (Netherlands, Alblasserdam; 1000)
134.19.179.234 nl, Tiaki (Netherlands, Alblasserdam; 1000)
213.152.186.172 nl, Tianyi (Netherlands, Alblasserdam; 1000)
213.152.161.148 nl, Zibal (Netherlands, Alblasserdam; 1000)
82.102.27.194 no, Camelopardalis (Norway, Oslo; 1000)
82.102.27.170 no, Cepheus (Norway, Oslo; 1000)
185.206.225.50 no, Fomalhaut (Norway, Oslo; 1000)
82.102.27.162 no, Gemini (Norway, Oslo; 1000)
185.206.225.58 no, Ophiuchus (Norway, Oslo; 1000)
103.231.91.58 nz, Fawaris (New Zealand, Auckland; 1000)
91.207.102.162 ro, Alamak (Romania, Bucharest; 1000)
86.105.9.66 ro, Canes (Romania, Bucharest; 1000)
152.89.160.130 rs, Alnitak (Serbia, Belgrade; 1000)
128.127.104.79 se, Ain (Sweden, Stockholm; 10000)
62.102.148.149 se, Albali (Sweden, Uppsala; 1000)
62.102.148.142 se, Algieba (Sweden, Uppsala; 1000)
62.102.148.147 se, Algorab (Sweden, Uppsala; 1000)
62.102.148.145 se, Alrami (Sweden, Uppsala; 1000)
62.102.148.140 se, Altarf (Sweden, Uppsala; 1000)
62.102.148.151 se, Alula (Sweden, Uppsala; 1000)
62.102.148.150 se, Atria (Sweden, Uppsala; 1000)
62.102.148.141 se, Azmidiske (Sweden, Uppsala; 1000)
62.102.148.148 se, Benetnasch (Sweden, Uppsala; 1000)
79.142.76.243 se, Copernicus (Sweden, Stockholm; 1000)
62.102.148.144 se, Hatysa (Sweden, Uppsala; 1000)
128.127.105.183 se, Lupus (Sweden, Stockholm; 1000)
62.102.148.143 se, Menkab (Sweden, Uppsala; 1000)
62.102.148.146 se, Muphrid (Sweden, Uppsala; 1000)
31.3.152.99 se, Norma (Sweden, Stockholm; 1000)
103.254.153.68 sg, Antares (Singapore, Singapore; 1000)
185.200.116.210 sg, Auriga (Singapore, Singapore; 1000)
185.200.116.202 sg, Circinus (Singapore, Singapore; 1000)
185.200.116.218 sg, Delphinus (Singapore, Singapore; 1000)
185.200.117.130 sg, Hydra (Singapore, Singapore; 1000)
209.58.173.142 sg, Lacaille (Singapore, Singapore; 1000)
92.119.178.2 sg, Luyten (Singapore, Singapore; 1000)
209.58.183.86 sg, Struve (Singapore, Singapore; 1000)
185.200.116.130 sg, Triangulum (Singapore, Singapore; 1000)
91.231.84.39 ua, Alcor (Ukraine, Kiev; 1000)
173.44.55.154 us, Acamar (United States, Miami; 1000)
107.167.244.66 us, Alkes (United States, Los Angeles; 1000)
199.249.223.129 us, Aquila (United States, Fremont, California; 1000)
193.37.254.2 us, Bootes (United States, Phoenix, Arizona; 1000)
193.37.254.18 us, Chalawan (United States, Phoenix, Arizona; 1000)
199.249.230.41 us, Chamaeleon (United States, Dallas, Texas; 1000)
96.47.229.58 us, Cursa (United States, Miami; 1000)
185.228.19.146 us, Dimidium (United States, New York City; 1000)
199.249.230.36 us, Equuleus (United States, Dallas, Texas; 1000)
68.235.48.107 us, Fang (United States, Chicago, Illinois; 1000)
91.132.0.202 us, Gliese (United States, New York City; 1000)
37.120.132.82 us, Groombridge (United States, Los Angeles; 1000)
199.249.230.46 us, Helvetios (United States, Dallas, Texas; 1000)
64.42.179.58 us, Hercules (United States, Atlanta, Georgia; 1000)
193.37.254.26 us, Indus (United States, Phoenix, Arizona; 1000)
68.235.35.123 us, Kruger (United States, Chicago, Illinois; 1000)
199.249.230.21 us, Leo (United States, Dallas, Texas; 1000)
64.42.179.66 us, Libra (United States, Atlanta, Georgia; 1000)
194.36.111.58 us, Lich (United States, New York City; 1000)
199.249.230.6 us, Mensa (United States, Dallas, Texas; 1000)
107.167.244.50 us, Merope (United States, Los Angeles; 1000)
156.96.151.131 us, Metallah (United States, Pennsylvania; 1000)
64.42.179.42 us, Musca (United States, Atlanta, Georgia; 1000)
199.249.230.16 us, Pegasus (United States, Dallas, Texas; 1000)
193.37.254.34 us, Phoenix (United States, Phoenix, Arizona; 1000)
198.203.28.42 us, Pollux (United States, Jacksonville, Florida; 1000)
199.249.230.26 us, Ran (United States, Dallas, Texas; 1000)
107.167.244.82 us, Sabik (United States, Los Angeles; 1000)
64.42.179.34 us, Sculptor (United States, Atlanta, Georgia; 1000)
199.249.230.11 us, Scutum (United States, Dallas, Texas; 1000)
68.235.52.35 us, Sneden (United States, Chicago, Illinois; 1000)
37.120.132.90 us, Teegarden (United States, Los Angeles; 1000)
64.42.179.50 us, Ursa (United States, Atlanta, Georgia; 1000)
193.37.254.10 us, Virgo (United States, Phoenix, Arizona; 1000)
199.249.230.31 us, Volans (United States, Dallas, Texas; 1000)
199.249.230.1 us, Vulpecula (United States, Dallas, Texas; 1000)
173.44.55.178 us, Yildun (United States, Miami; 1000)

I believe it doesn't affect Eddie as it can pick servers on its own by pinging. But the "preferred" server and DNS responses are still dependent on the server logic, hence Ain sometimes ends up recommended as Earth or Europe server (currently not any longer) but seems to always be the preferred choice for Sweden. To quantify that, it's 350 out of 755 users connected to Swedish servers and unnecessarily getting insane jitter and latencies.

To proof it's not just me, https://lg.telia.net/ from AMS-IX showed 24ms to another Swedish server by AirVPN and approx. 340ms to Ain. Or AirVPN's own lookup: https://airvpn.org/routes/?q=128.127.104.79
PS: Is it an Intel CPU?

Edit: What I meant to say with this post (not only to start investigating Ain) that the "best server" logic should be not only working based on bandwidth load, but the server's relative latency times.
PS2: I do realize that it's currently listed as having issues (packet loss) but during my last week's connect afaik it wasn't. Logically, jitter/high latency begins before packet loss kicks in (networking and throughput theory) - https://airvpn.org/servers/Ain/ Edited ... by Stalinium
PS2

[blueport26 11]

Hello,

I can confirm what @Stalinium wrote. I live in center-east Europe (Poland) and constantly see high ping values to Ain (between 200-300ms). While for other servers in Sweden is's between 40-50ms.
Performing a trace route (doesn't matter if I'm connected to VPN) shows that the traffic is goes through obenetwork:


I found IP of different server from this datacenter/ISP. Below is a traceroute:
 

By comparing this results I can assume that:
1. It's not a problem with ISP used to host Ain server.
2. Ain response time spikes at the last hop.
I'm not a network engineer but I think this server is at it's peak throughput and that's why the ping is high.
If it truly has a 10Gbit uplink it's probably a hardware limitation (the CPU utilization is close to 100%).

...if my hypothesis is correct, what could be done (critic):
1. Ain should not be recommended in Eddie and DNS lookup, as there are others available with more free bandwidth.
2. For transparency reasons the status page could show server hardware utilization as a graph, or hypothetical maximum speed (limited by CPU/OpenVPN).

Best regards,
 

Edited ... by blueport26

[Staff 10014]

@Stalinium

Hello!

We might have underrated the non-linear growth of load over clients amount, which is very difficult to compute in advance because it depends not only on bandwidth required by a client, but (also) on an unknown variable, that is the amount of half.-connections established by single clients, which varies enormously over time and by single clients (different usages).

We are fine tuning and resolve the issue if necessary, thank you for the head up.

Kind regards
 

[Stalinium 47]

Just to provide an update: the Staff has indeed tweaked the server last week and it appears fine on monitoring Maybe more to come but it looks fixed now.
Its a great example of "include your users in the process" ❤️ no bouncing of support tickets back and forth due to outsourced support haha. Probably we've all been on the internet long enough to have experienced it first hand