ANSWERED Network drop while x-ferring files over VPN

[selbsmitleid 2]

Hi - Having an unusual problem here. This is a VM based Windows 7 instance running the experimental version of Eddie (I tried stable first, same result), with network lock enabled (ping enabled).

After about 3-5 minutes of a transfer over VPN - I lose connectivity within the VM. No pings, no VPN,... no nothing. Now, if the VPN is connected and just idling - it will sit there (seemingly) all day with no problem. My connection maxes out at 25mbps. I have tried throttling the app to use only 5. No improvement. The remedy for this, is to disable the LAN nic and then re-enable it, within Windows 7. After that, Eddie connects and does what it is supposed to do, until I start another transfer and then it will die again after 3-5 minutes.

As a test, I moved a few GB size files around the LAN here at 1gb (MTU 1500), without any VPN, (from/ to this instance) and there are no problems so, I can rule out a hardware issue from the ESXi host and an OS NIC issue. This problem specifically occurs while transferring files over VPN. I don't think there is anything too unusual in the log, but I have included it below. The last error line, I believe, is to be expected after a network drop.

I would be grateful if someone could look at this from another angle and see if they can see something. THANKS!

EDIT: So far,
- No events thrown in Event Viewer
- I have cleared DNS entries in the windows TCP settings and confirmed IPv6 is unticked
- I have thrown another CPU at it - just for fun since the proc was running 50-60%. This caused an even faster network drop at about 50 seconds. Second CPU is now gone..
- I have run this to reset the TCP stack, with no difference:

netsh int ip reset

 

 

! 2019.10.10 20:40:13 - Connecting to Aquila (United States of America, Fremont, California)

. 2019.10.10 20:40:13 - OpenVPN > OpenVPN 2.4.7 i686-w64-mingw32 [SSL (OpenSSL)] [LZO] [LZ4] [PKCS11] [AEAD] built on Apr 25 2019
. 2019.10.10 20:40:13 - OpenVPN > Windows version 6.1 (Windows 7) 64bit
. 2019.10.10 20:40:13 - OpenVPN > library versions: OpenSSL 1.1.0j  20 Nov 2018, LZO 2.10
. 2019.10.10 20:40:13 - Connection to OpenVPN Management Interface
. 2019.10.10 20:40:13 - OpenVPN > MANAGEMENT: TCP Socket listening on [AF_INET]127.0.0.1:3100
. 2019.10.10 20:40:13 - OpenVPN > Outgoing Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
. 2019.10.10 20:40:13 - OpenVPN > Outgoing Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
. 2019.10.10 20:40:13 - OpenVPN > Incoming Control Channel Encryption: Cipher 'AES-256-CTR' initialized with 256 bit key
. 2019.10.10 20:40:13 - OpenVPN > Incoming Control Channel Encryption: Using 256 bit message hash 'SHA256' for HMAC authentication
. 2019.10.10 20:40:13 - OpenVPN > TCP/UDP: Preserving recently used remote address: [AF_INET]199.249.223.132:443
. 2019.10.10 20:40:13 - OpenVPN > Socket Buffers: R=[8192->262144] S=[8192->262144]
. 2019.10.10 20:40:13 - OpenVPN > UDP link local (bound): [AF_INET]x.x.x.43:1194
. 2019.10.10 20:40:13 - OpenVPN > UDP link remote: [AF_INET]199.249.223.132:443
. 2019.10.10 20:40:13 - OpenVPN > MANAGEMENT: Client connected from [AF_INET]127.0.0.1:3100
. 2019.10.10 20:40:13 - OpenVPN > TLS: Initial packet from [AF_INET]199.249.223.132:443, sid=2579ca69 651fa6f0
. 2019.10.10 20:40:13 - OpenVPN > VERIFY OK: depth=1, C=IT, ST=IT, L=Perugia, O=airvpn.org, CN=airvpn.org CA, emailAddress=info@airvpn.org
. 2019.10.10 20:40:13 - OpenVPN > VERIFY KU OK
. 2019.10.10 20:40:13 - OpenVPN > Validating certificate extended key usage
. 2019.10.10 20:40:13 - OpenVPN > ++ Certificate has EKU (str) TLS Web Server Authentication, expects TLS Web Server Authentication
. 2019.10.10 20:40:13 - OpenVPN > VERIFY EKU OK
. 2019.10.10 20:40:13 - OpenVPN > VERIFY OK: depth=0, C=IT, ST=IT, L=Perugia, O=airvpn.org, CN=Aquila, emailAddress=info@airvpn.org
. 2019.10.10 20:40:14 - OpenVPN > Control Channel: TLSv1.2, cipher TLSv1.2 DHE-RSA-AES256-GCM-SHA384, 4096 bit RSA
. 2019.10.10 20:40:14 - OpenVPN > [Aquila] Peer Connection Initiated with [AF_INET]199.249.223.132:443
. 2019.10.10 20:40:15 - OpenVPN > SENT CONTROL [Aquila]: 'PUSH_REQUEST' (status=1)
. 2019.10.10 20:40:15 - OpenVPN > PUSH: Received control message: 'PUSH_REPLY,comp-lzo no,redirect-gateway ipv6 def1 bypass-dhcp,dhcp-option DNS 10.9.186.1,dhcp-option DNS6 fde6:7a:7d20:5ba::1,tun-ipv6,route-gateway 10.9.186.1,topology subnet,ping 10,ping-restart 60,ifconfig-ipv6 fde6:7a:7d20:5ba::1080/64 fde6:7a:7d20:5ba::1,ifconfig 10.9.186.130 255.255.255.0,peer-id 2,cipher AES-256-GCM'
. 2019.10.10 20:40:15 - OpenVPN > Pushed option removed by filter: 'redirect-gateway ipv6 def1 bypass-dhcp'
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: timers and/or timeouts modified
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: compression parms modified
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: --ifconfig/up options modified
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: route-related options modified
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: --ip-win32 and/or --dhcp-option options modified
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: peer-id set
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: adjusting link_mtu to 1625
. 2019.10.10 20:40:15 - OpenVPN > OPTIONS IMPORT: data channel crypto options modified
. 2019.10.10 20:40:15 - OpenVPN > Data Channel: using negotiated cipher 'AES-256-GCM'
. 2019.10.10 20:40:15 - OpenVPN > Outgoing Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
. 2019.10.10 20:40:15 - OpenVPN > Incoming Data Channel: Cipher 'AES-256-GCM' initialized with 256 bit key
. 2019.10.10 20:40:15 - OpenVPN > interactive service msg_channel=0
. 2019.10.10 20:40:15 - OpenVPN > ROUTE_GATEWAY x.x.x.253/255.255.255.0 I=11 HWADDR=xx:xx:xx:xx:xx
. 2019.10.10 20:40:15 - OpenVPN > GDG6: remote_host_ipv6=n/a
. 2019.10.10 20:40:15 - OpenVPN > NOTE: GetBestInterfaceEx returned error: Element not found.   (code=1168)
. 2019.10.10 20:40:15 - OpenVPN > ROUTE6: default_gateway=UNDEF
. 2019.10.10 20:40:15 - OpenVPN > open_tun
. 2019.10.10 20:40:15 - OpenVPN > TAP-WIN32 device [Local Area Connection] opened: \\.\Global\{B15109E2-0462-4671-8566-093BAC8ACDD8}.tap
. 2019.10.10 20:40:15 - OpenVPN > TAP-Windows Driver Version 9.23
. 2019.10.10 20:40:15 - OpenVPN > Set TAP-Windows TUN subnet mode network/local/netmask = 10.9.186.0/10.9.186.130/255.255.255.0 [SUCCEEDED]
. 2019.10.10 20:40:15 - OpenVPN > Notified TAP-Windows driver to set a DHCP IP/netmask of 10.9.186.130/255.255.255.0 on interface {B15109E2-0462-4671-8566-093BAC8ACDD8} [DHCP-serv: 10.9.186.254, lease-time: 31536000]
. 2019.10.10 20:40:15 - OpenVPN > Successful ARP Flush on interface [15] {B15109E2-0462-4671-8566-093BAC8ACDD8}
. 2019.10.10 20:40:16 - OpenVPN > NETSH: C:\Windows\system32\netsh.exe interface ipv6 set address interface=15 fde6:7a:7d20:5ba::1080 store=active
. 2019.10.10 20:40:17 - OpenVPN > NETSH: C:\Windows\system32\netsh.exe interface ipv6 set dns Local Area Connection static fde6:7a:7d20:5ba::1 validate=no
. 2019.10.10 20:40:17 - OpenVPN > add_route_ipv6(fde6:7a:7d20:5ba::/64 -> fde6:7a:7d20:5ba::1080 metric 0) dev Local Area Connection
. 2019.10.10 20:40:17 - OpenVPN > C:\Windows\system32\netsh.exe interface ipv6 add route fde6:7a:7d20:5ba::/64 interface=15 fe80::8 store=active
. 2019.10.10 20:40:17 - OpenVPN > env_block: add PATH=C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
. 2019.10.10 20:40:22 - OpenVPN > TEST ROUTES: 1/1 succeeded len=0 ret=1 a=0 u/d=up
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\route.exe ADD 199.249.223.132 MASK 255.255.255.255 10.10.9.253
. 2019.10.10 20:40:22 - OpenVPN > ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=10 and dwForwardType=4
. 2019.10.10 20:40:22 - OpenVPN > Route addition via IPAPI succeeded [adaptive]
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\route.exe ADD 0.0.0.0 MASK 128.0.0.0 10.9.186.1
. 2019.10.10 20:40:22 - OpenVPN > ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=10 and dwForwardType=4
. 2019.10.10 20:40:22 - OpenVPN > Route addition via IPAPI succeeded [adaptive]
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\route.exe ADD 128.0.0.0 MASK 128.0.0.0 10.9.186.1
. 2019.10.10 20:40:22 - OpenVPN > ROUTE: CreateIpForwardEntry succeeded with dwForwardMetric1=10 and dwForwardType=4
. 2019.10.10 20:40:22 - OpenVPN > Route addition via IPAPI succeeded [adaptive]
. 2019.10.10 20:40:22 - OpenVPN > add_route_ipv6(::/3 -> fde6:7a:7d20:5ba::1 metric -1) dev Local Area Connection
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\netsh.exe interface ipv6 add route ::/3 interface=15 fe80::8 store=active
. 2019.10.10 20:40:22 - OpenVPN > env_block: add PATH=C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
. 2019.10.10 20:40:22 - OpenVPN > add_route_ipv6(2000::/4 -> fde6:7a:7d20:5ba::1 metric -1) dev Local Area Connection
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\netsh.exe interface ipv6 add route 2000::/4 interface=15 fe80::8 store=active
. 2019.10.10 20:40:22 - OpenVPN > env_block: add PATH=C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
. 2019.10.10 20:40:22 - OpenVPN > add_route_ipv6(3000::/4 -> fde6:7a:7d20:5ba::1 metric -1) dev Local Area Connection
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\netsh.exe interface ipv6 add route 3000::/4 interface=15 fe80::8 store=active
. 2019.10.10 20:40:22 - OpenVPN > env_block: add PATH=C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
. 2019.10.10 20:40:22 - OpenVPN > add_route_ipv6(fc00::/7 -> fde6:7a:7d20:5ba::1 metric -1) dev Local Area Connection
. 2019.10.10 20:40:22 - OpenVPN > C:\Windows\system32\netsh.exe interface ipv6 add route fc00::/7 interface=15 fe80::8 store=active
. 2019.10.10 20:40:22 - OpenVPN > env_block: add PATH=C:\Windows\System32;C:\Windows;C:\Windows\System32\Wbem
. 2019.10.10 20:40:22 - Interface Local Area Connection metric changed from Automatic to 3, layer IPv4
. 2019.10.10 20:40:22 - Interface Local Area Connection metric changed from Automatic to 3, layer IPv6
. 2019.10.10 20:40:22 - DNS leak protection with packet filtering enabled.
. 2019.10.10 20:40:23 - DNS IPv4 of a network adapter forced (Local Area Connection, from automatic to 10.9.186.1)
. 2019.10.10 20:40:23 - DNS IPv6 of a network adapter forced (Local Area Connection, from automatic to fde6:7a:7d20:5ba::1)
. 2019.10.10 20:40:23 - Routes, added a new route, 199.249.223.130 for gateway 10.9.186.1
. 2019.10.10 20:40:23 - Routes, added a new route, 2620:7:6000:1:6747:2e2b:3ddb:cce2 for gateway fde6:7a:7d20:5ba::1
. 2019.10.10 20:40:23 - Flushing DNS
I 2019.10.10 20:40:29 - Checking route IPv4
I 2019.10.10 20:40:30 - Checking route IPv6
I 2019.10.10 20:40:35 - Checking DNS
! 2019.10.10 20:40:35 - Connected.
. 2019.10.10 20:40:35 - OpenVPN > Initialization Sequence Completed
. 2019.10.10 20:46:57 - OpenVPN > AEAD Decrypt error: bad packet ID (may be a replay): [ #342745 ] -- see the man page entry for --no-replay and --replay-window for more info or silence this warning with --mute-replay-warnings
 

Edited ... by selbsmitleid
Poor spelling

[selbsmitleid 2]

I think I solved this issue and it was an error on my end. I found that the VM was installed with the E1000 NIC profile and optimally, it should be using the VMXNET3 profile.

I'll continue to monitor but, I think this resolved the issue. My NFS storage adapter is working faster and better as well, after this change.

While I am embarrassed - I will leave this post up... in case it can help anyone else.