Calculate expected throughput for specific device

[htpc 9]

This is remotely related to another earlier topic but as the question is pretty specific, I though I'd better start a new topic.

On a low powered  Enigma2 box with a Broadcom 2-core (BCM7362) CPU I'm seeing lower than expected OpenVPN speeds. It's perfectly possible that the hardware just doesn't support better speeds but before I invest in more powerful hardware I wanted to make sure that all possible errors were eliminated and all optimization options were utilized.

Here's what I did to test the openssl capabilities of the device:
 

openssl speed aes-128-cbc                                                          
Doing aes-128 cbc for 3s on 16 size blocks: 1545626 aes-128 cbc's in 2.84s                        
Doing aes-128 cbc for 3s on 64 size blocks: 442085 aes-128 cbc's in 2.94s                         
Doing aes-128 cbc for 3s on 256 size blocks: 114144 aes-128 cbc's in 2.92s                        
Doing aes-128 cbc for 3s on 1024 size blocks: 28866 aes-128 cbc's in 2.95s                        
Doing aes-128 cbc for 3s on 8192 size blocks: 3591 aes-128 cbc's in 2.89s                         
Doing aes-128 cbc for 3s on 16384 size blocks: 1784 aes-128 cbc's in 2.82s                        
OpenSSL 1.1.1l  24 Aug 2021                                                                       
built on: Sat Oct 15 10:37:44 2022 UTC                                                            
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr)                                  
compiler: mipsel-oe-linux-gcc  -mel -mabi=32 -mhard-float -march=mips32 --sysroot=recipe-sysroot -
Os -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=                      -fdebug-prefix
-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=  -mel -mab
i=32 -mhard-float -march=mips32 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_
ASM -DSHA256_ASM -DAES_ASM -DNDEBUG                                                               
The 'numbers' are in 1000s of bytes per second processed.                                         
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes        
aes-128 cbc       8707.75k     9623.62k    10007.15k    10019.93k    10179.06k    10364.91k 

This suggests a more than 80 Mbit/s capability for SSL

Then I conducted a theoretical OpenVPN throughput test with the following output:
 

time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc                                                                                        
Sat Dec 10 12:43:11 2022 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode                                                                                                                                                              
real    1m31,731s                                                                                 
user    1m30,007s                                                                                 
sys     0m0,712s  

Following calculation examples from other posts this suggests a theoretical throughput capability of (3200/91,731) approx. 35 Mbit/s.

The speeds I'm seeing though are +/- 8 Mbit/s when running online speed tests resulting in choppy playback of HD video streams that run smoothly on other devices on the same network or when disabling OpenVPN on the device.

My config looks like this:
 

client
dev tun
remote xxx.xxx.xxx.xxx 443
resolv-retry infinite
nobind
persist-key
persist-tun
auth-nocache
verb 3
explicit-exit-notify 5
push-peer-info
setenv UV_IPV6 yes
ca "ca.crt"
cert "user.crt"
key "user.key"
remote-cert-tls server
cipher AES-128-CBC
ncp-disable
comp-lzo no
proto udp
fast-io
tls-auth "ta.key" 1
log /etc/openvpn/openvpn.log

I also tried to further optimize with different values for those parameters without any improvement:
 

sndbuf 0
rcvbuf 0
txqueuelen 2000

Is there anything suspicious in the numbers I posted? Are the slow speeds I see to be expected despite the suggested higher numbers of the conducted tests? Is there anything else I can try to improve the throughput? Or do I have to live with the fact that the device is simply underpowered?

Thanks in advance!

 

[go558a83nk 362]

8 hours ago, htpc said:

This is remotely related to another earlier topic but as the question is pretty specific, I though I'd better start a new topic.

On a low powered  Enigma2 box with a Broadcom 2-core (BCM7362) CPU I'm seeing lower than expected OpenVPN speeds. It's perfectly possible that the hardware just doesn't support better speeds but before I invest in more powerful hardware I wanted to make sure that all possible errors were eliminated and all optimization options were utilized.

Here's what I did to test the openssl capabilities of the device:
 

openssl speed aes-128-cbc                                                          
Doing aes-128 cbc for 3s on 16 size blocks: 1545626 aes-128 cbc's in 2.84s                        
Doing aes-128 cbc for 3s on 64 size blocks: 442085 aes-128 cbc's in 2.94s                         
Doing aes-128 cbc for 3s on 256 size blocks: 114144 aes-128 cbc's in 2.92s                        
Doing aes-128 cbc for 3s on 1024 size blocks: 28866 aes-128 cbc's in 2.95s                        
Doing aes-128 cbc for 3s on 8192 size blocks: 3591 aes-128 cbc's in 2.89s                         
Doing aes-128 cbc for 3s on 16384 size blocks: 1784 aes-128 cbc's in 2.82s                        
OpenSSL 1.1.1l  24 Aug 2021                                                                       
built on: Sat Oct 15 10:37:44 2022 UTC                                                            
options:bn(64,32) rc4(char) des(long) aes(partial) blowfish(ptr)                                  
compiler: mipsel-oe-linux-gcc  -mel -mabi=32 -mhard-float -march=mips32 --sysroot=recipe-sysroot -
Os -pipe -g -feliminate-unused-debug-types -fmacro-prefix-map=                      -fdebug-prefix
-map=                      -fdebug-prefix-map=                      -fdebug-prefix-map=  -mel -mab
i=32 -mhard-float -march=mips32 -DOPENSSL_USE_NODELETE -DOPENSSL_PIC -DOPENSSL_BN_ASM_MONT -DSHA1_
ASM -DSHA256_ASM -DAES_ASM -DNDEBUG                                                               
The 'numbers' are in 1000s of bytes per second processed.                                         
type             16 bytes     64 bytes    256 bytes   1024 bytes   8192 bytes  16384 bytes        
aes-128 cbc       8707.75k     9623.62k    10007.15k    10019.93k    10179.06k    10364.91k 

This suggests a more than 80 Mbit/s capability for SSL

Then I conducted a theoretical OpenVPN throughput test with the following output:
 

time openvpn --test-crypto --secret /tmp/secret --verb 0 --tun-mtu 20000 --cipher aes-128-cbc                                                                                        
Sat Dec 10 12:43:11 2022 disabling NCP mode (--ncp-disable) because not in P2MP client or server mode                                                                                                                                                              
real    1m31,731s                                                                                 
user    1m30,007s                                                                                 
sys     0m0,712s  

Following calculation examples from other posts this suggests a theoretical throughput capability of (3200/91,731) approx. 35 Mbit/s.

The speeds I'm seeing though are +/- 8 Mbit/s when running online speed tests resulting in choppy playback of HD video streams that run smoothly on other devices on the same network or when disabling OpenVPN on the device.

My config looks like this:
 

client
dev tun
remote xxx.xxx.xxx.xxx 443
resolv-retry infinite
nobind
persist-key
persist-tun
auth-nocache
verb 3
explicit-exit-notify 5
push-peer-info
setenv UV_IPV6 yes
ca "ca.crt"
cert "user.crt"
key "user.key"
remote-cert-tls server
cipher AES-128-CBC
ncp-disable
comp-lzo no
proto udp
fast-io
tls-auth "ta.key" 1
log /etc/openvpn/openvpn.log

I also tried to further optimize with different values for those parameters without any improvement:
 

sndbuf 0
rcvbuf 0
txqueuelen 2000

Is there anything suspicious in the numbers I posted? Are the slow speeds I see to be expected despite the suggested higher numbers of the conducted tests? Is there anything else I can try to improve the throughput? Or do I have to live with the fact that the device is simply underpowered?

Thanks in advance!

 

setting the buffers to "0" just means the default for the OS, doesn't it?  I'm thinking it needs to be bigger, not the default.

Also, you might try messing with MTU/MSS stuff.

[htpc 9]

Thanks for your answer.

I tried higher values for the buffers as well already. 

tun-mtu und mssfix is a good recommendation. Just tried that. Evaluated the MTU size with the ping fragment method and set it accordingly in the client config. There is indeed a minimal speed increase but almost negligible. 

[go558a83nk 362]

1 minute ago, htpc said:

Thanks for your answer.

I tried higher values for the buffers as well already. 

tun-mtu und mssfix is a good recommendation. Just tried that. Evaluated the MTU size with the ping fragment method and set it accordingly in the client config. There is indeed a minimal speed increase but almost negligible. 

I've always had good luck using "mssfix 0" actually.  And Also setting tun-mtu to something crazy high so the virtual adapter isn't a bottleneck. 

[OpenSourcerer 1435]

20 hours ago, htpc said:

This is remotely related to another earlier topic but as the question is pretty specific, I though I'd better start a new topic.

Thank you for this. But you can still link to it here.
 

20 hours ago, htpc said:

This suggests a more than 80 Mbit/s capability for SSL

It depends. Do these tests use all available cores? Because OpenVPN uses only one, so I'd assume, in your case, half of that.

Rest might just be a mix of MTU, mssfix/fragment, server, protocol, port, IP version and cipher choice. My immediate recommendations: Since you're on MIPS, use CHACHA20-POLY1305 instead of anything AES, and connect with v6.

[htpc 9]

2 minutes ago, OpenSourcerer said:

It depends. Do these tests use all available cores? Because OpenVPN uses only one, so I'd assume, in your case, half of that.

Well, I'm not sure about that but very probably that's the case. Also because then the two test (openssl and OpenVPN) would report very similar numbers. Didn't even think about that.

3 minutes ago, OpenSourcerer said:

Rest might just be a mix of MTU, mssfix/fragment, server, protocol, port, IP version and cipher choice. My immediate recommendations: Since you're on MIPS, use CHACHA20-POLY1305 instead of anything AES, and connect with v6.

I wanted to try and switch to CHACHA20-POLY1305 before because it was suggested in other places as well and because of the missing AES-IN capability of the CPU. Unfortunately whenever I try, I get an error message in the logs that the cipher is not supported (obviously by the OpenVPN version on the device - which is 2.4.3 btw) and hence OpenVPN connection fails altogether. A shame because the simple openssl speedtest (which supports ChaCha) also suggest up to 50% speed improvement over AES-128-CBC.