Traffic split on Debian 7

[snarl817 0]

I hjave a Debian 7 system at home that I access remotely using ssh. The system runs a media server for the home network, as well as a torrent daemon (transmission). I ONLY want AirVPN to handle the torrent traffic. Unfortunately, I was under the impression that starting the VPN client would spin up a seperate network interface specifically for the VPN and I could use iptables to route torrent traffic through the VPN. The openvpn client instead schwacked my routing table...and since I'm remote, I have no way to shut it down.

 

So, the question I have is how do I configure OpenVPN/AirVPN to use a TAP interface with a local IP address on a different subnet from my current network? Reconfiguring Transmission to bind to a specific address seems pretty easy.

 

I don't need DNS routed through the VPN...I just need something on a 172.16.X.0/24 subnet as the local VPN endpoint.

[corrado 100]

Easiest solution is running AirVPN + transmission in a virtual machine. Without vm it gets a little more complicated - a good starting point is using cgroups. Here's a thread on serverfault.com that explains such a solution in detail. Writing my own OpenVPN client I did the exact opposite - run selected applications outside the tunnel based on the same approach. For a machine that is primarily accessed remotely by ssh you definitely need some iptables magic to get it all working. It's vital to edit your OpenVPN config files to use "route-noexec" to prevent OpenVPN from setting its standard routes. In a scenario such as your that has to be handled manually.

[snarl817 0]

I was hoping to not have to do that...but it IS doable. I just need to add a 2nd virtual ethernet adapter to the VM to route NFS traffic back to the host. Otherwise, downloaded content is stuck inside the VM.

 

I used to use NeoRouter to access my system remotely, and it created an nrtap interface on a 10.5.1/24 subnet. I'll play with OpenVPN to see if it's possible to configure it in this manner...configuring Transmission to bind to a non-standard interface is trivial. I have noticed that most products immediately assume that the sysadmin has no understanding of multi-homed hosts and just default to the primary network interface. This isn't helped by the fact that very few sysadmins I've interacted with really DO understand multi-homing.

[NaDre 159]

I was hoping to not have to do that...

 

See the second half of this post:

 

https://airvpn.org/topic/14158-question-run-airvpn-as-non-primary-network-adapter/?p=27398

 

...

 

Or are you using Linux? The following assumes so.

 

Linux uses a different "host model". See:

 

http://en.wikipedia.org/wiki/Host_model

 

If you do not want to change the OpenVPN config, then you still need to restore the default gateway in the default routing table in exactly the same way. And this will allow you to switch back and forth between real interface and VPN interface as default in the same way (removing and inserting the routing table entries with 192.0.0.0 mask).

 

But then on Linux you will need to do "source address routing" to have a program use the VPN interface. Binding the VPN interface address is not enough. Something like what is done here:

 

https://openvz.org/Source_based_routing

 

You won't need a "throw" rule.

 

With the source address routing in place, there is no danger of Linux fall-back to the default gateway for traffic bound to the VPN interface. So firewall config is not needed.

 

UPDATE:

 

I thought I should show my configuration (more or less) for doing this.

 

My config files all contain this line at the top:

 

config common/myroute.ovpni

In the same folder where I have the config files I have a subfolder named "common":

$ ls -l common
total 2
-rwxrwx---+ 1 user None 284 Jul 16 20:51 myroute.ovpni
-rwxrwx---+ 1 user None 176 Jun 30 16:02 up.sh

The file myroute.ovpni contains this:

script-security 2
up ./common/up.sh
route-nopull
redirect-gateway def1
route 0.0.0.0 192.0.0.0 net_gateway
route 64.0.0.0 192.0.0.0 net_gateway
route 128.0.0.0 192.0.0.0 net_gateway
route 192.0.0.0 192.0.0.0 net_gateway
sndbuf 524288
rcvbuf 524288

The file up.sh contains this:

#!/bin/bash
/sbin/ip rule del from $ifconfig_local table 10001
/sbin/ip rule add from $ifconfig_local table 10001
/sbin/ip route add default via $route_vpn_gateway table 10001

 

I am still using this method on a Debian server. I also use source address routing with Transmission on my Windows system at home (very different than Linux).

 

UPDATE:

 

Actually, my set up has evolved since then. It does IPv6 for example.

 

My script to start OpenVPN and my current "--up" script are pasted in below. No changes to any OpenVPN config files needed. This is just meant as an example. Pick out the bits you are interested in. I don't want to go to great lengths explaining unless you are interested. So ask if you have questions.

 

#!/bin/bash
pushd ~/openvpn_client_run/$1 > /dev/null
f=`echo _*.ovpn`
if [ ! -f "$f" ]; then
echo no OVPN file to start in ~/openvpn_client_run/$1 | tee ~/tmp/openvpn_client_run_$1.log
exit
fi
. openvpn_which
sudo $OPENVPN_BIN \
--setenv CLIENT_CONF $1 \
--config $f \
--script-security 2 \
--up $HOME/bin/openvpn_client_up \
--route-noexec \
--writepid ~/tmp/openvpn_client_run_$1.pid \
| tee ~/tmp/openvpn_client_run_$1.log
popd > /dev/null

 

#!/bin/bash

echo entered $0
echo arguments: $*
echo working directory: `pwd`
echo ============= environment ...
env
echo .. environment =============

# derived from /etc/openvpn/update-resolv-conf
split_into_parts()
{
part1="$1"
part2="$2"
part3="$3"
}
VPN_DNS=
VPN_v6_DNS=
for optionvarname in ${!foreign_option_*} ; do
option="${!optionvarname}"
echo "$option"
split_into_parts $option
if [ "$part1" = "dhcp-option" ] ; then
if [ "$part2" = "DNS" ] ; then
VPN_DNS="${VPN_DNS:+$VPN_DNS }$part3"
elif [ "$part2" = "DNS6" ] ; then
VPN_v6_DNS="${VPN_v6_DNS:+$VPN_v6_DNS }$part3"
fi
fi
done

VPNIP=/home/user/tmp/openvpn_client_run_VPNIP_${CLIENT_CONF}
echo writing $VPNIP ...
cat <<EOF >$VPNIP
VPN_IF=$1
VPN_GATEWAY=$route_vpn_gateway
VPN_LOCAL=$ifconfig_local
VPN_v6_RMT=$ifconfig_ipv6_remote
VPN_v6_LOCAL=$ifconfig_ipv6_local
VPN_v6_BITS=$ifconfig_ipv6_netbits
VPN_DNS="$VPN_DNS"
VPN_v6_DNS="$VPN_v6_DNS"
EOF
cat $VPNIP

HASH=$((16#`echo $CLIENT_CONF | md5sum | while read a b; do echo $a; done`))
TABLE=${HASH:1:5}
echo adding ip rules and routes as table $TABLE ...
/sbin/ip -4 rule del table $TABLE
/sbin/ip -4 rule add from $ifconfig_local table $TABLE
/sbin/ip -4 route add default via $route_vpn_gateway dev $1 table $TABLE
/sbin/ip -4 rule list
/sbin/ip -4 route list table $TABLE
if [ "$ifconfig_ipv6_local" != "" ]; then
/sbin/ip -6 rule del table $TABLE
/sbin/ip -6 rule add from $ifconfig_ipv6_local table $TABLE
/sbin/ip -6 route add default via fe80::8 dev $1 table $TABLE
/sbin/ip -6 rule list
/sbin/ip -6 route list table $TABLE
fi

[NaDre 159]

...

The openvpn client instead schwacked my routing table...and since I'm remote, I have no way to shut it down.

 ...

 

Just realized that your real issue is remote access. The method in the previous post will solve that, but there may be a simpler solution. See these posts (again just meant as examples):

 

https://airvpn.org/topic/12274-ubuntu-vm-cant-connect-through-openvpn/?p=44812

 

 

I'm having the same scenario with my Digital Ocean droplet.

 

Upon successfully connecting to AirVPN (via TCP:443), my terminal session would hang and I can no longer SSH back into the box (port 22 and 443) using the original IP or the AirVPN IP.

 

I would have to power cycle the droplet to disconnect VPN and SSH again via the original IP.

 

What is the best way to have the target droplet/server be running VPN, but still SSH into it?

If you can live with the VPN not being the default route, you can do it like this:

 

https://airvpn.org/topic/14634-problems-using-air-vpn-as-non-default-route/?p=29391

 

https://airvpn.org/topic/14158-question-run-airvpn-as-non-primary-network-adapter/?p=27398

 

On a VPS (rather than a VirtualBox VM on your PC) it may make more sense to replace the contents of myroute.ovpni described there with this:

script-security 2
up ./common/up.sh
route-nopull
redirect-private

You will need to bind whatever programs you want to use the VPN to the VPN interface.

 

===

 

UPDATE:

 

For completeness, the comments below may help demonstrate what the issue is.

 

As quick and dirty way to sustain the SSH connection, add a routing table entry to direct traffic to your SSH client over the original gateway. Something like this:

sudo route add -host 111.222.333.444 gw 555.666.777.1

There, "111.222.333.444" would be the address you connected from (as shown when you do "echo $SSH_CLIENT"), and "555.666.777.1" is the original default gateway (the entry with a "Genmask" of "0.0.0.0" when you do "/sbin/route -n").

 

SSH connections from anywhere else will still fail.

 

===

 

UPDATE 2:

 

I did not actually explain the problem above. The problem is that the default gateway gets changed by OpenVPN, and that breaks your current SSH connection unless you set up appropriate routes before you start OpenVPN.

 

Here is a more general purpose solution than what was in "UPDATE" above.

 

It is assumed here that the default gateway interface before OpenVPN is started is "eth0". This is the usual convention

for Linux systems.

 

It should ensure that when a connection to eth0 is made, even if eth0 is not the default gateway interface anymore, response packets for the connection back on eth0 again.

# set "connection" mark of connection from eth0 when first packet of connection arrives
sudo iptables -t mangle -A PREROUTING -i eth0 -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

# set "firewall" mark for response packets in connection with our connection mark
sudo iptables -t mangle -A OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

# our routing table with eth0 as gateway interface
sudo ip route add default dev eth0 table 3412

# route packets with our firewall mark using our routing table
sudo ip rule add fwmark 4321 table 3412

UPDATE to UPDATE 2:

 

The above works fine for me on Debian Jessie. But on an older Wheezy system I have just found that I need to add "via" to the routing table entry:

# our routing table with eth0 as gateway interface
sudo ip route add default dev eth0 via 12.345.67.89 table 3412

There "12.345.67.89" must be the original non-VPN gateway.

 

 

https://airvpn.org/topic/12274-ubuntu-vm-cant-connect-through-openvpn/?do=findComment&comment=67336

 

I hesitate to do this because I do not want to promise to help troubleshoot or maintain these scripts. Or even explain them (I have probably forgotten details myself). But here are two scripts I have in my "~/bin" folder on a VPS.

 

They determine the name of the gateway interface and its IP address for you. And there is optional code at the end (avoided by "exit") to show IPTABLES entries for troubleshooting.

 

You need to make these files executable:

chmod uog+x ~/bin/native_if_return_on
chmod uog+x ~/bin/native_if_return_off

===> native_if_return_on:

#!/bin/bash

ROUTE=`ip route show table main | grep default -`
#echo ROUTE=$ROUTE
TOK=($ROUTE)
#GW=${TOK[2]}
#echo GW=$GW
IF=${TOK[4]}
#echo IF=$IF

sudo iptables -t mangle -D PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234
sudo iptables -t mangle -A PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

sudo iptables -t mangle -D OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321
sudo iptables -t mangle -A OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

sudo ip route del all table 3412
#sudo ip route add default via $GW dev $IF table 3412
sudo ip route add $ROUTE table 3412

sudo ip rule del fwmark 4321
sudo ip rule add fwmark 4321 table 3412

# no IPv6
exit

ROUTE=`ip -6 route show table main | grep default -`
#echo ROUTE=$ROUTE
TOK=($ROUTE)
#GW=${TOK[2]}
#echo GW=$GW
IF=${TOK[4]}
#echo IF=$IF

sudo ip6tables -t mangle -D PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234
sudo ip6tables -t mangle -A PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

sudo ip6tables -t mangle -D OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321
sudo ip6tables -t mangle -A OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

sudo ip -6 route del all table 3412
#sudo ip -6 route add default via $GW dev $IF table 3412
sudo ip -6 route add $ROUTE table 3412

sudo ip -6 rule del fwmark 4321
sudo ip -6 rule add fwmark 4321 table 3412

exit

sudo iptables -t mangle -L -v
ip rule show
ip route list table 3412

sudo ip6tables -t mangle -L -v
ip -6 rule show
ip -6 route list table 3412

===> native_if_return_off:

#!/bin/bash

ROUTE=`ip route show table main | grep default -`
#echo ROUTE=$ROUTE
TOK=($ROUTE)
IF=${TOK[4]}
#echo IF=$IF

sudo iptables -t mangle -D PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

sudo iptables -t mangle -D OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

sudo ip route del all table 3412

sudo ip rule del fwmark 4321

# no IPv6
exit

ROUTE=`ip -6 route show table main | grep default -`
#echo ROUTE=$ROUTE
TOK=($ROUTE)
IF=${TOK[4]}
#echo IF=$IF

sudo ip6tables -t mangle -D PREROUTING -i $IF -m conntrack --ctstate NEW -j CONNMARK --set-mark 1234

sudo ip6tables -t mangle -D OUTPUT -m connmark --mark 1234 -j MARK --set-mark 4321

sudo ip -6 route del all table 3412

sudo ip -6 rule del fwmark 4321

exit

sudo iptables -t mangle -L -v
ip rule show
ip route list table 3412

sudo ip6tables -t mangle -L -v
ip -6 rule show
ip -6 route list table 3412

 

[anonymous0x15 0]

I have also needed some split-tunneling lately and came across a number of long sequences of routing commands and iptables commands. I started getting frustrated having to type in commands, modify the openvpn config files, and ensure that the connection was still up. I'm on Ubuntu 18.04 and I recently wrote a little ruby script that will set up a separate vpn-user for you and keep running as a daemon to ensure your vpn is still up. It still needs lots of work, but if anyone wants to try it, it's called vpnh.

Edited ... by anonymous0x15
grammatical typo