How To Set Up pfSense 2.1 for AirVPN

[pfSense_fan 181]

@pfsense_fan: could you tell me what a replay error looks like in my logs and I'll check mine.

I can't see anything obvious and Im running your revised original new settings 

 

 

 

You will need to change the "Verb 3" setting in the advanced line to "verb 4" and save. It will literally say something about replay if you have it. It means packets are arriving out of order. Not a huge deal, but something that can be fixed.

[dIecbasC 38]

I'm running verb 4 already and can't see any replay errors. Strange - let me know if you want me to try anything to help diagnose it etc. 

[pfSense_fan 181]

I'm running verb 4 already and can't see any replay errors. Strange - let me know if you want me to try anything to help diagnose it etc. 

 

 

 

It's not strange, my setup is far more complex than yours. I have 16 NIC's installed currently, and many many tweaks. It's why I can't upload pictures for most steps of the guide... mine looks nothing like others screens will. None the less, my uploads on speedtest have been abnormally low since the update. I usually get about 12Mb. The 54 download is normal during most hours of the day.. i'm on a 60 Mb plan currently

 

 

 

 

But yeah, a close inspection of my OpenVPN logs showed the send and receive buffers being overflowed. Doubled the buffer size using "sndbuf 131072;rcvbuf 131072;" and no Replays since.

 

EDIT: Also, I read you were thinking about adding more NIC's. If you are adding one quad port sure, otherwise just get a managed switch. The more I learn about this the more I realize that is what I should have done in the beginning. But hey, I learn by tinkering. If you do buy a quad port, get an i350. My PRO/1000 PT quad port eats 15 watts by itself. Old technology, old and larger silicon dies. Run hot as hell too, even with a large heat sink. The i210 quad on my board has more offloading and doesn't even need a heat sink. I am likely buying a Rangely board and selling my Quad port PRO/1000's. I might buy an i350 quad but will also be getting a switch and rack mounting it all in the basement along with NAS.

[knicker 11]

Hi pf sense fan,

 

Thanx for this excellent guide. A bit more extended than mine I guess...

Excellent two extra firewall rules!

 

Kind regards,

 

knicker

[pfSense_fan 181]

Hi pf sense fan,

 

Thanx for this excellent guide. A bit more extended than mine I guess...

Excellent two extra firewall rules!

 

Kind regards,

 

knicker

 

 

 

Absolutely! I, and everyone else, should thank you as well though. We all learned a lot from your guide! I was just unsettled by a few things my firewall logs showed as well as a few OpenVPN config file settings that were left unset.

 

If you have not done so yet, be sure to go over the two check boxes at the end of Step 5 as well as enter in all of the options into the advanced settings area on the OpenVPN CLient page.

 

Being an experienced pfSense user at this point, do you have any tips for the rest of us?

[pfSense_fan 181]

To those that were following it, I figured out the issue. I don't have the technical knowledge to explain it, but the upgrade to the 4096 bit keys somehow broke the Large Recieve Offloading feature i had been using up to that point with no issue. I disabled it and it "fixed" the upload issue.

 

 

Interestingly enough, I loose pretty much exactly 10% of my rated speed to the vpn tunnel now. The loss was barely noticible before.

[ffdise 6]

Thanks for the guide which has proved really helpful.

 

Is there any possibility of you posting the Backup configuration file for the 2 nic version?

[pfSense_fan 181]

Absolutly! Thanks for the feedback and welcome aboard! I remember how alien this all seemed to me just a few months ago. It's not so bad once you know what you're looking at. That's why I wrote this.... and I hope other users will share things they learn with the community as well!

 

About your question, are you referring to the actual config file backed up from pfSense or the guide I made?

 I'm not sure I follow.

 

It's not safe to post the entire backup because it also backs up certificates. It also messes things up if our interfaces have different names or use different drivers (em, igb etc) I can load config files for individual areas such as sysctls and bootloaders, and am looking into doing so after extensive testing.

 

All that being said, My configuration looks nothing like the guide... my setup is very complex. I wouldn't be able to back it up regardless.

Edited ... by pfSense_fan

[ffdise 6]

Hi Thanks, for your quick reply. I was interested in the actual config file backed up from pfSense; not for your very complex system, but for a simple 2 NIC system. I was just thinking that I could load this into a fresh pfSence inside VMware and then I could go through the screens side-by-side to make sure what I did was the same as your instructions.  However, I can see the problems with certificates and interface names.  

 

I basically have it all working, but on a couple of occasions the VPN connection appears to have been lost which then seems to have brought down the WAN interface as well. I then found I'd missed something in your guide (I used the one before the 2 nic extension so I had to improvise a bit) and so hopefully I may have fixed it. But it would be nice to be able to check because I find it all too easy to miss the odd check box etc.

 

What would be fantastic would be for you to build a simple basic 2 NIC system in VMware, delete your certificates and then put this config online. This would mean we could all have a working reference implementation which would be a great help to the the less IT gifted among us when things don't seem to be working correctly with the real router. Having said that, I'm sure you have more than enough things to do :-)  

 

Thanks anyway for the work you have done which has been hugely helpful.

[pfSense_fan 181]

Hi Thanks, for your quick reply. I was interested in the actual config file backed up from pfSense; not for your very complex system, but for a simple 2 NIC system. I was just thinking that I could load this into a fresh pfSence inside VMware and then I could go through the screens side-by-side to make sure what I did was the same as your instructions.  However, I can see the problems with certificates and interface names.  

 

My setup has never been like that though. I would not have a way to back it up. I made the guide as text so not only could it be edited easily, but you can also print it. It should be clear as on most pages I left nothing out, you can even use a pencil to mark the steps off as you go.

 

 

 

I basically have it all working, but on a couple of occasions the VPN connection appears to have been lost which then seems to have brought down the WAN interface as well. I then found I'd missed something in your guide (I used the one before the 2 nic extension so I had to improvise a bit) and so hopefully I may have fixed it. But it would be nice to be able to check because I find it all too easy to miss the odd check box etc.

 

What would be fantastic would be for you to build a simple basic 2 NIC system in VMware, delete your certificates and then put this config online. This would mean we could all have a working reference implementation which would be a great help to the the less IT gifted among us when things don't seem to be working correctly with the real router. Having said that, I'm sure you have more than enough things to do :-)  

 

Thanks anyway for the work you have done which has been hugely helpful.

 

 

Unfortunately for you and others I have little to no interest in running a virtual machine at this time, I really have no reason to do so, I would sooner buy another piece of hardware if I needed another platform.  That is just a personal preference as I care about performance and to me it is just one more thing to go wrong. That being said, I see no reason to do more than I have. There will be small additions, and updates for when pfSense 2.2 comes out, but I think it is important for anyone using this to take some time to understand it. After all using this is for security and privacy and we should never leave that in the hands of others! Trial and error is a good thing... we learn! It is also my hope that others in the community will chime in and share further knowledge. I am still learning and will continue to share as I do!

 

The same goes for you, I hope you will share what you learn when you get the VM working! Good luck!

[ffdise 6]

Thanks once again for taking the time to reply to my message. I guess I should leave the issue now, but I do think you may have slightly misunderstood the point I was trying to make (probably because I did not make it very well.) The point is not to run pFsence in a virtual machine. I don’t intend to do this either. It is just so people like me can poke about with a known good configuration that actually works and can be used as a test vehicle or starting point etc. That is how I got my first pfSence system to work. I just ran pFsence in a VM and had it create a new subnet inside my LAN (which was connected to the real world via a Linksys box that I have been using for years). I could test out all sorts of stuff this way without losing connection to the outside world, and so it was a good test system. When that was done, I just saved my system and then loaded into pfSence running on the real hardware. I’m moderately adept at VMWare (which, BTW, is free to download), and I can bring up a new machine and boot it from the pfSence iso file in about 5 minutes. If one were to add another 5 minutes to add in your instructions, a quick test, delete the certs, save the config, and then that would have it done (you then delete the VM). Actually I’d be more than happy to do all this myself and then offer to upload it for others. But..., I just don’t think it would be quite the same :-)

 

Please to not take my clumsy postings as in any kind of criticism. What you have done is quite wonderful and I am very grateful for huge about of time and care you have obviously taken. There is absolutely no reason for you to do more; it was just a suggestion that would be helpful to me, and maybe to others.

[Casper31 73]

pfsense_fan,

Thank you for the great tutorial.At the moment my pfsense running better than ever before.I looking forward to configure an openvpn server next to the client .I am not sure how to do this.

Will follow this tutorial,while it grows in time.

 

Greetings,Linze

[pfSense_fan 181]

Thanks once again for taking the time to reply to my message. I guess I should leave the issue now, but I do think you may have slightly misunderstood the point I was trying to make (probably because I did not make it very well.) The point is not to run pFsence in a virtual machine. I don’t intend to do this either. It is just so people like me can poke about with a known good configuration that actually works and can be used as a test vehicle or starting point etc. That is how I got my first pfSence system to work. I just ran pFsence in a VM and had it create a new subnet inside my LAN (which was connected to the real world via a Linksys box that I have been using for years). I could test out all sorts of stuff this way without losing connection to the outside world, and so it was a good test system. When that was done, I just saved my system and then loaded into pfSence running on the real hardware. I’m moderately adept at VMWare (which, BTW, is free to download), and I can bring up a new machine and boot it from the pfSence iso file in about 5 minutes. If one were to add another 5 minutes to add in your instructions, a quick test, delete the certs, save the config, and then that would have it done (you then delete the VM). Actually I’d be more than happy to do all this myself and then offer to upload it for others. But..., I just don’t think it would be quite the same :-)

 

Please to not take my clumsy postings as in any kind of criticism. What you have done is quite wonderful and I am very grateful for huge about of time and care you have obviously taken. There is absolutely no reason for you to do more; it was just a suggestion that would be helpful to me, and maybe to others.

While I did misunderstand what you were trying to do, not to worry I took no offense.

 

That being said, there is no way to back up any of the openVPN settings or the AirVPN_WAN settings as it all dissappers even if in the config(the certs and the interface don't exist yet, user Refresh and I tried this in private) Those things have to be done manually, which is why I believe spending just a bit more time understanding this is best! At this point I can do a start to finish installs in about 5 minutes, including the basic firewall rules. Soon you will too! Don't misconstrue my belief in the importance of understanding this! It is why I explained so much at the beginning, I believe it is important!

 

Again not to worry, I took nothing ill away from your post, sometimes the meaning behind text escapes all of us, I did not mean to convey that if I did in my response.

[pfSense_fan 181]

pfsense_fan,

Thank you for the great tutorial.At the moment my pfsense running better than ever before.I looking forward to configure an openvpn server next to the client .I am not sure how to do this.

Will follow this tutorial,while it grows in time.

 

Greetings,Linze

 

 

Glad to hear this! The more people share success or failures the more that can know this is successful and the better the guide can get as I ammend it.

 

It will be a month or two before I release the server guide, hang in there!

[dIecbasC 38]

I on the other hand have ran into a replay error in my logs. I have removed the tun-mtu and mssfix from my settings while I test.

 

Did you ever get to the bottom of your replay errors. I checked my log today after noticing my squid filter had stopped blocking adverts on my tablets and found that I had the replay errors you mentioned too. Download speeds are still 110mbit + so they aren't crippling my connection but theres something there which needs tweaking. I suspect that for some reason my WAN connection dropped which caused Squid filter  to fail too. I'll sort that after I clean up the replay errors. 

 

other than that, system continues to work very well thanks to your guide,

Ian

[pfSense_fan 181]

 

I on the other hand have ran into a replay error in my logs. I have removed the tun-mtu and mssfix from my settings while I test.

 

Did you ever get to the bottom of your replay errors. I checked my log today after noticing my squid filter had stopped blocking adverts on my tablets and found that I had the replay errors you mentioned too. Download speeds are still 110mbit + so they aren't crippling my connection but theres something there which needs tweaking. I suspect that for some reason my WAN connection dropped which caused Squid filter  to fail too. I'll sort that after I clean up the replay errors. 

 

other than that, system continues to work very well thanks to your guide,

Ian

 

 

 

Before I answer from my end, what replay error did you get? Some are harmless while others are a potential sign of attack, want to be sure we are comparing apples to apples here.

 

The short answer is yes I fixed it. Unfortunately I don't REALLY know what actually caused it. I have some ideas based on my observations though.

 

Removing tun-mtu 1500;mssfix 1400; fixed it for me. Odd, considering I needed those settings before for stable operation. I've been researching this bit by bit each day since then, trying to understand what has changed, but it's been a process of elimination cosidering I have many tweaks. That being said, I believe a combination of the MSS (Maximum Segment Size) of the operating system causing fragmenting and reassembling (The PF, the packet filter, reasembles MSS to 1460 if what I have researched is correct, which is too big for the VPN tunnel), the MBUF settings being too low and therefor filling and the further fragmenting of MSS under OpenVPN (the two MSS's are different due to the overhead in the VPN protocol) causing the network buffers to start dropping some packets, hence the replays.

 

I have my network wide MSS set to 1400, adjusting the default so it will work over the VPN without further processing. This is more efficient than using mssfix, at least from what I have read. I have also tweaked my interface drivers and other system tunables that are related.

 

If you are interested, I can start a private chat and we can discuss tweaks for you to test. I would hate to post such info at this time as I feel it is a layered issue and I don't want to prematurely post a "fix", if you even want to call it that. A replay on a high bandwidth, high latency connection (as are many servers at distance) is considered normal. That's not to say we can't tweak a bit and get rid of them though!

[dIecbasC 38]

hi,

These are the log entries.

Apr 19 02:20:53 pfsense openvpn[39063]: PID_ERR replay-window backtrack occurred [1] [sSL-1] [0_00000000000000000000000000000000000000000000000000000000000000] 0:25509 0:25508 t=1397870453[0] r=[-3,64,15,1,1] sl=[27,64,64,528]

 

I recalled your previous post where you mentioned disabling the mtu and mss fix entries so tried that and it cleared up my log from further entries (24hours+ now). I benchmarked my connection before and after and haven't noticed any difference in terms of raw throughput but I no errors has to be better so will leave it as is. 

 

Id be happy to take an offline discussion re optimisations and help fine tune etc. I would agree, this thread should remain focused around getting pfsense working without further complication. I'll PM you my details.

[pfSense_fan 181]

hi,

These are the log entries.

Apr 19 02:20:53 pfsense openvpn[39063]: PID_ERR replay-window backtrack occurred [1] [sSL-1] [0_00000000000000000000000000000000000000000000000000000000000000] 0:25509 0:25508 t=1397870453[0] r=[-3,64,15,1,1] sl=[27,64,64,528]

 

 

 

Good, we are talking about the same replay error.

 

For anyone following, the backtrack warning shown here is likely caused by the latency in the connection. There are potential tweaks that some of us will be testing to avoid this. If we determine a solution it will be added to the advanced options section.

[pfSense_fan 181]

---

---

---

 

Please Note!!!! Guide has been amended!!!

 

 

 

The Default string of options entered into the advanced area of the OpenVPN Client settings area has been amended!!!

 

There are no critical changes however it is highly recommended you update them. They harden the security of the connection by not allowing, under any circumstance, the use of lower encryption and/or security levels than intended by AirVPN. Please note you are already using these options as they are "pushed" by AirVPN when you connect. What these settings do is, in the unlikely event of a man in the middle attack, prevent you from having any other weak/er settings being pushed to you.

 

You may also notice I removed two of the settings included in the AirVPN OpenVPN config file. "persist-tun" and "persist-key" have been removed due to the fact pfSense automatically enters these "in the background". You can verify this yourself by going to Diagnostics > Edit File. Once there, enter the string "/var/etc/openvpn/client1.conf" (without the quotes of course) and click "load"(NOTE: be careful not to edit anything or click save while here. Exit by navigating back to the dashboard or closing the tab). You will then see all of the settings your OpenVPN client are using. If you did not remove "persist-tun" and "persist-key", they will be entered twice. If you did remove them, they will still be there, but only once.

 

The Understanding OpenVPN settings in pfSense and Entering OpenVPN Client Settings pages have been updated. Please review and update your settings.

 

 

The new OpenVPN client advanced settings string is as follows:

 

remote-cert-tls server;tls-cipher TLS-DHE-RSA-WITH-AES-256-CBC-SHA;keysize 256;auth SHA1;key-method 2;key-direction 1;comp-lzo no;verb 3;explicit-exit-notify 5;

[Rhymer52 2]

Many thanks for the guide and extensive discussion of using pfSense. I'm thinking of putting together a small system to do this.

 

Newbie question regarding wireless access. If I set up a small system as a router running pfSense (multiple NICs, of course) would I use my existing wireless router downsrtream from the pfSense box to provide wireless access to the network?

 

If not, what is the best way to provide network access for wireless devices?

 

Thanks!

[pfSense_fan 181]

Many thanks for the guide and extensive discussion of using pfSense. I'm thinking of putting together a small system to do this.

 

Newbie question regarding wireless access. If I set up a small system as a router running pfSense (multiple NICs, of course) would I use my existing wireless router downsrtream from the pfSense box to provide wireless access to the network?

 

If not, what is the best way to provide network access for wireless devices?

 

Thanks!

 

 

Correct, running a router in access point mode is the best solution, provided the router has access point mode. I use an Asus router this way.

[Casper31 73]

My pfsense is configured with a lan and an airvpn_lan as discribed by pfsense _fan.(B.t.w. works great.)

I discovered something strange:in my situation skype is not able to connect via airvpn_lan, switsing to lan solves this.Other internet trafic is no problem.

Can it be that skype will only connect if there are microsoft dns server avaiable?

Because of the pfsense rules only airvpn are there.

 

Gr,Linze

[pfSense_fan 181]

My pfsense is configured with a lan and an airvpn_lan as discribed by pfsense _fan.(B.t.w. works great.)

I discovered something strange:in my situation skype is not able to connect via airvpn_lan, switsing to lan solves this.Other internet trafic is no problem.

Can it be that skype will only connect if there are microsoft dns server avaiable?

Because of the pfsense rules only airvpn are there.

 

Gr,Linze

 

 

To try to diagnose any issue, please change the "verb 3" option in the advanced OpenVPN client settings to "verb 5" and save. Then try to connect to skype a few times. After failing a few times,

 

Go to:

 

http://192.168.1.1/diag_logs_openvpn.php

 

Report to me any errors in those logs.Please Note: delete or replace any instances of any ip address with x.x.x.x before sending. Always be careful of that in logs. Also please use the "code" option (that is underneath the emoticon option in the editor) to encapsulate your entry, as I have for the link I posted above. It is best to paste the log to a text editor first to search for and remove any IP addresses and unwanted formatting.

[Casper31 73]

My pfsense is configured with a lan and an airvpn_lan as discribed by pfsense _fan.(B.t.w. works great.)

I discovered something strange:in my situation skype is not able to connect via airvpn_lan, switsing to lan solves this.Other internet trafic is no problem.

Can it be that skype will only connect if there are microsoft dns server avaiable?

Because of the pfsense rules only airvpn are there.

 

Gr,Linze

 

 

To try to diagnose any issue, please change the "verb 3" option in the advanced OpenVPN client settings to "verb 5" and save. Then try to connect to skype a few times. After failing a few times,

 

Go to:

 

http://192.168.1.1/diag_logs_openvpn.php

 

Report to me any errors in those logs.Please Note: delete or replace any instances of any ip address with x.x.x.x before sending. Always be careful of that in logs. Also please use the "code" option (that is underneath the emoticon option in the editor) to encapsulate your entry, as I have for the link I posted above. It is best to paste the log to a text editor first to search for and remove any IP addresses and unwanted formatting.

After a hours of testing ,i discovered that the skype problem went away after connecting Pfsense directly to the internet.So  experimenting-after a an extra router- gave me more problems... .

The most important question I have at the moment.What is meaning of the next 6 lines in "192.168.123.1/diag_logs_openvpn.php "

 

  openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: write UDPv4: No buffer space available (code=55)   openvpn[54768]: Control Channel: TLSv1, cipher TLSv1/SSLv3 DHE-RSA-AES256-SHA, 4096 bit RSA   openvpn[54768]: Data Channel Decrypt: Using 160 bit message hash 'SHA1' for HMAC authentication

 

At the moment everything  is working as I wanted it to be .Again thank you forum and Pfsense-fan.

 

Linze

[pfSense_fan 181]

From some research done by myself and another user, this error:

 

 

write UDPv4: No buffer space available (code=55)

 

is caused by maxing out the speed capability of a tunnel.

 

 

That being said, we found this to be caused by the specific server we were connected to. In my case, my ISP was throttling on the server I was connected to.

 

Try a different server or protocol if it persists.

 

 

The Control and Data chanel messages are normal operation.