portforwarding pfsense wireguard port forwarding return traffic returning via WAN

[lewisw-j 0]

TLDR Issue: When port forwarding is enabled, traffic forwarded from the VPN is being return via the WAN.
So it's going AirVPN -> pfsense via VPN -> server -> pfsense via WAN -> Clear Internet.

Software: pfsense 2.7.2-RELEASE,
Package: WireGuard 0.2.1.
Server: ubuntu 22.04.03 LTS

I'm reaching out to the community because I'm out of ideas on how to fix what's going wrong. I'm not a network or firewall guru, I'm tech savy but clearly not enough to resolve my issue 😧.
I followed the pfSense baseline setup guide available at nguvu.org to configure my pfsense. While the guide primarily focuses on using OpenVPN for the VPN setup, I adapted the instructions to use WireGuard instead but clearly I've missed something or fundermentally misunderstanding something.

When I configure port forwarding using AirVPN with WireGuard to pfSense to my VPN network to a sever, I can see on a TCPDump the the initial inbound packets from AirVPN that's being port forwarded reaches the server, but each reply seems to vanish when returned to the router.
Using diag_packet_capture on pfsense, I can see the inbound traffic from AirVPN, but when the server replies, it's going out on the WAN interface. Clearly there's some sort of gateway issue.

I even tried to pay someone on Fiverr to fix it but they couldn't see any reason for it, they're claming it's a software defect, but seeing other people have somehow managed to do it, must be possible.

Have I missed something silly?

The port:


The Server:


These are my rules so far:

WAN:


WireGuard:


Floating:



VL20_VPN:


Note: Selective_Routing (or VPN whitelist) isn't set:


Gateways:


Port Forward:


Outbound NAT

Edited ... by lewisw-j
removing duplicated images.

[go558a83nk 364]

my quick look tells me you're perhaps not creating a firewall rule on the correct interface.  You've got a port forward rule for airvpn_wan_WG_0 but you're not showing a corresponding firewall rule for that interface for port forwarding.  when you create the port foreward rule use the filter rule association option at the bottom of the port forward rule setup to create new associated filter rule.  this will automatically put the necessary firewall rule on the correct interface.

also you're using an alias for NAT IP for the port forward rule.  this should be the IP of the device running the server.  I see no reason for an alias as it should be just one local IP, e.g. 192.168.2.22.

again, no reason for an alias for destination ports and NAT ports as AirVPN port forward rules can only forward 1 port each.

[SurprisedItWorks 49]

FWIW, I had a similar problem when I first set up Air port forwarding in dd-wrt to allow incoming connections to a wireguard server. Ultimately I solved it with an "ip rule..." (in linuxspeak) command in the wireguard interface's up script, and the corresponding removal of the rule in the downscript. The rule details routed all packets with the source port in question (dest port when Air routed incoming so source for replies) via the routing table associated with the VPN tunnel used for the forwarding. That tunnel had a separate routing table because Policy Based Routing (PBR) was in use. 

If that's all gobbledygook, you have my sympathies. Let's just say that I had to study up on routing tables, the rule table, PBR, and the ip command to get there, and I burned plenty of hours.  And after all that, it may not be at all the best way to solve the problem!  But the kludge that it is seems to be working on several routers and has been for over a year. Hope you get lucky and find something easier!

[benfitita 39]

There was somewhere a post about a similar problem, and maybe it was on this forum, pfsense or opnsense. I can't find it though. As far as I remember, the problem was with the autogenerated firewall rule. It didn't have proper gateway (or something similar) in advanced options. You can't modify auto-generated rules, so you need turn off auto-generated rule in port forwarding rule, setup it up manually and ensure gateway (or a similar option) is properly set.

PS. sorry for the vague post, but I hope this at least gives you some pointers where to look.

[lewisw-j 0]

On 3/5/2024 at 5:29 PM, go558a83nk said:

also you're using an alias for NAT IP for the port forward rule.  this should be the IP of the device running the server.  I see no reason for an alias as it should be just one local IP, e.g. 192.168.2.22.

Unless there is something technically different from using a IP over an Alias, then I just prefer using Alias, one place to update instead of going into each rule if I use the same information elsewhere like cross vlan etc, plus that information is still there if I hover over it.

On 3/5/2024 at 5:29 PM, go558a83nk said:

You've got a port forward rule for airvpn_wan_WG_0 but you're not showing a corresponding firewall rule for that interface for port forwarding.

The rule is there on the VL20_Port-Forward section but not in the AIRVPN rules though. I do have an outbound rule disabled in the screenshot as I was testing at the time. I edited it (and changed the discription) and set the gateway in the advanced section to try and force the packets to exit via the VPN_WAN. I should have included a screenshot of the rule. Like I said, I can see the traffic hit the server, but when the server replies, it's exiting via the WAN, not VPN_WAN gateway which shouldn't happen I'd have thought with my rules, but you might have a point, I'll redelete the rules and regenerate a new rule from the portforward.

But I'll go over your points and review/retest incase I've misunderstood your suggestions, but thanks go558a83nk for your suggestions.


  Edited ... by lewisw-j

[lewisw-j 0]

 

On 3/5/2024 at 7:58 PM, SurprisedItWorks said:

Ultimately I solved it with an "ip rule..." (in linuxspeak) command in the wireguard interface's up script

I've not seen that suggestion in my search online, this might be the only way to do it as right now, it's not acting like I'd expect. Again, I find it weird that OpenVPN works with these rules and Wireguard doesn't (well portforwarded packets are returning via the WAN, not wireguard-vpn-wan), so I'm thinking it's a routing or tagging issue maybe, idk. but thanks for the suggestion, I'll look into it
 

[lewisw-j 0]

 

On 3/6/2024 at 1:32 PM, benfitita said:

There was somewhere a post about a similar problem, and maybe it was on this forum, pfsense or opnsense. I can't find it though. As far as I remember, the problem was with the autogenerated firewall rule. It didn't have proper gateway (or something similar) in advanced options. You can't modify auto-generated rules, so you need turn off auto-generated rule in port forwarding rule, setup it up manually and ensure gateway (or a similar option) is properly set.

PS. sorry for the vague post, but I hope this at least gives you some pointers where to look.

I did see some posts changing the rules from automatic to hybrid/manual, Mine is set to manual as I really don't want rules being made that I didn't explicetly make myself. That said, I really want to try and replicate this issue on Opnsense, just not looking forward to the few hours of slowly duplicating all the settings over to find that it's still not working because of a setting I'm doing, but I guess there's only one way to find out
 

[lewisw-j 0]

Just an update from myself, I followed the guide and built an opnsense firewall which does the exact same thing:

Expected:
    Inbound:
        AirVPN (Wireguard Server) -> Firewall (Wireguard GW) - > VPN Server
    Outbound:
        VPN Server -> Firewall (Wireguard GW) -> AirVPN (Wireguard Server)

Result:
    Inbound:
        AirVPN (Wireguard Server) -> Firewall (Wireguard GW) - > VPN Server
    Outbound:
        VPN Server -> Firewall ? -> WAN

So I don't think it's pfsense itself if opnsense is doing the same thing, it's either something fundamental or my config, I'll keep digging.
That said, I'm seeing a few posts on reddit with the same issues:
     https://www.reddit.com/r/PFSENSE/comments/voklsu/anyone_gotten_port_forward_to_work_wwireguard/
     https://www.reddit.com/r/PFSENSE/comments/11x60g2/wantowireguardtolan_replyto_bug/

The second link has two workarounds, I guess I'll have to stick with that and figure out how to apply to my setup. Thanks anyway