Split tunnel VPN on UniFi USG

Let’s say sometimes you want to egress your home network over a VPN?  Maybe hide your traffic from your ISP who likes to snoop your traffic or insert ads?  Or maybe you want to get around geo-location blocks to stream some video only available in another country?  Installing a VPN client on your laptop is pretty easy, but might be harder on your Chromecast or other streaming device.

This article is going to try and provide a step-by-step how to configure your Ubiquiti USG series router/firewall + switch + AP to have a VLAN/SSID for “normal” mode and another VLAN/SSID for accessing the internet transparently over a VPN.  Devices you want to use the VPN just need to join the right WiFi network or have their switch port assigned the correct VLAN.  This config should also generally work for the EdgeRouter series, but you’ll need to do the configuration via the CLI instead of the JSON config file.  I suspect this should work on a DreamMachine or Dream Machine Pro, but I don’t own either of those and haven’t tested. (Nope, won’t work on the UDM or UDM-Pro. Neither support the config.gateway.json config file or the necessary policy routing features.)

First a shout out to rggn for making this post on the Ubiquiti forums. This how-to is significantly based on his work.

So before you start, you will need to have already configured your:

  • USG or USG Pro-4
  • Ubiquiti AP
  • Ubiquiti Managed Switch

and have that working on your network.   That means you will need some kind of UniFi Management Controller configured and managing your devices.

Please note that this guide is written around the USG. If you have the USG Pro-4 or other device, some details may be different for your hardware. For example, the interface names for the LAN and WAN port are different on the USG Pro-4. Hence, you’ll need to be extra careful just copy & pasting the contents below as they may not work for you.

Also, a warning about changing random values without knowing what you’re doing: some changes are perfectly benign and won’t be a problem. But other changes can cause weird issues or prevent the JSON config from being properly applied. Sadly, debugging such issues tends to be a real PITA. For example I picked things like “route table 100” instead of “route table 1” because the latter is reserved and will cause problems now. Same with vti64 because vti0 through 63 are reserved, but may nor not cause a problem depending on your config. Confusing? Yep. :(

As always, anytime you are editing the config.gateway.json if you are having problems, it helps to read the UniFi server logs on your controller or read the docs.

Step 1: Create a new Network for your VPN clients

This needs to be on your “internal” interface (LAN) and unless you have a good reason not to, I recommend setting the Purpose to be “Corporate”.  If for some reason, you want to create a “Guest” VPN network, you could do that, but you’ll have to change some of the later steps accordingly.  Write down the CIDR network address and VLAN ID you pick because you’ll need that info later.

Step 2: Create a new Wireless Network

Choose the VLAN from Step #1 and fill the rest of the form out as you’d like.  You probably want your User Group to be set to “Default”.

Step 3: Test your new network

Join this new SSID and make sure you can reach the internet normally.  If you selected this to be a “Corporate” network, you should be able to talk to devices on your normal/non-VPN network.

Step 4: Pick a VPN Provider

The USG can do like 40Mbps IPSec and 8Mbps OpenVPN.  USG Pro-4 should be able to do 150Mbps IPSec.   So I strongly recommend picking a vendor who can do IPSec.  I personally like Witopia PersonalVPN as they’re a lot less sketchy than many of the more popular services out there IMHO.  Next you’ll want to get their IPSec settings/config.  Hopefully your provider supports AES-128/SHA1 so you can use the hardware offloading on the USG.  You’ll want an IP address or hostname of at least one of their VPN servers to test with, your username/password and anything else you can find.

Step 5: Configure StrongSwan

You’re going to need to create some files on your USG.  For most people it’s easier to edit on your computer and then scp over the files and then move them to their final location.  You do you. Just realize you’ll need to be the root user (run `sudo su -`) in order to edit files in /config

If your VPN provider has a authentication certificate, you’ll need to place that in /etc/ipsec.d/cacerts

Create the StrongSwan IPSec config file. Edit as necessary. Note: Unfortunately, as far as I can tell, the USG doesn’t support AES-GCM, so we’re stuck with CBC mode. The USG will negotiate SHA2/256, but my testing shows performance is significantly impacted so you may want to stick with the old & deprecated SHA1.

/config/ipsec.conf

conn %default
   keyexchange = ikev2
   type = tunnel
   ike = aes256-sha1-modp2048
   esp = aes256-sha1

conn witopia
   dpddelay = 30s   # check peer liveness every 30s if there's no traffic
   dpdtimeout = 90s # peer is considered dead after 90s, re-establish IKE_SA
   dpdaction = restart
   reauth = yes
   ikelifetime = 24h
   rekey = yes
   auto = start

   leftsourceip = %config4   # auto-discovers eth0 IP address of USG which is DHCP
   leftsubnet = 0.0.0.0/0
   leftupdown = /config/ipsec-updown.sh
   leftauth = eap-md5              # eap-mschapv2 is also common
   eap_identity = YOUR_USER_NAME   # change this to your VPN username

   right = ipsec.sanfrancisco.witopia.net,ipsec.losangeles.witopia.net  # CSV list of VPN servers
   rightid = %any
   rightsubnet = 0.0.0.0/0
   rightauth = pubkey

This shell script is used to manage the VTI network interface for the VPN tunnel. You shouldn’t need to edit anything here 99% of the time.

/config/ipsec-updown.sh

#!/bin/bash

set -o nounset
set -o errexit

# $VTI_IFACE must match the interface in config.gateway.json
VTI_IFACE="vti64"

case "${PLUTO_VERB}" in
    up-client)
        echo "Creating tunnel interface ${VTI_IFACE}"
        ip tunnel add "${VTI_IFACE}" local "${PLUTO_ME}" remote "${PLUTO_PEER}" mode vti

        echo "Activating tunnel interface ${VTI_IFACE}"
        ip link set "${VTI_IFACE}" up

        echo "Adding ${PLUTO_MY_SOURCEIP} to ${VTI_IFACE}"
        ip addr add "${PLUTO_MY_SOURCEIP}" dev "${VTI_IFACE}"

        echo "Disabling IPsec policy (SPD) for ${VTI_IFACE}"
        sysctl -w "net.ipv4.conf.${VTI_IFACE}.disable_policy=1"

        DEFAULT_ROUTE="$(ip route show default | grep default | awk '{print $3}')"
        echo "Identified default route as ${DEFAULT_ROUTE}"
        echo "Adding route: ${PLUTO_PEER} via ${DEFAULT_ROUTE} dev ${PLUTO_INTERFACE}"
        ip route add "${PLUTO_PEER}" via "${DEFAULT_ROUTE}" dev "${PLUTO_INTERFACE}"
        ;;
    down-client)
        echo "Deleting interface ${VTI_IFACE}"
        ip tunnel del "${VTI_IFACE}"

        echo "Deleting route for ${PLUTO_PEER}"
        ip route del "${PLUTO_PEER}"
        ;;
esac

Note: Make sure to run `chmod 755 /config/ipsec-updown.sh` so the script is executable by StrongSwan.

/config/auth/ipsec.secrets

The password for your VPN authentication is stored in /config/auth/ipsec.secrets . It should have a single line in the format of:

USERNAME : EAP "PASSWORD"

Note: There are no quotes around your username (which must match your eap_identity in the ipsec.conf file), but there are quotes around your password.

/etc/strongswan.d/charon.conf

Create the /etc/strongswan.d/charon.conf file because we don’t want StrongSwan managing our routes:

charon {
    install_routes = no
    install_virtual_ip = no
}

By default, StrongSwan will update the DNS server config on your USG to use your VPN providers DNS servers (if they send that config as part of the VPN negotiation). If you don’t want to do that (maybe you’re running DNS over HTTPS using cloudflared) you’ll want to stop StrongSwan from editing /etc/resolv.conf on your USG. In that case, the above file needs to be a little different:

charon {
    install_routes = no
    install_virtual_ip = no
    plugins {
        resolve {
                file = /etc/resolv-ipsec.conf
        }
    }
}

That will cause StrongSwan to update a file that isn’t used by the USG operating system.

Next, bring up your IPSec tunnel and check its status by running the commands:


ipsec start
ipsec statusall

You may need to run `ipsec statusall` a few times as it can take a few seconds for IPSec to negotiate. What you’re looking for is the last lines of output to say something like this:

 Security Associations (1 up, 0 connecting):
     witopia[1]: ESTABLISHED 82 minutes ago, X.X.X.X[X.X.X.X]...45.89.173.148[ipsec.witopia.net]
     witopia[1]: IKEv2 SPIs: cd176258685bb875_i* 7d24710dcd73db6a_r, EAP reauthentication in 22 hours
     witopia[1]: IKE proposal: AES_GCM_16_256/PRF_HMAC_SHA2_384/ECP_521
     witopia{1}:  INSTALLED, TUNNEL, ESP SPIs: ccb98b7a_i c281efcd_o
     witopia{1}:  AES_CBC_256/HMAC_SHA1_96, 242860755 bytes_i (208182 pkts, 0s ago), 72228333 bytes_o (215696 pkts, 0s ago), rekeying in 8 minutes
     witopia{1}:   10.119.8.5/32 === 0.0.0.0/0

If not, check your config above and figure out what you got wrong and try:


ipsec restart
ipsec statusall

Until you get it right.

Step 6: Configure your UniFi Controller’s config.gateway.json

So now we have to configure some policy routing, so traffic from your VPN VLAN goes over the VPN and normal traffic exits normally. To make things simpler, I’m going to break up each block of the config file which on a CloudKey lives at /srv/unifi/data/sites/default/config.gateway.json. On my Ubuntu box, it’s at /var/lib/unifi/sites/default/config.gateway.json.

First, a really important thing. This file MUST BE VALID JSON. If you have one small mistake (like an extra comma) it won’t work. You may want to use a JSON linter like this one to make sure you don’t make a mistake.

Oh, and if you’re using an EdgeRouter instead of a USG, you’ll need to figure out the necessary CLI commands this JSON represents.

From top to bottom:

  1. Create firewall groups for each of my private networks. You’ll need this later because by default, UniFi places all your “Corporate” subnets in the same firewall group and that’s not good enough. Edit to taste.
  2. Disable source validation
  3. Create firewall rule(s) to mark traffic to use the VPN Tunnel. The key thing here is rule 1000 which marks traffic from our “vpn_network” to use routing table 100 so we can egress over the VPN. Edit to taste.
  4. Configure our maximum segment size for the VPN tunnel interface

{
    "firewall": {
        "group": {
            "network-group": {
               "main_network": {
                    "network": "172.16.1.0/24",
                    "description": "Main Network"
                },
                "vpn_network": {
                    "network": "172.16.2.0/24",
                    "description": "VPN SSID Network"
                },
                "work_network": {
                    "network": "172.16.3.0/24",
                    "description": "Work Network"
                }
            }
        },
        "source-validation": "disable",
        "modify": {
            "VPN_Tunnel": {
                "description": "VPN to Witopia",
                "rule": {
                    "1000": {
                        "description": "Policy Route from VPN network to vti64",
                        "log": "disable",
                        "action": "modify",
                        "modify": {
                            "table": "100"
                        },
                        "source": {
                            "group": {
                                "network-group": "vpn_network"
                            }
                        }
                    }
                }
            }
        },
        "options": {
            "mss-clamp": {
                "interface-type": [
                    "pppoe",
                    "pptp",
                    "vti"
                ],
                "mss": "1350"
            }
        }
    },
}

The next section tells UniFi that we’re using StrongSwan for the VPN. Probably shouldn’t need to edit anything here, or if you do, you’re advanced user. :)

    "vpn": {
        "ipsec": {
            "auto-firewall-nat-exclude": "enable",
            "include-ipsec-conf": "/config/ipsec.conf",
            "include-ipsec-secrets": "/config/auth/ipsec.secrets"
        }
    },

Configure our network interfaces. eth1 is the LAN and vti64 is the tunnel. If you’re using different hardware, you might need to change these values? The key thing here is we are saying that for inbound traffic on VLAN 100 (vif 100) on eth1 should use the “VPN_Tunnel” firewall rules we defined above. This completes the “policy” portion of our policy routing config.

    "interfaces": {
        "ethernet": {
            "eth1": {
                "vif": {
                    "100": {
                        "firewall": {
                            "in": {
                                "modify": "VPN_Tunnel"
                            }
                        }
                    }
                }
            }
        },
        "vti": {
            "vti64": {
                "description": "IPSec v2 VTI Interface for Witopia"
            }
        }
    },

Create routing table 100 with our default route over the VPN tunnel. This is the routing table clients on the VPN network will use. Add other routes if you need here. Note these routes only applies to traffic which matches firewall rule 2000 above. Edit this and those rules to taste.

    "protocols": {
        "static": {
            "table": {
                "100": {
                    "interface-route": {
                        "0.0.0.0/0": {
                            "next-hop-interface": {
                                "vti64": "''"
                            }
                        }
                    }
                }
            }
        }
    },

Make sure we have IPSec offload enabled for performance:

    "system": {
        "offload": {
            "ipsec": "enable"
        }
    },

This last block does a few things:

  1. If you want multicast for auto-discovery to work between your internal networks now that they’re split up, here’s how you do that. Just list the interfaces you want to repeat multicast messages between.
  2. Disable the built in NAT rules and replace them with our custom NAT rules. This is where those network-groups you defined in the very beginning of this step is useful because by default, UniFi will create a single “corporate-network” group which contains both the old and new networks and that won’t work.

    "service": {
        "mdns": {
            "repeater": {
                "interface": [
                    "eth1",
                    "eth1.100",
                    "eth1.200"
                ]
            }
        },
        "nat": {
            "rule": {
                "6001": {
                    "disable": "''"
                },
                "6002": {
                    "disable": "''"
                },
                "6003": {
                    "disable": "''"
                },
                "6010": {
                    "description": "Masq VPN network to vti64",
                    "log": "enable",
                    "protocol": "all",
                    "outbound-interface": "vti64",
                    "source": {
                        "group": {
                            "network-group": "vpn_network"
                        }
                    },
                    "type": "masquerade"
                },
                "6020": {
                    "description": "Masq Main Network to eth0",
                    "log": "enable",
                    "protocol": "all",
                    "outbound-interface": "eth0",
                    "source": {
                        "group": {
                            "network-group": "main_network"
                        }
                    },
                    "type": "masquerade"
                },
                "6030": {
                    "description": "Masq Work Network to eth0",
                    "log": "enable",
                    "protocol": "all",
                    "outbound-interface": "eth0",
                    "source": {
                        "group": {
                            "network-group": "work_network"
                        }
                    },
                    "type": "masquerade"
                }
            }
        }
    }
}

Step 7: Almost there!

Do a force provision of your USG so it gets the json config you just created. If you don’t have any syntax or schema errors, it might even work the first time. :) You can use a service like http://whatismyipaddress.com or just ask google and see if your IP address changes between your normal network and the VPN network.

Lastly, you’ll probably want to add a cron job on your USG to monitor your VPN and restart it should it go down for any reason:

Create /etc/cron.d/vpn-monitor

* * * * * root /usr/sbin/ipsec statusall | grep '0 up' && /usr/sbin/ipsec up witopia

Note that the name `witopia` has to match what you called your VPN in your /config/ipsec.conf file and you need to make this file has the permissions 644: `chmod 644 /etc/cron.d/vpn-monitor`.

You’re done!

If you reached this point, you’re done. Enjoy a beer or beverage of choice for a job well done!

One more thing

If you like this how-to and are interested in giving Witopia PersonalVPN a try, consider using my referal code. You get a 15% discount and I get an equal credit on my account. Note: Per the PersonalVPN TOS, you should sign up for the “Premier” service if you’re using their service with a router, but the “Basic” service should work just fine.

11 thoughts on “Split tunnel VPN on UniFi USG

  1. Hey thanks for the detailed tutorial, got it working! I’m using protonvpn just like the forum post.
    However i cannot get strongswan to respect the 24h reauth time, instead it keeps using the default 3h. I have tried using the file as in the original post and like in your example and both yield the same results.
    Here is what my file looks like if:
    conn %default
    keyexchange = ikev2
    type = tunnel

    conn protonvpn
    dpddelay = 30s # check peer liveness every 30s if there’s no traffic
    dpdtimeout = 90s # peer is considered dead after 90s, re-establish IKE_SA
    dpdaction = restart
    reauth = yes # ProtonVPN sets IKE_SA lifetime, see notes
    ikelifetime = 24h
    rekey = yes
    auto = start # can’t use auto=route, see notes
    keyexchange = ikev2
    type = tunnel

    leftsourceip = %config4
    leftsubnet = 0.0.0.0/0
    leftupdown = /config/ipsec-updown.sh
    leftauth = eap-mschapv2
    eap_identity = my_id_here

    right = us-fl-05.protonvpn.com,us-fl-21.protonvpn.com
    rightid = %any
    rightsubnet = 0.0.0.0/0
    rightauth = pubkey

    • Don’t know off the top of my head- but good chance ProtonVPN wants a 3hr rekey time and it’s being forced on that end. You should see what their proposal is and maybe check the logs (turn on verbose logging)

    • First off thanks synfinatic for this awesome post. I’ve currently got my USG doing the same via PPTP client (not as secure) and JSON file.

      Was going to try this with ProtonVPN too. Pardon my ignorance but what were/are the down sides to the 3hr reauth time vs the 24hr? In my application I’m just putting smart devices on the VPN network for extra security.
      Thanks in advance.

  2. Hi!
    thank you very much for this fantastic guide! it helped me a lot to understand how the rules of my usg4P works.
    I have successfully created rules that allow me to use vlan 40 to browse through an OpenVPN tunnel, the problem is that all devices within this vlan no longer see any other device placed on the others vlan on my network! Could you help me to understand what I need to add to the config.gateway.json file to allow this?

    It would be useful for example to allow a PC or a tablet on VLAN 40 (tunneled) to see my NAS on the main VLAN or on the IoT!

    Thank you really much for your time

    Here my config.gateway.json:

    {
    “firewall”:{
    “modify”:{
    “PBR_VPN”:{
    “rule”:{
    “20”:{
    “action”:”modify”,
    “description”:”traffic from VLan 40 to VPN Tunnel”,
    “modify”:{
    “table”:”20″
    },
    “source”:{
    “address”:”192.168.40.0/24″
    }
    }
    }
    }
    },
    “source-validation”:”disable”
    },
    “interfaces”:{
    “ethernet”:{
    “eth0”:{
    “vif”:{
    “40”:{
    “firewall”:{
    “in”:{
    “modify”:”PBR_VPN”
    }
    }
    }
    }
    }
    },
    “openvpn”:{

    “vtun0”:{
    “config-file”:”/config/user-data/openvpn/finland.nordvpn.com.udp.ovpn”,
    “description”:”OpenVPN Tunnel”
    }
    }
    },
    “protocols”:{
    “igmp-proxy”: {
    “interface”: {
    “eth0”: {
    “role”: “upstream”,
    “threshold”: “1”,
    “alt-subnet”: “0.0.0.0/0”
    },
    “eth0.10”: {
    “role”: “downstream”,
    “threshold”: “1”,
    “alt-subnet”: “0.0.0.0/0”
    }
    }
    },
    “static”:{
    “table”:{
    “20”:{
    “interface-route”:{
    “0.0.0.0/0”:{
    “next-hop-interface”:{
    “vtun0″:”””
    }
    }
    }
    }
    }
    }
    },
    “service”:{
    “nat”:{
    “rule”:{
    “5020”:{
    “description”:”OpenVPN Clients”,
    “log”:”disable”,
    “outbound-interface”:”vtun0″,
    “source”:{
    “address”:”192.168.40.0/24″
    },
    “type”:”masquerade”
    }
    }
    }
    }
    }

    • Is your VPN network is in the Corporate firewall group? Have you looked in the firewall logs to see why packets are being dropped? But my guess is there aren’t any firewall rules allowing that network to talk to the others.

      • Hi, thanks for reply,
        Yes it is, the network is located in the corporate group! what log file should i check in order to do this?
        when you talk about firewall roules you mean those created through the graphical interface? In that case I created one to let the networks communicate.
        “Putting me” in the tunneled VLAN and performing a traceroute to an ip of a NAS device within another vlan, the packets are always forwarded in the tunnel.

        maybe i should create some rules in the json file? if so how?

        Thanks again

        Fabio

        • What log file? Sorry, don’t know off the top of my head and I don’t actually use a USG anymore personally.

          I do notice you didn’t follow the instructions in the NAT rules section. you may need to disable some built in rules and you’ll need to add rules for the non-VPN subnet.

          Alternatively, you may need to add routes to your routing table 20 for the local subnets. I didn’t have to do that, but maybe something about your config is preventing the local subnets from being automatically added.

          In general, this setup is pretty complicated. I strongly suggest coping it as close as possible to avoid issues.

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.