Configuring InterVLAN Routing with a Layer 3 Switch and pfSense

Recently I was tasked with deploying a Layer 3 managed network switch alongside an existing pfSense firewall appliance for a relatively small network.

As a quick bit of a background the network consisted of around 10 VLANs which were all being terminated and routed on a pfSense firewall connected to an existing Layer 2 switch via a single 1Gbps trunk link (Router on a stick). There was then a requirement to swap out the existing Layer 2 switch and put a Layer 3 switch in its place to handle interVLAN routing between the VLANs to save resources on the firewall whilst increasing performance.

So to start this off I began documenting and making a high-level list of the steps:

  1. Remove the VLAN interfaces off PfSense
  2. Create the SVIs for each VLAN interface on the Layer 3 switch
  3. Enable IP Routing on the Layer 3 switch
  4. Configure the uplink port to pfSense LAN interface as a Routed Port
  5. Add static routes on pfSense back to the Layer 3 switch for each network
  6. Add firewall/NAT rules on pfSense for each network
  7. Add a default route on Layer 3 switch to PfSense

Note: I’m not going go into detail on removing interfaces on PfSense or creating VLANs, I already assume you are familar with this. In this example the switch configuration is based off a Cisco Catalyst 3560X, the steps may be different for other switch vendors. For Cisco you will need an IOS image and/or license which enables routing features.

First is to create the SVIs for each VLAN interface on Layer 3 switch:

Switch(config)# interface Vlan3
Switch(config-if)# ip address 172.16.3.1 255.255.255.0

Switch(config)# interface Vlan4
Switch(config-if)# ip address 172.16.4.1 255.255.255.0


Then we enable IP Routing globally on the switch:

Switch(config)# ip routing

The next stage is to configure the physical uplink going from the switch to the pfSense LAN interface. This can be referred to as a “Transit” network for traffic leaving the Layer 3 switch i.e. to the Internet. There a few ways this can be achieved, either by creating a dedicated VLAN interface with an SVI or configuring a physical switch port as a Routed Port using the “no switchport” command then giving it a dedicated IP address – I will be using this method but in most cases it is normally recommended to use a small subnet mask such as a /30 for the transit network.

In this example 172.16.1.1 will be the routed port IP address and 172.16.1.2 will be the pfSense LAN interface address.

Switch(config)# interface GigabitEthernet1/4
Switch(config-if)# description Routed Port to pfSsense LAN Interface
Switch(config-if)# no switchport
Switch(config-if)# ip address 172.16.1.1 255.255.255.252

For pfSense to know about the networks we need add static routes back to Layer 3 switch. First to go System > Routing > Gateways and click “Add” and enter the IP address of the Layer 3 switch routed port.

Under System > Routing > Static Routes click “Add” and add each of the networks for the various VLANs on the Layer 3 switch, selecting the Layer 3 Switch as the gateway.

For hosts in each of the various VLANs to get out to the internet Firewall and Outbound NAT rules must be created for each network on pfSense. Firstly, navigate to Firewall > NAT > Outbound and check the existing rules – if using automatic outbound NAT pfSense will have already added in the required rules for the networks otherwise these will need to be added manually.

Next navigate to Firewall > Rules > LAN and add pass rules for the various networks.

At this point pfSense is now aware of each of the networks on the Layer 3 switch and is configured to route their traffic outbound to the Internet. The last and final stage is to add a default route for all traffic not destined for the Layer 3 switch to pfSense – this will provide each of the VLANs with Internet access.

To do this login to the Layer 3 Switch and enter the following command:

Switch(config)# ip route 0.0.0.0 0.0.0.0 172.16.1.2

Now InterVLAN routing should be working successfully on the Layer 3 switch and the hosts on each of those networks should have Internet access through the pfSense firewall.

With this setup there are couple of things to keep in mind…

  • Restricting traffic between each of the VLANs must be performed by creating ACLs (Access Control Lists) on the Layer 3 switch as opposed to using Firewall rules on pfSense – this can be less flexible and user friendly.
  • Adding additional VLAN SVIs on the Layer 3 switch will require adding the appropriate static routes and Firewall/NAT rules to pfSense for those networks to enable Internet access if needed.

I hope this helps anyone looking to configure InterVLAN routing with a Layer 3 switch and pfSense.

Over and out! 🙂


24 Replies to “Configuring InterVLAN Routing with a Layer 3 Switch and pfSense”

  1. Thanks, just what i’m searching for. Btw, what do you use for dhcp/dns server in this scenario? Is pfSense DHCP/DNS still possible?

    • Hi Andy, glad I was able to help.

      In this scenario the environment was using Active Directory so internal DNS was handled by the Domain Controllers and DHCP was running on two domain-joined Windows Servers in a failover configuration. The IP Helper command was set under each VLAN interface to specify each of the DHCP servers e.g “ip helper-address 172.16.4.13”

      Currently, pfSense can only use DHCP on directly connected interfaces/sub interfaces where it must be the gateway for any VLANs. As a workaround you could enable and configure DHCP on your Layer 3 switch.
      https://www.ciscopress.com/articles/article.asp?p=1574301&seqNum=5

      As for DNS, if you aren’t using Active Directory then you could simply point clients to the transit LAN address of pfSense as their DNS server. So long as pfSense has external DNS servers specified under General Setup the DNS resolver will handle these requests.

      Hope this helps 🙂

      • Greig,
        Appreciate the quick response. Not running AD so decided spin up Debian vm with isc-dhcp to test with pfSense DNS. Thanks!

        • That would work also, anything that is capable of running a DHCP sever with multiple scopes really. So long as an IP Helper address for the DHCP server is added under each VLAN interface you should be good to go.

  2. I came across this thread because I’ve been basically deploying this setup for a while and wanted to have a true DMZ setup. My setup is slightly different than this one but it works basically the same. I’ll try to give a little background without being too wordy:

    -pfSense has a physical link to my Cisco switch
    -pfSense LAN has an ip address outside of any others (just because it forced me to give it an IP address. It’s 10.5.0.3 just fyi)
    -pfSense has all VLANs that are set on the switch set under VLANs.
    -All of these corresponding VLANs are set up as individual interfaces that ride the physical LAN connection between the 2 boxes.
    -VLANs are 5, 6, 7, 8, 9, 411, 222
    -VLAN IP addresses on the switch are:
    5 = 10.0.0.1/24
    6 = 10.6.0.1/24
    7 = 10.7.0.1/24
    8 = 10.8.0.1/24
    9 = 10.9.0.1/24
    (Transit) 411 = 172.20.1.1/24
    (DMZ) 222 = 192.168.5.1/24

    On the pfsense:
    VLAN 5 Interface = 10.0.0.2/24
    VLAN 6 Interface = 10.6.0.2/24
    VLAN 7 Interface = 10.7.0.2/24
    VLAN 8 Interface = 10.8.0.2/24
    VLAN 9 Interface = 10.9.0.2/24
    VLAN 411 Interface = 172.20.1.2/24
    VLAN 222 Interface = 192.168.5.2/24

    -Only 1 gateway needed to be set in pfSense this way, which is the default gateway to the WAN. This method allows you to see the traffic from the pfSense graphs on each interface. All traffic from the switch traverses across VLAN 411 and vice versa. All is happy without having to set static routes on the pfSense.

    Now for the problem 🙁

    I’ve set proper ACLs up on the switch to isolate different VLANs for various reasons. Works great. You can still “see” the segregated traffic in the states table on the pfSense if you look in the transit VLAN, which is to be expected. Not a big deal since the separated VLANs thus far were for things like IoT, guest networks, etc.

    What I’m finding regarding the DMZ is that even when I set my ACLs up on my switch to deny access to all other VLANs for the DMZ, including my transit (for reference again, the transit is 172.20.1.1 on the switch) and only allow it to go out through the corresponding DMZ VLAN interface that has been setup on the pfSense, it still takes a route through my transit. I’m guessing because it is the default gateway or gateway of last resort (I have it set on the switch as 0.0.0.0 0.0.0.0 172.20.1.2).

    This seems strange to me because I’ve set up an additional static route on the switch as 192.168.5.0 255.255.255.0 192.168.5.2 (Again, 192.168.5.1 is my switch DMZ VLAN address and .2 is the DMZ interface address on the pfSense). I even made a gateway of 192.168.5.1 and put it on the DMZ interface, set a route of 192.168.5.2/32 to 192.168.5.1 on the pfSense (it wouldn’t let me do the /24 since the interface is using that exact CIDR) and that was no help.

    If I put a rule directly on the pfSense transit VLAN interface to block specifically IP addresses of the DMZ, then they don’t hit that interface, but it also breaks internet access for the DMZ. I’m really not sure what to do here now. Oh, I also do have IP Helper addresses set on each VLAN on the switch. These addresses correspond to the ones on the pfSense as a “just in case”, since pfSense is also my DHCP server.

    If anybody has any ideas, I’d be forever grateful, thanks!
    Sorry for the lengthy post!

    • Hi Donnell,

      You will have to choose which device handles all inter-VLAN routing, the firewall or layer 3 switch, you cannot have both devices handling routing for all networks at the same time otherwise you could end up with some asymmetric routing issues.

      In many environments, layer 3 switches handle inter-VLAN routing between internal subnets where higher network throughput is required such as between clients and servers. DMZ networks typically have their interfaces terminate on the firewall rather than an SVI on a layer 3 switch, that way packet filtering and inspection can be applied.

      For a DMZ I would create the VLAN ID on the layer 3 switch without an SVI then designate a port on your layer 3 switch as a trunk port for that VLAN to connect up to a spare interface on the firewall. From there you can assign this interface an IP address in the desired subnet you want.

      There are ways of achieving packet filtering at the firewall level while keeping the SVIs on the layer 3 switch by utilising VRFs with “zoning” however this can be complex and is normally only utilised within large scale data centre environments. It also requires switches with more advanced feature sets or higher licensing in order to be used.

      To prevent having to create static routes on the firewall for internal networks behind the layer 3 switch you can use a dynamic routing protocol such as OSPF or BGP between the layer 3 switch and pfSense, this will also depend on the license and/or feature set on your switch. Note, you will still need to have a default static route on your Layer 3 switch to forward onto pfSense but you won’t require any static routes on pfSense itself for the networks on the layer 3 switch.

      Quagga and FRR are two packages available in pfSense that can do dynamic routing. I would generally recommend using FRR these days but both are essentially the same to a degree.
      https://docs.netgate.com/pfsense/en/latest/recipes/dynamic-routing-basics.html

      I will maybe revisit this post in the near future and cover configurating OSPF between a Cisco Layer 3 switch and pfSense.

      I hope this helps! 🙂

    • Hi Mal,

      There weren’t any major issues with pfSense in the original configuration it handled inter-VLAN routing just fine. It was more to do with increasing bandwidth for traffic traversing between data heavy VLANs. Originally all VLANs were trunked from the switch to pfSense using a single 1Gbps ethernet connection which was fine for a small number of VLANs where traffic utilisation was small, however this then became a bottleneck for bigger networks which had larger data requirements such as Servers and VDI.

      Normally this could be overcome by combining 2 or more links into a LAGG using LACP from the switch to pfSense, however in my situation there was a lack of available physical ports between the switch and firewall.

      From a performance standpoint, a layer 3 switch will typically be much faster at handling inter-VLAN routing and packet processing compared to a firewall as it able to do this at wire speed using hardware ASICs vs a firewall which will be be required to use CPU cycles and the physical bandwidth that is available between it and a switch.

      With that said, you do loose some flexibility regarding granular control of traffic and packet filtering as unlike firewalls, switch Access Control Lists are not stateful and can become cumbersome to manage and maintain compared to firewall rules.

      It all depends on your requirements really. In previous places where I have worked inter-VLAN routing was usually handled via very large and powerful layer 3 multi-gigabit switches which acted as the core/distribution for the entire network. Getting a similar level of performance from firewalls probably wouldn’t have been possible and the costs involved in doing so would also have been much more compared to the layer 3 switches.

  3. First off thank you for the details guide, it helps me a lot! I’m in the plannign stage for my home network, I would like a pfSense and a Cisco 3750G switch and a couple of L2 swithces, along with a few WAPs

    In step 4, you configured the uplink port to pfSense as a Routed Port. From what I understand, the whole subnet (/24 or /30, whatever) will be dedicated to these two devices, and if I have more l2 switches and WAPs, they need to be on a another subnet for management. So what about the second option, i.e, creating a Transit or Management VLAN and put all network infrastructurs together? Can you give me a hint on how to go about to do that? Thank you so much!

    • Hi Ewon,

      That is correct, if you have more Layer 2 switches or other network devices you wish to manage such as WAPs then you would ideally create a dedicated management VLAN for these rather than using the transit network between the Layer 3 switch and firewall as this is more secure.

      In my scenario I would typically create the management VLAN ID on the Layer 3 switch and assign an SVI to it. From there I would trunk this VLAN ID down to my additional switches and WAPs. If your Layer 3 switch in handling inter-vlan routing then any downstream Layer 2 switches will need their default gateway address set to the management VLAN SVI on the Layer 3 switch.

      If you are using multiple VLANs on WAPs such as one for a WiFi subnet and another for the management VLAN then you would also set the switch ports connecting to these as trunk ports and add both VLAN IDs. You may also want to create a DHCP scope for your management VLAN to prevent having to statically assign devices.

      For example:

      Switch 1:

      SW1(config)#vlan 3
      SW1(config-vlan)#name “Management VLAN”
      SW1(config-vlan)#exit

      SW1(config)#vlan 7
      SW1(config-vlan)#name “WiFi VLAN”
      SW1(config-vlan)#exit

      SW1(config)#interface vlan 3
      SW1(config-if)#ip address 172.16.3.1 255.255.255.0
      SW1(config-if)#no shutdown
      SW1(config-if)#exit

      SW1(config)#interface vlan 7
      SW1(config-if)#ip address 172.16.7.1 255.255.255.0
      SW1(config-if)#no shutdown
      SW1(config-if)#exit

      SW1(config)#interface gi0/24
      SW1(config-if)#description “Trunk Port to Switch 2”
      SW1(config-if)#switchport mode trunk
      SW1(config-if)#switchport trunk allowed vlan add 3
      SW1(config-if)#no shutdown

      Switch 2:

      SW2(config)#vlan 3
      SW2(config-vlan)#name “Management VLAN”
      SW2(config-vlan)#exit

      SW2(config)#vlan 7
      SW2(config-vlan)#name “WiFi VLAN”
      SW2(config-vlan)#exit

      SW2(config)#interface gi0/24
      SW2(config-if)#description “Trunk Port to Switch 1”
      SW2(config-if)#switchport mode trunk
      SW2(config-if)#switchport trunk allowed vlan add 3
      SW2(config-if)#no shutdown

      SW2(config)# interface gi0/23
      SW2(config-if)#description “WAP Trunk Port”
      SW2(config-if)#switchport mode trunk
      SW2(config-if)#switchport mode trunk allowed vlan add 3,7

      SW2(config)# ip default-gateway 172.16.3.1

      I hope this helps.

  4. Hi!

    I followed your steps and it worked like magic!! so happy that my layer 3 switch is able to ping google now!

    However, I have 1 problem, when I connected windows server 2016 assigned to VLAN 10 to the layer 3 switch, it can ping the other vlans but cannot ping pfsense. switchport access to the vlan was done but no internet still. What seems to be the issue? Hoping for your response, that would help a lot!! 🙂

    • Hi Nicole,

      Many thanks.

      For the issue you are having on the Windows Server 2016 machine, I would first go back to pfSense and double check the appropriate firewall rules have been added for the VLAN 10 subnet under Firewall > Rules > LAN. Also check that Outbound NAT rules are configured properly for the VLAN 10 subnet.

      Finally make sure you have configured the appropriate static routes on pfSense for the VLAN networks behind the Layer 3 switch.

      I hope this helps.

  5. In step 4 you configured the uplink port to pfSense as a Routed port. Can you explain how to use a vlan? I have a few L2 switches and APs that I would put them here as well. Say my pfSense is at 10.1.0.254, L3 switch VLAN SVI at 10.1.0.1, can I use /24 mask? Is the port a trunk or access? Thanks

    • Hi Ewon,

      Typically, for security reasons you would put other Layer 2 switches and APs on a dedicated management VLAN and keep the /30 transit network between the Layer 3 switch and firewall.

      If you wanted to use a standard switchport and VLAN for this rather than a routed port then you could do the following:

      SW1(config)#vlan 2
      SW1(config-vlan)#name “Transit VLAN”
      SW1(config-vlan)#exit

      SW1(config)#interface vlan 2
      SW1(config-if)#ip address 172.16.2.1 255.255.255.252
      SW1(config-if)#no shutdown
      SW1(config-if)#exit

      SW1(config)#interface gi0/22
      SW1(config-if)#description “Uplink Port to pfSense Firewall”
      SW1(config-if)#switchport mode access
      SW1(config-if)#switchport access vlan 2
      SW1(config-if)#no shutdown

      With this configuration the pfSense LAN interface would then be assigned 172.16.2.2/30.

      I hope this helps.

      • I get it, a small (/30) subnet mask for the router and core switch is the recommended way to go, being a transit VLAN or routed port. Network is just so much fun. It requires a lot of thinking and planning, but when in the end everything connects and works, phew, it feels like magic

  6. Hi!

    I find this helpful for our GNS3 capstone project. In our setup, we have 2 multilayer switches configured with VRRP per vlan group. We have 2 different gateways(addresses of both MLS1 and MLS2) and we declared them in pfsense(as you did in this guide). However, setting static routes on pfsense means that we can only use a single gateway in pfsense for our VLAN networks. We cannot set 2 gateways for the same VLAN group. Do you have any recommendations for the best thing to do in our setup? Thank you very much.

    • Hi Odinson,

      In your scenario the best thing to do would be avoiding the use of static routes between your Layer 3 switches and pfSense and opt for a dynamic routing protocol such as OSPF.

      pfSense supports OSPF with the FRR Package.

      If you really wanted to continue using static routes then your Layer 3 switches would either need to support Stacking or use MLAG technology such as Cisco’s VPC. This would allow you to connect both switches to pfSense via an LACP Port-Channel link and from there you would only need to specify the VRRP VIP address used for the transit network between the switches and pfSense.

      It has been a while since I have used VRRP but from what I remember there would normally be 3 IP addresses required for each VLAN. One for each VRRP member and a shared VIP address that would be used between each that would act as the gateway address for clients behind the switches.

      In the case of VLAN SVIs you would generally have something similar to below. Note: I’m assuming that VLAN 10 is the transit VLAN between your Layer 3 switches and pfSense

      Switch 1:

      SW1(config)# interface vlan 10
      SW1(config-if-vlan10)# ip address 10.10.10.2
      SW1(config-if-vlan10)# vrrp 1
      SW1(config-if-vlan10)# vrrp 1 10.10.10.1
      SW1(config-if)#vrrp 1 priority 10

      Switch 2:

      SW2(config)# interface vlan 10
      SW2(config-if-vlan10)# ip address 10.10.10.3
      SW2(config-if-vlan10)# vrrp 1
      SW2(config-if-vlan10)# vrrp 1 10.10.10.1
      SW2(config-if)#vrrp 1 priority 20

      Let’s assume that pfSense has 10.10.10.4 in this subnet. In which case you would define 10.10.10.1 as the gateway used for your static routes back to the the VLANs on the Layer 3 switches.

      I hope this helps.

  7. Very good write up. I followed it and got everything working except for my web servers. My NAT config that I used prior no longer works. Am I missing a static route.

    I basically have it like this

    wan->pfsense(nat)->vlan

    pfsense 10.5.1.1
    switch 10.5.1.2
    webserver:172.16.50.50

    i feel like it is probably something really simple. everything else works just fine. I just cant access my servers from my public ip.

    Any thoughts would be greatly appreciated

  8. In my scenario, I have configured network as follows.
    ISP Router –> pfSense * –> switch ( L2 )
    Luckily I have 5 X 1 Gbps ports. Hence I put 4 ports on pfSense as LAGG with 4 ports on the switch. I have created 7 vlans using Lagg port on pfSense only. With this, I have 7 interfaces on pfSense. Advantage is easy setup on switch and on pfSense. No need to define gateways on switch. I can create separate set of rules for each of vlan.
    And everything is working as expected.
    I am neither a Cisco / Hardcore Networking guy.
    Am I doing anything wrong ?

  9. Hi,
    We have two teams T1 and T2 working in a small office.
    T1 need full access to the internet whereas T2 need restricted (to few URLs) access to the internet.

    I have following setup:
    ISP Router pfsense cisco 3750g T1, T2

    Should I create vLANs on pfsense or on the switch?
    If I create vLANs on the switch, the directions in this article will work, right?

    If I create vLANs on pfsense I’d need some help configuring the switch.
    What’s the best method to create a whitelist to allow restricted internet access to T2?

    Thanks in advance.

Leave a Reply to Greig Mitchell Cancel reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.