Configuring InterVLAN Routing with a Layer 3 Switch and pfSense

Recently I was tasked with deploying a Layer 3 managed network switch alongside an existing pfSense firewall appliance for a relatively small network.

As a quick bit of a background the network consisted of around 10 VLANs which were all being terminated and routed on a pfSense firewall connected to an existing Layer 2 switch via a single 1Gbps trunk link (Router on a stick). There was then a requirement to swap out the existing Layer 2 switch and put a Layer 3 switch in its place to handle interVLAN routing between the VLANs to save resources on the firewall whilst increasing performance.

So to start this off I began documenting and making a high-level list of the steps:

  1. Remove the VLAN interfaces off PfSense
  2. Create the SVIs for each VLAN interface on the Layer 3 switch
  3. Enable IP Routing on the Layer 3 switch
  4. Configure the uplink port to pfSense LAN interface as a Routed Port
  5. Add static routes on pfSense back to the Layer 3 switch for each network
  6. Add firewall/NAT rules on pfSense for each network
  7. Add a default route on Layer 3 switch to PfSense

Note: I’m not going go into detail on removing interfaces on PfSense or creating VLANs, I already assume you are familar with this. In this example the switch configuration is based off a Cisco Catalyst 3560X, the steps may be different for other switch vendors. For Cisco you will need an IOS image and/or license which enables routing features.

First is to create the SVIs for each VLAN interface on Layer 3 switch:

Switch(config)# interface Vlan3
Switch(config-if)# ip address 172.16.3.1 255.255.255.0

Switch(config)# interface Vlan4
Switch(config-if)# ip address 172.16.4.1 255.255.255.0


Then we enable IP Routing globally on the switch:

Switch(config)# ip routing

The next stage is to configure the physical uplink going from the switch to the pfSense LAN interface. This can be referred to as a “Transit” network for traffic leaving the Layer 3 switch i.e. to the Internet. There a few ways this can be achieved, either by creating a dedicated VLAN interface with an SVI or configuring a physical switch port as a Routed Port using the “no switchport” command then giving it a dedicated IP address – I will be using this method but in most cases it is normally recommended to use a small subnet mask such as a /30 for the transit network.

In this example 172.16.1.1 will be the routed port IP address and 172.16.1.2 will be the pfSense LAN interface address.

Switch(config)# interface GigabitEthernet1/4
Switch(config-if)# description Routed Port to pfSsense LAN Interface
Switch(config-if)# no switchport
Switch(config-if)# ip address 172.16.1.1 255.255.255.252

For pfSense to know about the networks we need add static routes back to Layer 3 switch. First to go System > Routing > Gateways and click “Add” and enter the IP address of the Layer 3 switch routed port.

Under System > Routing > Static Routes click “Add” and add each of the networks for the various VLANs on the Layer 3 switch, selecting the Layer 3 Switch as the gateway.

For hosts in each of the various VLANs to get out to the internet Firewall and Outbound NAT rules must be created for each network on pfSense. Firstly, navigate to Firewall > NAT > Outbound and check the existing rules – if using automatic outbound NAT pfSense will have already added in the required rules for the networks otherwise these will need to be added manually.

Next navigate to Firewall > Rules > LAN and add pass rules for the various networks.

At this point pfSense is now aware of each of the networks on the Layer 3 switch and is configured to route their traffic outbound to the Internet. The last and final stage is to add a default route for all traffic not destined for the Layer 3 switch to pfSense – this will provide each of the VLANs with Internet access.

To do this login to the Layer 3 Switch and enter the following command:

Switch(config)# ip route 0.0.0.0 0.0.0.0 172.16.1.2

Now InterVLAN routing should be working successfully on the Layer 3 switch and the hosts on each of those networks should have Internet access through the pfSense firewall.

With this setup there are couple of things to keep in mind…

  • Restricting traffic between each of the VLANs must be performed by creating ACLs (Access Control Lists) on the Layer 3 switch as opposed to using Firewall rules on pfSense – this can be less flexible and user friendly.
  • Adding additional VLAN SVIs on the Layer 3 switch will require adding the appropriate static routes and Firewall/NAT rules to pfSense for those networks to enable Internet access if needed.

I hope this helps anyone looking to configure InterVLAN routing with a Layer 3 switch and pfSense.

Over and out! πŸ™‚


12 Replies to “Configuring InterVLAN Routing with a Layer 3 Switch and pfSense”

  1. Thanks, just what i’m searching for. Btw, what do you use for dhcp/dns server in this scenario? Is pfSense DHCP/DNS still possible?

    • Hi Andy, glad I was able to help.

      In this scenario the environment was using Active Directory so internal DNS was handled by the Domain Controllers and DHCP was running on two domain-joined Windows Servers in a failover configuration. The IP Helper command was set under each VLAN interface to specify each of the DHCP servers e.g “ip helper-address 172.16.4.13”

      Currently, pfSense can only use DHCP on directly connected interfaces/sub interfaces where it must be the gateway for any VLANs. As a workaround you could enable and configure DHCP on your Layer 3 switch.
      https://www.ciscopress.com/articles/article.asp?p=1574301&seqNum=5

      As for DNS, if you aren’t using Active Directory then you could simply point clients to the transit LAN address of pfSense as their DNS server. So long as pfSense has external DNS servers specified under General Setup the DNS resolver will handle these requests.

      Hope this helps πŸ™‚

      • Greig,
        Appreciate the quick response. Not running AD so decided spin up Debian vm with isc-dhcp to test with pfSense DNS. Thanks!

        • That would work also, anything that is capable of running a DHCP sever with multiple scopes really. So long as an IP Helper address for the DHCP server is added under each VLAN interface you should be good to go.

  2. I came across this thread because I’ve been basically deploying this setup for a while and wanted to have a true DMZ setup. My setup is slightly different than this one but it works basically the same. I’ll try to give a little background without being too wordy:

    -pfSense has a physical link to my Cisco switch
    -pfSense LAN has an ip address outside of any others (just because it forced me to give it an IP address. It’s 10.5.0.3 just fyi)
    -pfSense has all VLANs that are set on the switch set under VLANs.
    -All of these corresponding VLANs are set up as individual interfaces that ride the physical LAN connection between the 2 boxes.
    -VLANs are 5, 6, 7, 8, 9, 411, 222
    -VLAN IP addresses on the switch are:
    5 = 10.0.0.1/24
    6 = 10.6.0.1/24
    7 = 10.7.0.1/24
    8 = 10.8.0.1/24
    9 = 10.9.0.1/24
    (Transit) 411 = 172.20.1.1/24
    (DMZ) 222 = 192.168.5.1/24

    On the pfsense:
    VLAN 5 Interface = 10.0.0.2/24
    VLAN 6 Interface = 10.6.0.2/24
    VLAN 7 Interface = 10.7.0.2/24
    VLAN 8 Interface = 10.8.0.2/24
    VLAN 9 Interface = 10.9.0.2/24
    VLAN 411 Interface = 172.20.1.2/24
    VLAN 222 Interface = 192.168.5.2/24

    -Only 1 gateway needed to be set in pfSense this way, which is the default gateway to the WAN. This method allows you to see the traffic from the pfSense graphs on each interface. All traffic from the switch traverses across VLAN 411 and vice versa. All is happy without having to set static routes on the pfSense.

    Now for the problem πŸ™

    I’ve set proper ACLs up on the switch to isolate different VLANs for various reasons. Works great. You can still “see” the segregated traffic in the states table on the pfSense if you look in the transit VLAN, which is to be expected. Not a big deal since the separated VLANs thus far were for things like IoT, guest networks, etc.

    What I’m finding regarding the DMZ is that even when I set my ACLs up on my switch to deny access to all other VLANs for the DMZ, including my transit (for reference again, the transit is 172.20.1.1 on the switch) and only allow it to go out through the corresponding DMZ VLAN interface that has been setup on the pfSense, it still takes a route through my transit. I’m guessing because it is the default gateway or gateway of last resort (I have it set on the switch as 0.0.0.0 0.0.0.0 172.20.1.2).

    This seems strange to me because I’ve set up an additional static route on the switch as 192.168.5.0 255.255.255.0 192.168.5.2 (Again, 192.168.5.1 is my switch DMZ VLAN address and .2 is the DMZ interface address on the pfSense). I even made a gateway of 192.168.5.1 and put it on the DMZ interface, set a route of 192.168.5.2/32 to 192.168.5.1 on the pfSense (it wouldn’t let me do the /24 since the interface is using that exact CIDR) and that was no help.

    If I put a rule directly on the pfSense transit VLAN interface to block specifically IP addresses of the DMZ, then they don’t hit that interface, but it also breaks internet access for the DMZ. I’m really not sure what to do here now. Oh, I also do have IP Helper addresses set on each VLAN on the switch. These addresses correspond to the ones on the pfSense as a “just in case”, since pfSense is also my DHCP server.

    If anybody has any ideas, I’d be forever grateful, thanks!
    Sorry for the lengthy post!

    • Hi Donnell,

      You will have to choose which device handles all inter-VLAN routing, the firewall or layer 3 switch, you cannot have both devices handling routing for all networks at the same time otherwise you could end up with some asymmetric routing issues.

      In many environments, layer 3 switches handle inter-VLAN routing between internal subnets where higher network throughput is required such as between clients and servers. DMZ networks typically have their interfaces terminate on the firewall rather than an SVI on a layer 3 switch, that way packet filtering and inspection can be applied.

      For a DMZ I would create the VLAN ID on the layer 3 switch without an SVI then designate a port on your layer 3 switch as a trunk port for that VLAN to connect up to a spare interface on the firewall. From there you can assign this interface an IP address in the desired subnet you want.

      There are ways of achieving packet filtering at the firewall level while keeping the SVIs on the layer 3 switch by utilising VRFs with “zoning” however this can be complex and is normally only utilised within large scale data centre environments. It also requires switches with more advanced feature sets or higher licensing in order to be used.

      To prevent having to create static routes on the firewall for internal networks behind the layer 3 switch you can use a dynamic routing protocol such as OSPF or BGP between the layer 3 switch and pfSense, this will also depend on the license and/or feature set on your switch. Note, you will still need to have a default static route on your Layer 3 switch to forward onto pfSense but you won’t require any static routes on pfSense itself for the networks on the layer 3 switch.

      Quagga and FRR are two packages available in pfSense that can do dynamic routing. I would generally recommend using FRR these days but both are essentially the same to a degree.
      https://docs.netgate.com/pfsense/en/latest/recipes/dynamic-routing-basics.html

      I will maybe revisit this post in the near future and cover configurating OSPF between a Cisco Layer 3 switch and pfSense.

      I hope this helps! πŸ™‚

    • Hi Mal,

      There weren’t any major issues with pfSense in the original configuration it handled inter-VLAN routing just fine. It was more to do with increasing bandwidth for traffic traversing between data heavy VLANs. Originally all VLANs were trunked from the switch to pfSense using a single 1Gbps ethernet connection which was fine for a small number of VLANs where traffic utilisation was small, however this then became a bottleneck for bigger networks which had larger data requirements such as Servers and VDI.

      Normally this could be overcome by combining 2 or more links into a LAGG using LACP from the switch to pfSense, however in my situation there was a lack of available physical ports between the switch and firewall.

      From a performance standpoint, a layer 3 switch will typically be much faster at handling inter-VLAN routing and packet processing compared to a firewall as it able to do this at wire speed using hardware ASICs vs a firewall which will be be required to use CPU cycles and the physical bandwidth that is available between it and a switch.

      With that said, you do loose some flexibility regarding granular control of traffic and packet filtering as unlike firewalls, switch Access Control Lists are not stateful and can become cumbersome to manage and maintain compared to firewall rules.

      It all depends on your requirements really. In previous places where I have worked inter-VLAN routing was usually handled via very large and powerful layer 3 multi-gigabit switches which acted as the core/distribution for the entire network. Getting a similar level of performance from firewalls probably wouldn’t have been possible and the costs involved in doing so would also have been much more compared to the layer 3 switches.

Leave a Reply

Your email address will not be published. Required fields are marked *

*

This site uses Akismet to reduce spam. Learn how your comment data is processed.