16. VXLAN EVPN Multihoming

VXLAN EVPN Multihoming Configuration

This is the topology we used in the VXLAN EVPN configuration example1. before configuring multihoming, make sure that VXLAN EVPN is already configured in this topology.

VXLAN EVPN Configuration Example1

VXLAN EVPN Multihoming Reference

The only difference in this topology is that the CE switch is connected to two leaf switches, Leaf1 and Leaf2 and aggregated two links in Port-channel 2 in a way that if both links are connected to a single switch.

!!! CE

feature lacp
!
interface Ethernet1/1
  switchport mode trunk
  channel-group 2 mode active

interface Ethernet1/2
  switchport mode trunk
  channel-group 2 mode active

This can be possible with the help of Route Type 4 , which leaf switches advertise “Ethernet Segment Identifier” and “system-mac” between each other.

When two switches want to offer multihoming solution for a customer, the same “segment identifier” and “system-mac” must be configured in all interfaces from different switches that are connected to the same customer. So both switches understand that they participate in the same ethernet-segment.

!!! leaf1 / leaf2

evpn esi multihoming
!
interface port-channel 2
  switchport mode trunk
  switchport trunk allowed vlan 140-141
  ethernet-segment 7
    system-mac 0102.0304.0506
!
Interface Ethernet1/4
  Switchport mode trunk
  Switchport trunk allowed vlan 140,141
  Channel-group 2 mode active
!
interface eth1/1-2 
 evpn multihoming core-tracking

Shared system-mac helps that two switches send LACP packets with the same MAC address. therefore customer switch beleive that receiving LACP packets in different links come from a single switch.

With route type 4, in unlike vPC solution, we have no restriction on the number of switches. A customer can connect to more than two switches for the purpose of redundancy, but this does not seem to be so applicable.

As you can see in our topology, interface Eth1/4 of leaf1 and leaf2 switches are configured in port-channel 2 and the same ethernet-segment identifier and system-mac are configured for both leaf switches.  I have chosen Ethernet-segment identifier 7 and system-mac 0102.0304.0506 to be seen easily in BGP table.

With the command “maximum-path ibgp 2” in all leaf and spine switches, we allow all other leaf switches to choose both Leaf1 and Leaf2 switches as the best path to reach MAC and IP addresses behind Leaf1 and Leaf2 switches so traffic will be load balanced between leaf1 and leaf2 switches.

!!! leaf and spine switches

router bgp 65000
 address-family l2vpn evpn
  maximum-paths ibgp 2

The question now is how a loop is prevented when BUM traffic is forwarded. Fortunately, in shared segment, only one of the switches become Designated Forwarder (DF) for each VLAN, which is allowed to send or forward receiving BUM traffic in the shared ethernet segment. If, for example, leaf1 becomes DF in the shared segment for VLAN 10, this means that only leaf1 is allowed to forward BUM traffic sending or receiving in shared Ethernet segment if it belongs to VLAN 10.

soon, we will see how to monitor which switch is designated Forwarder for a particular VLAN.

with “evpn multihoming core-tracking” feature, if all uplinks from a leaf switch to spine switches fail, then shared ethernet segment connected to that leaf switch will be automatically shut down. Otherwise, traffic receiving in the leaf switch that has no connection to spine switches will be discarded.

VXLAN Multihoming Monitoring

Let’s check route type 4 in BGP L2VPN EVN address-family  with command “show bgp l2vpn evpn route-type 4”.

leaf1(config-if-range)# show bgp l2vpn evpn route-type 4
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 192.168.0.11:27001   (ES [0301.0203.0405.0600.0007 0])
BGP routing table entry for [4]:[0301.0203.0405.0600.0007]:[32]:[192.168.15.11]/
136, version 34
Paths: (1 available, best #1)
Flags: (0x000002) (high32 00000000) on xmit-list, is not in l2rib/evpn
Multipath: iBGP

  Advertised path-id 1
  Path type: local, path is valid, is best path, no labeled nexthop
  AS-Path: NONE, path locally originated
    192.168.15.11 (metric 0) from 0.0.0.0 (192.168.0.11)
      Origin IGP, MED not set, localpref 100, weight 32768
      Extcommunity: ENCAP:8 RT:0102.0304.0506

  Path-id 1 advertised to peers:
    192.168.0.1        192.168.0.2    
BGP routing table entry for [4]:[0301.0203.0405.0600.0007]:[32]:[192.168.15.12]/
136, version 36
Paths: (1 available, best #1)
Flags: (0x000012) (high32 00000000) on xmit-list, is in l2rib/evpn, is not in HW
Multipath: iBGP

  Advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop
             Imported from 192.168.0.12:27001:[4]:[0301.0203.0405.0600.0007]:[32
]:[192.168.15.12]/136 
  AS-Path: NONE, path sourced internal to AS
    192.168.15.12 (metric 81) from 192.168.0.1 (192.168.0.2)
      Origin IGP, MED not set, localpref 100, weight 0
      Extcommunity: ENCAP:8 RT:0102.0304.0506
      Originator: 192.168.0.12 Cluster list: 192.168.0.2 

  Path-id 1 not advertised to any peer

Route Distinguisher: 192.168.0.12:27001
BGP routing table entry for [4]:[0301.0203.0405.0600.0007]:[32]:[192.168.15.12]/
136, version 35
Paths: (2 available, best #2)
Flags: (0x000002) (high32 00000000) on xmit-list, is not in l2rib/evpn, is not i
n HW
Multipath: iBGP

  Path type: internal, path is valid, not best reason: Neighbor Address, no labe
led nexthop
  AS-Path: NONE, path sourced internal to AS
    192.168.15.12 (metric 81) from 192.168.0.2 (192.168.0.2)
      Origin IGP, MED not set, localpref 100, weight 0
      Extcommunity: ENCAP:8 RT:0102.0304.0506
      Originator: 192.168.0.12 Cluster list: 192.168.0.2 

  Advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop
             Imported to 1 destination(s)
  AS-Path: NONE, path sourced internal to AS
    192.168.15.12 (metric 81) from 192.168.0.1 (192.168.0.2)
      Origin IGP, MED not set, localpref 100, weight 0
      Extcommunity: ENCAP:8 RT:0102.0304.0506
      Originator: 192.168.0.12 Cluster list: 192.168.0.2 

  Path-id 1 not advertised to any peer

As you can see both leaf1 and leaf2 switches with next-hop address 192.168.15.11 and 192.168.15.12 advertise route-type 4 with ethernet-segment 7 and system-mac 01.0203.0405.06. so they understand that they participate in the same segment.

With the command “show nve ethernet-segment” you can monitor which switch for which VLAN “Designated Forwarder” is.

leaf1(config-if-range)# show nve ethernet-segment 

ESI: 0301.0203.0405.0600.0007
   Parent interface: port-channel2
  ES State: Up 
  Port-channel state: Up
  NVE Interface: nve1 
   NVE State: Up 
   Host Learning Mode: control-plane
  Active Vlans: 140-141 
   DF Vlans: 140 
   Active VNIs: 50140-50141 
  CC failed for VLANs:  
  VLAN CC timer: 0 
  Number of ES members: 2 
  My ordinal: 0 
  DF timer start time: 00:00:00 
  Config State: config-applied 
  DF List: 192.168.15.11 192.168.15.12  
  ES route added to L2RIB: True
  EAD/ES routes added to L2RIB: True
  EAD/EVI route timer age: not running 
----------------------------------------

leaf2(config-if-range)# show nve ethernet-segment 

ESI: 0301.0203.0405.0600.0007
   Parent interface: port-channel2
  ES State: Up 
  Port-channel state: Up
  NVE Interface: nve1 
   NVE State: Up 
   Host Learning Mode: control-plane
  Active Vlans: 140-141 
   DF Vlans: 141 
   Active VNIs: 50140-50141 
  CC failed for VLANs:  
  VLAN CC timer: 0 
  Number of ES members: 2 
  My ordinal: 1 
  DF timer start time: 00:00:00 
  Config State: config-applied 
  DF List: 192.168.15.11 192.168.15.12  
  ES route added to L2RIB: True
  EAD/ES routes added to L2RIB: True
  EAD/EVI route timer age: not running 
----------------------------------------