Saturday, June 24, 2017

VXLAN EVPN with Multi-Tenants Using Cisco Nexus 9000v

I just wanted to share my lab experience setting up, configuring and verifying VXLAN EVPN using Cisco Nexus 9000v and CSR1000v virtual platforms.  The Nexus NX-OSv was a great tool because it allows anyone to explore NX-OS technologies otherwise only reserved for those who had access to very expensive physical hardware Nexus 9K switches.  The CSR1000v is also a great tool and can be used to setup VXLAN flood and learn, however the NX-OSv opened the door to learn the next gen of datacenter technologies, VXLAN EVPN.

The information here is not meant to be a detailed tutorial on VXLAN EVPN but a configuration and operational look at the technology.  So please be aware that some previous knowledge of the technology is required.  The references section does contains a collection of links to some good documentation and useful articles.  I recommend anyone to read up on it.

Here’s a breakdown of the components used in my lab environment.
  • VMWare ESXi 6.5 (standalone)
  • (5) Cisco NX-OSv 9000: version 7.0(3)I6(1)
    • Used 2 vCPUs instead of the default 4.  It seems to run fine.
    • Used default RAM at 8Gb
  • (2) CSR1000v: version 16.4.2 [Everest]
    •  Used default vCPU setting of 1
    • Used default RAM at 4Gb
  • (8) Damn Small Linux (DSL) 4.x as test hosts

Diagram & Topology


  • Spine/Leaf Clos architecture
    • 2 Spines
    • 2 Leafs, 1 Border Leaf
  • Underlay Infrastructure
    • IP Address Space: 10.0.0.0/24
      • Each link addressed using /30's
    • IGP: OSPF
    • Multicast: Anycast RP PIM with RP configured on Spines
    • Control Plane Protocol: iBGP
    • BGP Route Reflectors configured on Spines
  • Multi-Tenant Setup - 2 Tenants
    • TENANT_A (Orange)
      • User IP Address Space: 172.16.0.0/22
    • TENANT_B (Gold)
      • User IP Address Space: 192.168.0.0/22
  • VLAN to VNI mapping
    • TENANT_A 
      • VLAN 100 → L2 VNI 10000
      • VLAN 200 → L2 VNI 20000
      • VLAN 900 → L3 VNI 90000
    • TENANT_B 
      • VLAN 300 → L2 VNI 30000
      • VLAN 400 → L2 VNI 40000
      • VLAN 910 → L3 VNI 91000
  • WAN Edge Router
    • eBGP to Border Leaf3
    • eBGP to WAN

Configurations

Spine_01

version 7.0(3)I6(1)
hostname DC1_SPINE_01
vdc DC1_SPINE_01 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim

no password strength-check
username admin password 5 <removed>  role network-admin
no ip domain-lookup
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:12:41:178:194:165
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:80:86:160:176:48
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 100.100.100.100 group-list 225.0.0.0/8
ip pim bsr rp-candidate loopback1 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8
ip pim anycast-rp 100.100.100.100 1.1.1.1
ip pim anycast-rp 100.100.100.100 2.2.2.2
vlan 1

vrf context management
  ip route 0.0.0.0/0 10.1.1.254 name MGMT_DEFAULT

interface Ethernet1/1
  description TO LEAF_01
  no switchport
  mtu 9216
  ip address 10.0.0.1/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description TO LEAF_02
  no switchport
  mtu 9216
  ip address 10.0.0.5/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  description TO LEAF_03
  no switchport
  mtu 9216
  ip address 10.0.0.17/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  shutdown
  mtu 9216

interface Ethernet1/5
  shutdown
  mtu 9216

interface mgmt0
  description MANAGEMENT
  vrf member management
  ip address 10.1.1.210/24

interface loopback0
  description RID
  ip address 1.1.1.1/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description MCAST ANYCAST RP
  ip address 100.100.100.100/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy run start
cli alias name ll sh ip route
cli alias name p ping
cli alias name tr traceroute
cli alias name q exit
cli alias name lo exit
cli alias name sr sh running-config
cli alias name sibsle show bgp l2vpn evpn summary
cli alias name sible sh bgp l2vpn evpn
cli alias name slema show l2route evpn mac all
cli alias name slemia show l2route evpn mac-ip all
cli alias name llm sh ip mroute
cli alias name sion sh ip ospf neighbors
line console
  exec-timeout 60
line vty
  exec-timeout 60
boot nxos bootflash:/nxos.7.0.3.I6.1.bin 
router ospf 100
  router-id 1.1.1.1
  passive-interface default
router bgp 65500
  router-id 1.1.1.1
  address-family l2vpn evpn
  neighbor 3.3.3.3
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 4.4.4.4
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 5.5.5.5
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client

Spine_02

version 7.0(3)I6(1)
hostname DC1_SPINE_02
vdc DC1_SPINE_02 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim

no password strength-check
username admin password 5 <removed> role network-admin
no ip domain-lookup
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:12:41:178:194:165
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:80:86:160:176:48
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

ip pim rp-address 100.100.100.100 group-list 225.0.0.0/8
ip pim bsr rp-candidate loopback1 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8
ip pim anycast-rp 100.100.100.100 1.1.1.1
ip pim anycast-rp 100.100.100.100 2.2.2.2
vlan 1

no cdp enable
vrf context management
  ip route 0.0.0.0/0 10.1.1.254 name MGMT_DEFAULT


interface Ethernet1/1
  description TO LEAF_02
  no switchport
  mtu 9216
  ip address 10.0.0.9/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description TO LEAF_01
  no switchport
  mtu 9216
  ip address 10.0.0.13/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  description TO LEAF_03
  no switchport
  mtu 9216
  ip address 10.0.0.21/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/4
  shutdown
  mtu 9216

interface Ethernet1/5
  shutdown
  mtu 9216

interface mgmt0
  description MANAGEMENT
  vrf member management
  ip address 10.1.1.211/24

interface loopback0
  description RID
  ip address 2.2.2.2/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode

interface loopback1
  description MCAST ANYCAST RP
  ip address 100.100.100.100/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy run start
cli alias name ll sh ip route
cli alias name p ping
cli alias name tr traceroute
cli alias name q exit
cli alias name lo exit
cli alias name sr sh running-config
cli alias name sibsle show bgp l2vpn evpn summary
cli alias name sible sh bgp l2vpn evpn
cli alias name llm sh ip mroute
cli alias name sion sh ip ospf neighbors
line console
  exec-timeout 60
line vty
  exec-timeout 60
boot nxos bootflash:/nxos.7.0.3.I6.1.bin 
router ospf 100
  router-id 2.2.2.2
  passive-interface default
router bgp 65500
  router-id 2.2.2.2
  neighbor 3.3.3.3
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 4.4.4.4
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client
  neighbor 5.5.5.5
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
      route-reflector-client

Leaf_01

version 7.0(3)I6(1)
hostname DC1_LEAF_01
vdc DC1_LEAF_01 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 <removed> role network-admin
no ip domain-lookup
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:12:41:178:194:165
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:80:86:160:176:48
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

vlan 1,100,200,300,400,900,910
fabric forwarding anycast-gateway-mac 0000.dc01.dc01
ip pim rp-address 100.100.100.100 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 100
  vn-segment 10000
vlan 200
  vn-segment 20000
vlan 300
  vn-segment 30000
vlan 400
  vn-segment 40000
vlan 900
  vn-segment 90000
vlan 910
  vn-segment 91000

vrf context TENANT_A
  vni 90000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context TENANT_B
  vni 91000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
  ip route 0.0.0.0/0 10.1.1.254 name MGMT_DEFAULT

interface Vlan1

interface Vlan100
  description TENANT_A VL100
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  description TENANT_A VL200
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan300
  description TENANT_B VL300
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan400
  description TENANT_B VL400
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan900
  description TENANT_A VL900 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip forward

interface Vlan910
  description TENANT_B VL910 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip forward

interface nve1
  no shutdown
  description VTEP_01
  source-interface loopback0
  host-reachability protocol bgp
  member vni 10000
    suppress-arp
    mcast-group 225.1.0.1
  member vni 20000
    suppress-arp
    mcast-group 225.2.0.1
  member vni 30000
    suppress-arp
    mcast-group 225.3.0.1
  member vni 40000
    suppress-arp
    mcast-group 225.4.0.1
  member vni 90000 associate-vrf
  member vni 91000 associate-vrf

interface Ethernet1/1
  description TO SPINE_01
  no switchport
  mtu 9216
  ip address 10.0.0.2/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description TO SPINE_02
  no switchport
  mtu 9216
  ip address 10.0.0.14/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  mtu 9216

interface Ethernet1/4
  description TO HOST4 - VL100
  switchport access vlan 100
  mtu 9216

interface Ethernet1/5
  description TO HOST5 - VL200
  switchport access vlan 200
  mtu 9216

interface Ethernet1/6
  description TO HOST1 - VL300
  switchport access vlan 300
  mtu 9216

interface Ethernet1/7
  description TO HOST12 - VL400
  switchport access vlan 400
  mtu 9216

interface mgmt0
  description MANAGEMENT
  vrf member management
  ip address 10.1.1.212/24

interface loopback0
  description RID - VTEP
  ip address 3.3.3.3/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy run start
cli alias name ll sh ip route
cli alias name p ping
cli alias name tr traceroute
cli alias name q exit
cli alias name lo exit
cli alias name sr sh running-config
cli alias name sibsle show bgp l2vpn evpn summary
cli alias name sible sh bgp l2vpn evpn
cli alias name snn sh nve vni
cli alias name snp sh nve peer
cli alias name slema show l2route evpn mac all
cli alias name slemia show l2route evpn mac-ip all
cli alias name lla sh ip route vrf TENANT_A
cli alias name svi show vxlan interface
cli alias name siasd show ip arp suppression-cache detail
cli alias name snipd sh nve internal platform interface nve1 detail
cli alias name snpd sh nve peer detail
cli alias name llm sh ip mroute
cli alias name sion sh ip ospf neighbors
cli alias name llb sh ip route vrf TENANT_B
cli alias name shmac show system internal l2fwder mac
cli alias name clms clear mac address-table datapath static
cli alias name clmd clear mac address-table datapath dynamic
line console
  exec-timeout 60
line vty
  exec-timeout 60
boot nxos bootflash:/nxos.7.0.3.I6.1.bin 
router ospf 100
  router-id 3.3.3.3
  passive-interface default
router bgp 65500
  router-id 3.3.3.3
  neighbor 1.1.1.1
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 2.2.2.2
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf TENANT_A
    address-family ipv4 unicast
      advertise l2vpn evpn
  vrf TENANT_B
    address-family ipv4 unicast
      advertise l2vpn evpn
evpn
  vni 10000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 40000 l2
    rd auto
    route-target import auto
    route-target export auto

Leaf_02

version 7.0(3)I6(1)
hostname DC1_LEAF_02
vdc DC1_LEAF_02 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 <removed>  role network-admin
no ip domain-lookup
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:12:41:178:194:165
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:80:86:160:176:48
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

vlan 1,10,100,200,300,400,900,910
fabric forwarding anycast-gateway-mac 0000.dc01.dc01
ip pim rp-address 100.100.100.100 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 100
  vn-segment 10000
vlan 200
  vn-segment 20000
vlan 300
  vn-segment 30000
vlan 400
  vn-segment 40000
vlan 900
  vn-segment 90000
vlan 910
  vn-segment 91000

no cdp enable
vrf context TENANT_A
  vni 90000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context TENANT_B
  vni 91000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
  ip route 0.0.0.0/0 10.1.1.254 name MGMT_DEFAULT


interface Vlan1

interface Vlan100
  description TENANT_A VL100
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  description TENANT_A VL200
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan300
  description TENANT_B VL300
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan400
  description TENANT_B VL400
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan900
  description TENANT_A VL900 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip forward

interface Vlan910
  description TENANT_B VL910 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip forward

interface nve1
  no shutdown
  description VTEP_02
  source-interface loopback0
  host-reachability protocol bgp
  member vni 10000
    suppress-arp
    mcast-group 225.1.0.1
  member vni 20000
    suppress-arp
    mcast-group 225.2.0.1
  member vni 30000
    suppress-arp
    mcast-group 225.3.0.1
  member vni 40000
    suppress-arp
    mcast-group 225.4.0.1
  member vni 90000 associate-vrf
  member vni 91000 associate-vrf

interface Ethernet1/1
  description TO SPINE_02
  no switchport
  mtu 9216
  ip address 10.0.0.10/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description TO SPINE_01
  no switchport
  mtu 9216
  ip address 10.0.0.6/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  shutdown
  mtu 9216

interface Ethernet1/4
  description TO HOST6 - VL100
  switchport access vlan 100
  mtu 9216

interface Ethernet1/5
  description TO HOST7 - VL200
  switchport access vlan 200
  mtu 9216

interface Ethernet1/6
  description TO HOST2 - VL300
  switchport access vlan 300
  mtu 9216

interface Ethernet1/7
  description TO HOST13 - VL400
  switchport access vlan 400
  mtu 9216

interface mgmt0
  description MANAGEMENT
  vrf member management
  ip address 10.1.1.213/24

interface loopback0
  description RID - VTEP
  ip address 4.4.4.4/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy run start
cli alias name ll sh ip route
cli alias name p ping
cli alias name tr traceroute
cli alias name q exit
cli alias name lo exit
cli alias name sr sh running-config
cli alias name sibsle show bgp l2vpn evpn summary
cli alias name sible sh bgp l2vpn evpn
cli alias name snn sh nve vni
cli alias name snp sh nve peer
cli alias name slema show l2route evpn mac all
cli alias name slemia show l2route evpn mac-ip all
cli alias name lla sh ip route vrf TENANT_A
cli alias name svi show vxlan interface
cli alias name siasd show ip arp suppression-cache detail
cli alias name snipd sh nve internal platform interface nve1 detail
cli alias name snpd sh nve peer detail
cli alias name llm sh ip mroute
cli alias name sion sh ip ospf neighbors
cli alias name llb sh ip route vrf TENANT_B
cli alias name shmac show system internal l2fwder mac
cli alias name clms clear mac address-table datapath static
cli alias name clmd clear mac address-table datapath dynamic
line console
  exec-timeout 60
line vty
  exec-timeout 60
boot nxos bootflash:/nxos.7.0.3.I6.1.bin 
router ospf 100
  router-id 4.4.4.4
  passive-interface default
router bgp 65500
  router-id 4.4.4.4
  neighbor 1.1.1.1
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 2.2.2.2
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf TENANT_A
    address-family ipv4 unicast
      advertise l2vpn evpn
  vrf TENANT_B
    address-family ipv4 unicast
      advertise l2vpn evpn
evpn
  vni 10000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 40000 l2
    rd auto
    route-target import auto
    route-target export auto

Leaf_03

version 7.0(3)I6(1)
hostname DC1_LEAF_03
vdc DC1_LEAF_03 id 1
  limit-resource vlan minimum 16 maximum 4094
  limit-resource vrf minimum 2 maximum 4096
  limit-resource port-channel minimum 0 maximum 511
  limit-resource u4route-mem minimum 248 maximum 248
  limit-resource u6route-mem minimum 96 maximum 96
  limit-resource m4route-mem minimum 58 maximum 58
  limit-resource m6route-mem minimum 8 maximum 8

feature telnet
nv overlay evpn
feature ospf
feature bgp
feature pim
feature fabric forwarding
feature interface-vlan
feature vn-segment-vlan-based
feature nv overlay

no password strength-check
username admin password 5 <removed> role network-admin
no ip domain-lookup
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:12:41:178:194:165
snmp-server user admin auth md5 <removed> priv <removed> localizedkey engineID 128
:0:0:9:3:0:80:86:160:176:48
rmon event 1 description FATAL(1) owner PMON@FATAL
rmon event 2 description CRITICAL(2) owner PMON@CRITICAL
rmon event 3 description ERROR(3) owner PMON@ERROR
rmon event 4 description WARNING(4) owner PMON@WARNING
rmon event 5 description INFORMATION(5) owner PMON@INFO

vlan 1,100,200,300,400,900,910
fabric forwarding anycast-gateway-mac 0000.dc01.dc01
ip pim rp-address 100.100.100.100 group-list 225.0.0.0/8
ip pim ssm range 232.0.0.0/8
vlan 100
  vn-segment 10000
vlan 200
  vn-segment 20000
vlan 300
  vn-segment 30000
vlan 400
  vn-segment 40000
vlan 900
  vn-segment 90000
vlan 910
  vn-segment 91000

vrf context TENANT_A
  vni 90000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context TENANT_B
  vni 91000
  rd auto
  address-family ipv4 unicast
    route-target both auto
    route-target both auto evpn
vrf context management
  ip route 0.0.0.0/0 10.1.1.254 name MGMT_DEFAULT

interface Vlan1

interface Vlan100
  description TENANT_A VL100
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan200
  description TENANT_A VL200
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip address 172.16.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan300
  description TENANT_B VL300
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.1.254/24
  fabric forwarding mode anycast-gateway

interface Vlan400
  description TENANT_B VL400
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip address 192.168.2.254/24
  fabric forwarding mode anycast-gateway

interface Vlan900
  description TENANT_A VL900 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_A
  ip forward

interface Vlan910
  description TENANT_B VL910 L3VNI
  no shutdown
  mtu 9216
  vrf member TENANT_B
  ip forward

interface nve1
  no shutdown
  description VTEP_03
  source-interface loopback0
  host-reachability protocol bgp
  member vni 10000
    suppress-arp
    mcast-group 225.1.0.1
  member vni 20000
    suppress-arp
    mcast-group 225.2.0.1
  member vni 30000
    suppress-arp
    mcast-group 225.3.0.1
  member vni 40000
    suppress-arp
    mcast-group 225.4.0.1
  member vni 90000 associate-vrf
  member vni 91000 associate-vrf

interface Ethernet1/1
  description TO SPINE_01
  no switchport
  mtu 9216
  ip address 10.0.0.18/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/2
  description TO SPINE_02
  no switchport
  mtu 9216
  ip address 10.0.0.22/30
  ip ospf network point-to-point
  no ip ospf passive-interface
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
  no shutdown

interface Ethernet1/3
  no switchport
  mtu 9216

interface Ethernet1/4
  description TO CE_R07 - OTV LAN
  switchport mode trunk
  switchport trunk allowed vlan 100,200,300,400
  mtu 9216

interface Ethernet1/5
  description TO CE7 - EXT L3
  no switchport
  mtu 9216
  no shutdown

interface Ethernet1/5.90
  description TO CE7 - TENANT A L3VNI EXT L3
  mtu 9216
  encapsulation dot1q 900
  vrf member TENANT_A
  ip address 172.16.3.253/30
  no shutdown

interface Ethernet1/5.91
  description TO CE7 - TENANT B L3VNI EXT L3
  mtu 9216
  encapsulation dot1q 910
  vrf member TENANT_B
  ip address 192.168.3.253/30
  no shutdown

interface mgmt0
  description MANAGEMENT
  vrf member management
  ip address 10.1.1.214/24

interface loopback0
  description RID - VTEP
  ip address 5.5.5.5/32
  ip router ospf 100 area 0.0.0.0
  ip pim sparse-mode
cli alias name wr copy run start
cli alias name ll sh ip route
cli alias name p ping
cli alias name tr traceroute
cli alias name q exit
cli alias name lo exit
cli alias name sr sh running-config
cli alias name sibsle show bgp l2vpn evpn summary
cli alias name sible sh bgp l2vpn evpn
cli alias name snn sh nve vni
cli alias name snp sh nve peer
cli alias name slema show l2route evpn mac all
cli alias name slemia show l2route evpn mac-ip all
cli alias name lla sh ip route vrf TENANT_A
cli alias name svi show vxlan interface
cli alias name siasd show ip arp suppression-cache detail
cli alias name snipd sh nve internal platform interface nve1 detail
cli alias name snpd sh nve peer detail
cli alias name llm sh ip mroute
cli alias name sion sh ip ospf neighbors
cli alias name llb sh ip route vrf TENANT_B
cli alias name shmac show system internal l2fwder mac
cli alias name clms clear mac address-table datapath static
cli alias name clmd clear mac address-table datapath dynamic
line console
  exec-timeout 60
line vty
  exec-timeout 60
boot nxos bootflash:/nxos.7.0.3.I6.1.bin 
router ospf 100
  router-id 5.5.5.5
  passive-interface default
router bgp 65500
  router-id 5.5.5.5
  neighbor 1.1.1.1
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  neighbor 2.2.2.2
    remote-as 65500
    update-source loopback0
    address-family l2vpn evpn
      send-community
      send-community extended
  vrf TENANT_A
    timers bgp 7 21
    address-family ipv4 unicast
      advertise l2vpn evpn
    neighbor 172.16.3.254
      remote-as 65000
      address-family ipv4 unicast
        send-community
        send-community extended
  vrf TENANT_B
    timers bgp 7 21
    address-family ipv4 unicast
      advertise l2vpn evpn
    neighbor 192.168.3.254
      remote-as 65000
      address-family ipv4 unicast
        send-community
        send-community extended
evpn
  vni 10000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 20000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 30000 l2
    rd auto
    route-target import auto
    route-target export auto
  vni 40000 l2
    rd auto
    route-target import auto
    route-target export auto

WAN_Edge (CE_R07)

version 16.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname CE_R07
!
boot-start-marker
boot-end-marker
!
!
vrf definition TENANT_A
 rd 65000:1
 !
 address-family ipv4
 exit-address-family
!
vrf definition TENANT_B
 rd 65000:2
 !
 address-family ipv4
 exit-address-family
!
logging buffered 16000
no logging console
enable secret 5 <removed>
!
no aaa new-model
!
ip vrf MGMT
!
!
!
!
!
!
!
!
!

no ip domain lookup
ip domain name domain.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
!
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn <removed>
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!
username admin secret 5 <removed>
!
redundancy
!
!
!
!
!
!

!
!
!
!
!
!
!
!
!
!
!
!






!
!
interface Loopback0
 description RID
 ip address 70.70.70.70 255.255.255.255
!
interface GigabitEthernet1
 mtu 9216
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
!
interface GigabitEthernet1.3000
 description WAN TO PE_R13 - TENANT A VPLS
 encapsulation dot1Q 3000
 vrf forwarding TENANT_A
 ip address 172.16.3.1 255.255.255.248
!
interface GigabitEthernet1.3001
 description WAN TO PE_R13 - TENANT B VPLS
 encapsulation dot1Q 3001
 vrf forwarding TENANT_B
 ip address 192.168.3.1 255.255.255.248
!
interface GigabitEthernet2
 mtu 9216
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
!
interface GigabitEthernet2.90
 description TO VXLAN EVPN DC1_LEAF_03 - L3VNI TENANT A
 encapsulation dot1Q 900
 vrf forwarding TENANT_A
 ip address 172.16.3.254 255.255.255.252
!
interface GigabitEthernet2.91
 description TO VXLAN EVPN DC1_LEAF_03 - L3VNI TENANT B
 encapsulation dot1Q 910
 vrf forwarding TENANT_B
 ip address 192.168.3.254 255.255.255.252
!
interface GigabitEthernet3
 mtu 9216 
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
!
interface GigabitEthernet4
 description MANAGEMENT
 ip vrf forwarding MGMT
 ip address 10.1.1.227 255.255.255.0
 negotiation auto
 no mop enabled
 no mop sysid
!
router bgp 65000
 bgp router-id 70.70.70.70
 bgp log-neighbor-changes
 !
 address-family ipv4
 exit-address-family
 !
 address-family ipv4 vrf TENANT_A
  neighbor DC1_LEAF3_TENANT_A peer-group
  neighbor DC1_LEAF3_TENANT_A remote-as 65500
  neighbor DC1_LEAF3_TENANT_A description TO DC1 VXLAN TENANT A
  neighbor DC1_LEAF3_TENANT_A timers 7 21
  neighbor DC1_LEAF3_TENANT_A send-community both
  neighbor BRANCH_CE17_TENANT_A peer-group
  neighbor BRANCH_CE17_TENANT_A remote-as 65100
  neighbor BRANCH_CE17_TENANT_A description TO BRANCH CE17 TENANT A
  neighbor BRANCH_CE17_TENANT_A timers 7 21
  neighbor BRANCH_CE17_TENANT_A send-community both
  neighbor 172.16.3.3 peer-group BRANCH_CE17_TENANT_A
  neighbor 172.16.3.3 activate
  neighbor 172.16.3.253 peer-group DC1_LEAF3_TENANT_A
  neighbor 172.16.3.253 activate
 exit-address-family
 !
 address-family ipv4 vrf TENANT_B
  neighbor DC1_LEAF3_TENANT_B peer-group
  neighbor DC1_LEAF3_TENANT_B remote-as 65500
  neighbor DC1_LEAF3_TENANT_B description TO DC1 VXLAN TENANT B
  neighbor DC1_LEAF3_TENANT_B timers 7 21
  neighbor DC1_LEAF3_TENANT_B send-community both
  neighbor BRANCH_CE17_TENANT_B peer-group
  neighbor BRANCH_CE17_TENANT_B remote-as 65100
  neighbor BRANCH_CE17_TENANT_B description TO BRANCH CE17 TENANT B
  neighbor BRANCH_CE17_TENANT_B timers 7 21
  neighbor BRANCH_CE17_TENANT_B send-community both
  neighbor 192.168.3.3 peer-group BRANCH_CE17_TENANT_B
  neighbor 192.168.3.3 activate
  neighbor 192.168.3.253 peer-group DC1_LEAF3_TENANT_B
  neighbor 192.168.3.253 activate
 exit-address-family
!
!
virtual-service csr_mgmt
!
no ip forward-protocol nd
no ip http server
ip http authentication local
no ip http secure-server
!
ip bgp-community new-format
ip tftp source-interface GigabitEthernet4
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.1.1.254 name MGMT_DEFAULT
!
!
!
!
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!
alias exec ll sh ip route
alias exec sibs sh ip bgp summary
alias exec sib sh ip bgp
alias exec cibs clear ip bgp * soft
alias exec sibsa sh ip bgp vpnv4 vrf TENANT_A summary 
alias exec siba sh ip bgp vpnv4 vrf TENANT_A
alias exec lla sh ip route vrf TENANT_A
alias exec pa ping vrf TENANT_A
alias exec sibsb sh ip bgp vpnv4 vrf TENANT_B summary 
alias exec sibb sh ip bgp vpnv4 vrf TENANT_B
alias exec llb sh ip route vrf TENANT_B
alias exec pb ping vrf TENANT_B
!
line con 0
 logging synchronous
 stopbits 1
line vty 0
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
line vty 1
 exec-timeout 60 0
 logging synchronous
 login local
 length 0
 transport input telnet ssh
line vty 2 4
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
!
ntp server pool.ntp.org
!
!
!
!
!
end


Testing Results

As proof of concept, I only tested VXLAN connectivity with hosts in TENANT_A.  The testing protocol used here is for Host4 to simply ping & trace the other hosts (5, 6 and 7) in the DC, and also to the external branch host outside the DC network to verify full reachability.

  • Host4 (DC Source)
    • IP: 172.16.1.1/24
    • MAC: 000c.2977.ea21
    • VLAN 100
    • Location: Leaf1


  • Host6 (DC Destination)
    • IP: 172.16.1.2
    • MAC: 000c.2998.c13c
    • VLAN 100
    • Location: Leaf2
  • Host5 (DC Destination)
    • IP: 172.16.2.1
    • MAC: 000c.29a8.acde
    • VLAN 200
    • Location: Leaf1
  • Host7 (DC Destination)
    • IP: 172.16.2.2
    • MAC: 000c.292d.161b
    • VLAN 200
    • Location: Leaf2
  • Host3 (Branch Destination)
    • IP: 172.16.100.1
    • Location: External WAN
Connectivity test results from Host4 are below
  • Host4 pings to other DC Hosts

  • Host4 ARP cache
    • Default gateway MAC is the Distributed Anycast Gateway

  • Host4 traceroutes to other DC Hosts

  • Host4 ping and trace to branch host



VXLAN EVPN Infrastructure Verification

Peering Information and Verification

Spine underlay IGP peering (OSPF)

DC1_SPINE_01# sh ip ospf neighbors
 OSPF Process ID 100 VRF default
 Total number of neighbors: 3
 Neighbor ID     Pri State            Up Time  Address         Interface
 3.3.3.3           1 FULL/ -          1w0d     10.0.0.2        Eth1/1
 4.4.4.4           1 FULL/ -          1w0d     10.0.0.6        Eth1/2
 5.5.5.5           1 FULL/ -          6d23h    10.0.0.18       Eth1/3

DC1_SPINE_02# sh ip ospf neighbors
 OSPF Process ID 100 VRF default
 Total number of neighbors: 3
 Neighbor ID     Pri State            Up Time  Address         Interface
 4.4.4.4           1 FULL/ -          1w0d     10.0.0.10       Eth1/1
 3.3.3.3           1 FULL/ -          1w0d     10.0.0.14       Eth1/2
 5.5.5.5           1 FULL/ -          6d23h    10.0.0.22       Eth1/3


Leaf underlay IGP peering (OSPF)

DC1_LEAF_01# sh ip ospf neighbors
 OSPF Process ID 100 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 1.1.1.1           1 FULL/ -          1w0d     10.0.0.1        Eth1/1
 2.2.2.2           1 FULL/ -          1w0d     10.0.0.13       Eth1/2

DC1_LEAF_02# sh ip ospf neighbors
 OSPF Process ID 100 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 2.2.2.2           1 FULL/ -          1w0d     10.0.0.9        Eth1/1
 1.1.1.1           1 FULL/ -          1w0d     10.0.0.5        Eth1/2


DC1_LEAF_03# sh ip ospf neighbors
 OSPF Process ID 100 VRF default
 Total number of neighbors: 2
 Neighbor ID     Pri State            Up Time  Address         Interface
 1.1.1.1           1 FULL/ -          6d23h    10.0.0.17       Eth1/1
 2.2.2.2           1 FULL/ -          6d23h    10.0.0.21       Eth1/2


Spine to Leaf iBGP peering for address family L2VPN EVPN

DC1_SPINE_01# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 1.1.1.1, local AS number 65500
BGP table version is 109238, L2VPN EVPN config peers 3, capable peers 3
21 network entries and 21 paths using 4536 bytes of memory
BGP attribute entries [17/2652], BGP AS path entries [3/34]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
3.3.3.3         4 65500   66854   92169   109238    0    0     1w0d 8        
4.4.4.4         4 65500   68134   90511   109238    0    0     1w0d 8        
5.5.5.5         4 65500   54529   96769   109238    0    0    6d23h 5  


DC1_SPINE_02# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 2.2.2.2, local AS number 65500
BGP table version is 108948, L2VPN EVPN config peers 3, capable peers 3
21 network entries and 21 paths using 4536 bytes of memory
BGP attribute entries [17/2652], BGP AS path entries [3/34]
BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
3.3.3.3         4 65500   66855   92166   108948    0    0     1w0d 8        
4.4.4.4         4 65500   68141   90502   108948    0    0     1w0d 8        
5.5.5.5         4 65500   54534   96773   108948    0    0    6d23h 5 


Leaf to Spine iBGP peering for address family L2VPN EVPN

DC1_LEAF_01# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 3.3.3.3, local AS number 65500
BGP table version is 126312, L2VPN EVPN config peers 2, capable peers 2
38 network entries and 51 paths using 7696 bytes of memory
BGP attribute entries [32/4992], BGP AS path entries [3/34]
BGP community entries [0/0], BGP clusterlist entries [4/16]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.1.1         4 65500   39968   21311   126312    0    0     1w0d 13       
2.2.2.2         4 65500   39967   21311   126312    0    0     1w0d 13

DC1_LEAF_02# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 4.4.4.4, local AS number 65500
BGP table version is 128780, L2VPN EVPN config peers 2, capable peers 2
38 network entries and 51 paths using 7696 bytes of memory
BGP attribute entries [32/4992], BGP AS path entries [3/34]
BGP community entries [0/0], BGP clusterlist entries [4/16]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.1.1         4 65500   38877   28637   128780    0    0     1w0d 13       
2.2.2.2         4 65500   38868   28639   128780    0    0     1w0d 13 

DC1_LEAF_03# sh bgp l2vpn evpn summary
BGP summary information for VRF default, address family L2VPN EVPN
BGP router identifier 5.5.5.5, local AS number 65500
BGP table version is 108507, L2VPN EVPN config peers 2, capable peers 2
45 network entries and 61 paths using 8696 bytes of memory
BGP attribute entries [36/5616], BGP AS path entries [3/34]
BGP community entries [0/0], BGP clusterlist entries [4/16]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
1.1.1.1         4 65500   35939   22995   108507    0    0    6d23h 16       
2.2.2.2         4 65500   35939   22995   108507    0    0    6d23h 16


Leaf3 (Border Leaf) BGP peering to external router (CE_R07)

DC1_LEAF_03# sh ip bgp vrf TENANT_A summary
BGP summary information for VRF TENANT_A, address family IPv4 Unicast
BGP router identifier 172.16.2.254, local AS number 65500
BGP table version is 57, IPv4 Unicast config peers 1, capable peers 1
5 network entries and 5 paths using 568 bytes of memory
BGP attribute entries [5/780], BGP AS path entries [1/10]
BGP community entries [0/0], BGP clusterlist entries [4/16]

Neighbor        V    AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
172.16.3.254    4 65000   89956   86926       57    0    0     1w0d 1 


External WAN Edge Router (CE_R07) BGP peering to Border Leaf3

CE_R07#sh ip bgp vpnv4 vrf TENANT_A summary
BGP router identifier 70.70.70.70, local AS number 65000
BGP table version is 79, main routing table version 79
5 network entries using 1280 bytes of memory
5 path entries using 640 bytes of memory
9/4 BGP path/bestpath attribute entries using 2592 bytes of memory
7 BGP AS-PATH entries using 232 bytes of memory
1 BGP extended community entries using 24 bytes of memory
0 BGP route-map cache entries using 0 bytes of memory
0 BGP filter-list cache entries using 0 bytes of memory
BGP using 4768 total bytes of memory
BGP activity 36/23 prefixes, 53/36 paths, scan interval 60 secs

Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd
172.16.3.3      4        65100    5334    5345       79    0    0 10:01:26        1
172.16.3.253    4        65500   86942   89973       79    0    0 1w0d


External WAN Edge Router (CE_R07) Route Table

CE_R07#sh ip route vrf TENANT_A

Routing Table: TENANT_A
Codes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP
       D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area
       N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2
       E1 - OSPF external type 1, E2 - OSPF external type 2
       i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2
       ia - IS-IS inter area, * - candidate default, U - per-user static route
       o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP
       a - application route
       + - replicated route, % - next hop override, p - overrides from PfR

Gateway of last resort is not set

      172.16.0.0/16 is variably subnetted, 9 subnets, 4 masks
B        172.16.1.1/32 [20/0] via 172.16.3.253, 1d10h
B        172.16.1.2/32 [20/0] via 172.16.3.253, 1d10h
B        172.16.2.1/32 [20/0] via 172.16.3.253, 1d10h
B        172.16.2.2/32 [20/0] via 172.16.3.253, 1d10h
C        172.16.3.0/29 is directly connected, GigabitEthernet1.3000
L        172.16.3.1/32 is directly connected, GigabitEthernet1.3000
C        172.16.3.252/30 is directly connected, GigabitEthernet2.90
L        172.16.3.254/32 is directly connected, GigabitEthernet2.90
B        172.16.100.0/24 [20/0] via 172.16.3.3, 10:04:42


Underlay Multicast Information & Verification

PIM RP on Spines

DC1_SPINE_01# sh ip pim rp
PIM RP Status Information for VRF "default"
BSR: Not Operational
Auto-RP disabled
BSR RP Candidate policy: None
BSR RP policy: None
Auto-RP Announce policy: None
Auto-RP Discovery policy: None

Anycast-RP 100.100.100.100 members:
  1.1.1.1*  2.2.2.2 

RP: 100.100.100.100*, (0), uptime: 2w5d, expires: never,
  priority: 255, RP-source: (local), group ranges:
      225.0.0.0/8
DC1_SPINE_02# sh ip pim rp
PIM RP Status Information for VRF "default"
BSR: Not Operational
Auto-RP disabled
BSR RP Candidate policy: None
BSR RP policy: None
Auto-RP Announce policy: None
Auto-RP Discovery policy: None

Anycast-RP 100.100.100.100 members:
  1.1.1.1  2.2.2.2* 

RP: 100.100.100.100*, (0), uptime: 2w5d, expires: never,
  priority: 255, RP-source: (local), group ranges:
      225.0.0.0/8


PIM Neighbors

DC1_SPINE_01# sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.2        Ethernet1/1          1w0d      00:01:39  1        yes     n/a
10.0.0.6        Ethernet1/2          1w0d      00:01:35  1        yes     n/a
10.0.0.18       Ethernet1/3          1w0d      00:01:26  1        yes     n/a

DC1_SPINE_02# sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.10       Ethernet1/1          1w0d      00:01:39  1        yes     n/a
10.0.0.14       Ethernet1/2          1w0d      00:01:31  1        yes     n/a
10.0.0.22       Ethernet1/3          1w0d      00:01:28  1        yes     n/a

DC1_LEAF_01# sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.1        Ethernet1/1          1w0d      00:01:20  1        yes     n/a
10.0.0.13       Ethernet1/2          1w0d      00:01:34  1        yes     n/a

DC1_LEAF_02# sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.9        Ethernet1/1          1w0d      00:01:43  1        yes     n/a
10.0.0.5        Ethernet1/2          1w0d      00:01:44  1        yes     n/a

DC1_LEAF_03# sh ip pim neighbor
PIM Neighbor Status for VRF "default"
Neighbor        Interface            Uptime    Expires   DR       Bidir-  BFD
                                                         Priority Capable State
10.0.0.17       Ethernet1/1          1w0d      00:01:43  1        yes     n/a
10.0.0.21       Ethernet1/2          1w0d      00:01:21  1        yes     n/a


Multicast Ping Verification (Spine_01)
(Used only one VTEP multicast group 225.1.0.1 for testing brevity)

DC1_SPINE_01# ping multicast 225.1.0.1 interface e1/1 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.2: icmp_seq=0 ttl=254 time=3.117 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.2: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded ---

DC1_SPINE_01# ping multicast 225.1.0.1 interface e1/2 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.6: icmp_seq=0 ttl=254 time=3.07 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.6: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded ---

DC1_SPINE_01# ping multicast 225.1.0.1 interface e1/3 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.18: icmp_seq=0 ttl=254 time=2.665 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.18: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded --


Multicast ping verification (Spine2)
(Used only one VTEP multicast group 225.1.0.1 for testing brevity)


DC1_SPINE_02# ping multicast 225.1.0.1 interface e1/1 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.10: icmp_seq=0 ttl=254 time=3.121 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.10: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded ---

DC1_SPINE_02# ping multicast 225.1.0.1 interface e1/2 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.14: icmp_seq=0 ttl=254 time=2.806 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.14: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded ---

DC1_SPINE_02# ping multicast 225.1.0.1 interface e1/3 count 1
PING 225.1.0.1 (225.1.0.1): 56 data bytes
64 bytes from 10.0.0.22: icmp_seq=0 ttl=254 time=3.082 ms

--- 225.1.0.1 ping multicast statistics ---
1 packets transmitted,
From member 10.0.0.22: 1 packet received, 0.00% packet loss
--- in total, 1 group member responded ---


VXLAN Overlay Verification

VLAN to VN-Segment (VNI)

DC1_LEAF_01# sh vxlan
Vlan           VN-Segment
====           ==========
100            10000
200            20000
300            30000
400            40000
900            90000
910            91000
DC1_LEAF_02# sh vxlan
Vlan           VN-Segment
====           ==========
100            10000
200            20000
300            30000
400            40000
900            90000
910            91000
DC1_LEAF_03# sh vxlan
Vlan           VN-Segment
====           ==========
100            10000
200            20000
300            30000
400            40000
900            90000
910            91000


VTEP peering between Leaf Switches

DC1_LEAF_01# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac      
--------- ---------------  ----- --------- -------- -----------------
nve1      4.4.4.4          Up    CP        1w0d     000c.2984.640e  
nve1      5.5.5.5          Up    CP        6d23h    000c.29b7.b1d3

DC1_LEAF_02# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac      
--------- ---------------  ----- --------- -------- -----------------
nve1      3.3.3.3          Up    CP        1w0d     000c.2910.ac2a  
nve1      5.5.5.5          Up    CP        6d23h    000c.29b7.b1d3

DC1_LEAF_03# sh nve peers
Interface Peer-IP          State LearnType Uptime   Router-Mac      
--------- ---------------  ----- --------- -------- -----------------
nve1      3.3.3.3          Up    CP        6d23h    000c.2910.ac2a  
nve1      4.4.4.4          Up    CP        6d23h    000c.2984.640e


Displays associated VNI’s to multicast groups to bridge domains

DC1_LEAF_01# sh nve vni
Codes: CP - Control Plane        DP - Data Plane         
       UC - Unconfigured         SA - Suppress ARP
      
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      10000    225.1.0.1         Up    CP   L2 [100]           SA  
nve1      20000    225.2.0.1         Up    CP   L2 [200]           SA  
nve1      30000    225.3.0.1         Up    CP   L2 [300]           SA  
nve1      40000    225.4.0.1         Up    CP   L2 [400]           SA  
nve1      90000    n/a               Up    CP   L3 [TENANT_A]          
nve1      91000    n/a               Up    CP   L3 [TENANT_B] 

DC1_LEAF_02# sh nve vni
Codes: CP - Control Plane        DP - Data Plane         
       UC - Unconfigured         SA - Suppress ARP
      
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      10000    225.1.0.1         Up    CP   L2 [100]           SA  
nve1      20000    225.2.0.1         Up    CP   L2 [200]           SA  
nve1      30000    225.3.0.1         Up    CP   L2 [300]           SA  
nve1      40000    225.4.0.1         Up    CP   L2 [400]           SA  
nve1      90000    n/a               Up    CP   L3 [TENANT_A]          
nve1      91000    n/a               Up    CP   L3 [TENANT_B]

DC1_LEAF_03# sh nve vni
Codes: CP - Control Plane        DP - Data Plane         
       UC - Unconfigured         SA - Suppress ARP
      
Interface VNI      Multicast-group   State Mode Type [BD/VRF]      Flags
--------- -------- ----------------- ----- ---- ------------------ -----
nve1      10000    225.1.0.1         Up    CP   L2 [100]           SA  
nve1      20000    225.2.0.1         Up    CP   L2 [200]           SA  
nve1      30000    225.3.0.1         Up    CP   L2 [300]           SA  
nve1      40000    225.4.0.1         Up    CP   L2 [400]           SA  
nve1      90000    n/a               Up    CP   L3 [TENANT_A]          
nve1      91000    n/a               Up    CP   L3 [TENANT_B]



BGP EVPN Control-Plane Verification

Show IP Route

The route table on Leaf1 for TENANT_A shows a nice summary of how VXLAN is routing the traffic.  The output here looks like your typical route table but remote hosts learned via BGP control-plane has additional information related to VXLAN EVPN, such as Segment/VNI ID and VTEP tunnel endpoint ID’s.  Also, locally attached hosts are learned via the NX-OS Host Mobility Manager (HMM), a software process that’s responsible for host learning and cleanup as hosts move within the fabric.

DC1_LEAF_01# sh ip route vrf TENANT_A
IP Route Table for VRF "TENANT_A"
'*' denotes best ucast next-hop
'**' denotes best mcast next-hop
'[x/y]' denotes [preference/metric]
'%<string>' in via output denotes VRF <string>

172.16.1.0/24, ubest/mbest: 1/0, attached
*via 172.16.1.254, Vlan100, [0/0], 08:01:03, direct
172.16.1.1/32, ubest/mbest: 1/0, attached
*via 172.16.1.1, Vlan100, [190/0], 08:01:03, hmm
172.16.1.2/32, ubest/mbest: 1/0
*via 4.4.4.4%default, [200/0], 08:01:03, bgp-65500, internal, tag 65500 (evpn) segid: 90000 tunnelid: 0x4040404 encap: VXLAN

172.16.1.254/32, ubest/mbest: 1/0, attached
*via 172.16.1.254, Vlan100, [0/0], 08:01:03, local
172.16.2.0/24, ubest/mbest: 1/0, attached
    *via 172.16.2.254, Vlan200, [0/0], 08:01:03, direct
172.16.2.1/32, ubest/mbest: 1/0, attached
*via 172.16.2.1, Vlan200, [190/0], 08:01:03, hmm
172.16.2.2/32, ubest/mbest: 1/0
*via 4.4.4.4%default, [200/0], 08:01:03, bgp-65500, internal, tag 65500 (evpn) segid: 90000 tunnelid: 0x4040404 encap: VXLAN

172.16.2.254/32, ubest/mbest: 1/0, attached
*via 172.16.2.254, Vlan200, [0/0], 08:01:03, local
172.16.100.0/24, ubest/mbest: 1/0
*via 5.5.5.5%default, [200/0], 07:37:12, bgp-65500, internal, tag 65000 (evpn) segid: 90000 tunnelid: 0x5050505 encap: VXLAN


Show BGP L2VPN EVPN

A deeper look at the BGP L2VPN EVPN table will reveal more, such as MAC to IP mapping, what VNI they are associated to and VTEP next-hop address.  Below is a quick analysis from Leaf1’s perspective of the routes in each VNI and what it means.

  • Route Distinguisher (RD) 3.3.3.3:32867 is attached to  Leaf1’s VNI 10000/VLAN 100
    • MAC/IP route type 2 for MAC 000c.2977.ea21, host IP 172.16.1.1 is a local route
    • MAC/IP route type 2 for MAC 000c.2998.c13c, host IP 172.16.1.2 is reachable via 4.4.4.4 (Leaf2)
  • Route Distinguisher (RD) 3.3.3.3:32967 is attached to Leaf1’s VNI 20000/VLAN 200
    • MAC/IP route type 2 for MAC 000c.292d.161b, host IP 172.16.2.2 is reachable  via 4.4.4.4 (Leaf2)
    • MAC/IP route type 2 for MAC 000c.29a8.acde, host IP 172.16.2.1 is a local route
  • Route Distinguisher (RD) 3.3.3.3:3 is attached to Leaf1’s VNI 90000/VLAN 900 or the L3 VNI.
    • MAC/IP route type 2 for MAC 000c.292d.161b, host IP 172.16.2.2 is reachable via 4.4.4.4 (Leaf2)
    • MAC/IP route type 2 for MAC 000c.2998.c13c, host IP 172.16.1.2 is reachable via 4.4.4.4 (Leaf2)
    • IP subnet route type 5 for prefix 172.16.100.0/24 is reachable via 5.5.5.5 (Leaf3)

DC1_LEAF_01# sh bgp l2vpn evpn
BGP routing table information for VRF default, address family L2VPN EVPN
BGP table version is 167764, local router ID is 3.3.3.3
Status: s-suppressed, x-deleted, S-stale, d-dampened, h-history, *-valid, >-best
Path type: i-internal, e-external, c-confed, l-local, a-aggregate, r-redist, I-injected
Origin codes: i - IGP, e - EGP, ? - incomplete, | - multipath, & - backup

   Network            Next Hop            Metric     LocPrf     Weight Path
Route Distinguisher: 3.3.3.3:32867    (L2VNI 10000)
*>l[2]:[0]:[0]:[48]:[000c.2977.ea21]:[0]:[0.0.0.0]/216
                      3.3.3.3                           100      32768 i
*>i[2]:[0]:[0]:[48]:[000c.2998.c13c]:[0]:[0.0.0.0]/216
                      4.4.4.4                           100          0 i
x i                   4.4.4.4                           100          0 i
*>l[2]:[0]:[0]:[48]:[000c.2977.ea21]:[32]:[172.16.1.1]/272
                      3.3.3.3                           100      32768 i
*>i[2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/272
                      4.4.4.4                           100          0 i


Route Distinguisher: 3.3.3.3:32967    (L2VNI 20000)
*>i[2]:[0]:[0]:[48]:[000c.292d.161b]:[0]:[0.0.0.0]/216
                      4.4.4.4                           100          0 i
*>l[2]:[0]:[0]:[48]:[000c.29a8.acde]:[0]:[0.0.0.0]/216
                      3.3.3.3                           100      32768 i
*>i[2]:[0]:[0]:[48]:[000c.292d.161b]:[32]:[172.16.2.2]/272
                      4.4.4.4                           100          0 i
*>l[2]:[0]:[0]:[48]:[000c.29a8.acde]:[32]:[172.16.2.1]/272
                      3.3.3.3                           100      32768 i


..snip..


Route Distinguisher: 3.3.3.3:3    (L3VNI 90000)
*>i[2]:[0]:[0]:[48]:[000c.292d.161b]:[32]:[172.16.2.2]/272
                      4.4.4.4                           100          0 i
*>i[2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/272
                      4.4.4.4                           100          0 i
*>i[5]:[0]:[0]:[24]:[172.16.100.0]:[0.0.0.0]/224
                      5.5.5.5                           100          0 65000 65100 i


To explain how the information in the BGP L2VPN EVPN table is used, I’ll use some traffic scenarios as examples.  We will assume the source host is directly connected off Leaf1.

  • Host 172.16.1.1 (VLAN100, VNI10000) → Host 172.16.1.2 (VLAN100, VNI10000) will use the BGP entry in the L2VNI 10000 (RD 3.3.3.3:32867) since the destination host is in the same VLAN/VNI.  It will be reachable via 4.4.4.4 (Leaf2) and bridged to destination host.
  • Host 172.16.2.1 (VLAN200, VNI20000) → Host 172.16.2.2 (VLAN200, VNI20000) will use the BGP entry in the L2VNI 20000 (RD 3.3.3.3:32967) since the destination host is in the same VLAN/VNI.  It will be reachable via 4.4.4.4 (Leaf2) and bridged to destination host.
  • Host 172.16.1.1 (VLAN100, VNI10000) → Host 172.16.2.2 (VLAN200, VNI20000) will use the BGP entry in the L3VNI 90000 (RD 3.3.3.3:3). It will route using the L3VNI first to 4.4.4.4 (Leaf2) then bridged to the destination host in VLAN200, VNI20000.
  • Host 172.16.1.1 (VLAN100, VNI10000) → Host 172.16.100.1 (External Network) will use the BGP entry in the L3VNI 90000 (RD 3.3.3.3:3).  It will route using the L3VNI first to 5.5.5.5 (Leaf3) then handed off to the WAN edge router.


Show BGP L2VPN EVPN <IP Address> | MAC Address

Displaying the BGP L2VPN EVPN on a per-route/mac entry can offer a lot more information for troubleshooting purposes.  However reading it can be very cryptic and overwhelming.  It goes to show the wealth of information stored in the BGP control-plane.

Personally, part of the confusion I encountered while trying to read the information was that this single output contained multiple entries for all VNI's.  I was wondering why there was so much repeated information for a single host?  Below, I'll try to attempt to break this down.  Hopefully I got this down correctly.

So the raw EVPN BGP NLRI below from Leaf1 for host 172.16.1.2 contains an entry that was learned from another Leaf with IP  4.4.4.4 with RD 4.4.4.4:32867.  It learned two routes, one from Spine1 with IP 1.1.1.1 and the other from Spine2 with IP 2.2.2.2.  Looking at the RR cluster list is a easy way to tell the source.

BGP chose the route from 1.1.1.1 as the best path and VNI10000 and VNI90000 imported that route because they have the same route target values (RT: 65500:10000  RT: 65500:90000).  Then they slap on their own route distinguishers (RD) to make them unique within each VNI.


DC1_LEAF_01# sh bgp l2vpn evpn 172.16.1.2
BGP routing table information for VRF default, address family L2VPN EVPN
Route Distinguisher: 3.3.3.3:32867    (L2VNI 10000)
BGP routing table entry for [2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/272, version 10523
Paths: (1 available, best #1)
Flags: (0x000212) on xmit-list, is in l2rib/evpn, is not in HW

  Advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop, in rib
             Imported from 4.4.4.4:32867:[2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/144
  AS-Path: NONE, path sourced internal to AS
    4.4.4.4 (metric 81) from 1.1.1.1 (1.1.1.1)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 10000 90000
      Extcommunity:  RT:65500:10000 RT:65500:90000 ENCAP:8 Router MAC:000c.2984.640e
      Originator: 4.4.4.4 Cluster list: 1.1.1.1

  Path-id 1 not advertised to any peer

Route Distinguisher: 4.4.4.4:32867
BGP routing table entry for [2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/272, version 10518
Paths: (2 available, best #1)
Flags: (0x000202) on xmit-list, is not in l2rib/evpn, is not in HW, is locked

  Advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop
  AS-Path: NONE, path sourced internal to AS
    4.4.4.4 (metric 81) from 1.1.1.1 (1.1.1.1)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 10000 90000
      Extcommunity:  RT:65500:10000 RT:65500:90000 ENCAP:8 Router MAC:000c.2984.640e
      Originator: 4.4.4.4 Cluster list: 1.1.1.1

  Path type: internal, path is valid, not best reason: Neighbor Address, no labeled nexthop
  AS-Path: NONE, path sourced internal to AS
    4.4.4.4 (metric 81) from 2.2.2.2 (2.2.2.2)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 10000 90000
      Extcommunity:  RT:65500:10000 RT:65500:90000 ENCAP:8 Router MAC:000c.2984.640e
      Originator: 4.4.4.4 Cluster list: 2.2.2.2

  Path-id 1 not advertised to any peer

Route Distinguisher: 3.3.3.3:3    (L3VNI 90000)
BGP routing table entry for [2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/272, version 10524
Paths: (1 available, best #1)
Flags: (0x000202) on xmit-list, is not in l2rib/evpn, is not in HW

  Advertised path-id 1
  Path type: internal, path is valid, is best path, no labeled nexthop
             Imported from 4.4.4.4:32867:[2]:[0]:[0]:[48]:[000c.2998.c13c]:[32]:[172.16.1.2]/144
  AS-Path: NONE, path sourced internal to AS
    4.4.4.4 (metric 81) from 1.1.1.1 (1.1.1.1)
      Origin IGP, MED not set, localpref 100, weight 0
      Received label 10000 90000
      Extcommunity:  RT:65500:10000 RT:65500:90000 ENCAP:8 Router MAC:000c.2984.640e
      Originator: 4.4.4.4 Cluster list: 1.1.1.1

  Path-id 1 not advertised to any peer


Furthermore, the two slides below shows there are additional information embedded into the NLRI for the different route types.
  • Route Type 2
  • Route Type 5


Show MAC Address-Table

This command show L2 information, similarly to most switches.  The output should be familiar but here it will contain additional information such as the destination VTEP and its address.
  • Local Hosts 
  • Remote Hosts
DC1_LEAF_01# show system internal l2fwder mac
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
G   900    000c.2910.ac2a    static   -          F     F   sup-eth1(R)
*   100    000c.2977.ea21   dynamic   00:04:00   F     F     Eth1/4 
*   100    000c.2998.c13c    static   -          F     F  (0x47000002) nve-peer2 4.4.4.4 
*   200    000c.29a8.acde   dynamic   00:01:08   F     F     Eth1/5 
G   200    000c.2910.ac2a    static   -          F     F   sup-eth1(R)
G   100    000c.2910.ac2a    static   -          F     F   sup-eth1(R)
*   200    000c.292d.161b    static   -          F     F  (0x47000002) nve-peer2 4.4.4.4 
    1           1         -00:00:dc:01:dc:01         -             1


DC1_LEAF_02# show system internal l2fwder mac
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   100    000c.2977.ea21    static   -          F     F  (0x47000002) nve-peer2 3.3.3.3 
*   100    000c.2998.c13c   dynamic   00:00:38   F     F     Eth1/4 
*   200    000c.29a8.acde    static   -          F     F  (0x47000002) nve-peer2 3.3.3.3 
G   900    000c.2984.640e    static   -          F     F   sup-eth1(R)
*   200    000c.292d.161b   dynamic   00:00:14   F     F     Eth1/5 
G   100    000c.2984.640e    static   -          F     F   sup-eth1(R)
G   200    000c.2984.640e    static   -          F     F   sup-eth1(R)
    1           1         -00:00:dc:01:dc:01         -             1

  • Border Leaf3 has no local hosts.
DC1_LEAF_03# show system internal l2fwder mac
Legend:
        * - primary entry, G - Gateway MAC, (R) - Routed MAC, O - Overlay MAC
        age - seconds since last seen,+ - primary entry using vPC Peer-Link,
        (T) - True, (F) - False, C - ControlPlane MAC
   VLAN     MAC Address      Type      age     Secure NTFY Ports
---------+-----------------+--------+---------+------+----+------------------
*   100    000c.2977.ea21    static   -          F     F  (0x47000002) nve-peer2 3.3.3.3 
*   100    000c.2998.c13c    static   -          F     F  (0x47000001) nve-peer1 4.4.4.4 
*   200    000c.29a8.acde    static   -          F     F  (0x47000002) nve-peer2 3.3.3.3 
G   200    000c.29b7.b1d3    static   -          F     F   sup-eth1(R)
*   200    000c.292d.161b    static   -          F     F  (0x47000001) nve-peer1 4.4.4.4 
G   100    000c.29b7.b1d3    static   -          F     F   sup-eth1(R)
*   100    001e.bd7a.d8bc   dynamic   00:04:43   F     F     Eth1/4 
1        1           1         -00:00:dc:01:dc:01         -             1-             1


Note:  Per the Cisco NX-OSv 9000 Guide, the virtual platform has a substitute command to show the mac address-table (see below).





Show L2ROUTE EVPN

The L2ROUTE EVPN command displays the EVPN L2 RIB, similar to the show mac address-table output. Additional information here can help during the troubleshooting process.

DC1_LEAF_01# sh l2route evpn mac all

Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv (AD):Auto-Delete(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (O):Re-Originated

Topology    Mac Address    Prod   Flags         Seq No     Next-Hops     
----------- -------------- ------ ------------- ---------- ----------------
100         000c.2977.ea21 Local  L,            0          Eth1/4        
100         000c.2998.c13c BGP    SplRcv        0          4.4.4.4       
100         001e.bd7a.d8bc BGP    Rcv           0          5.5.5.5        
200         000c.292d.161b BGP    SplRcv        0          4.4.4.4       
200         000c.29a8.acde Local  L,            0          Eth1/5        
200         001e.bd7a.d8bc BGP    Rcv           0          5.5.5.5           
900         000c.2984.640e VXLAN  Rmac          0          4.4.4.4       
900         000c.29b7.b1d3 VXLAN  Rmac          0          5.5.5.5       


DC1_LEAF_01# sh l2route evpn mac-ip all
Flags -(Rmac):Router MAC (Stt):Static (L):Local (R):Remote (V):vPC link
(Dup):Duplicate (Spl):Split (Rcv):Recv(D):Del Pending (S):Stale (C):Clear
(Ps):Peer Sync (Ro):Re-Originated
Topology    Mac Address    Prod   Flags         Seq No     Host IP         Next-Hops     
----------- -------------- ------ ---------- --------------- ---------------
100         000c.2977.ea21 HMM    --            0          172.16.1.1     Local         
100         000c.2998.c13c BGP    --            0          172.16.1.2     4.4.4.4       
200         000c.29a8.acde HMM    --            0          172.16.2.1     Local         
200         000c.292d.161b BGP    --            0          172.16.2.2     4.4.4.4


Conclusion

So here you have it folks.  Hopefully I got this all right but if not let me know in the comments. I had fun and learned a lot during this process and I wanted to give back to the community.

For my next lab, I'll take this further and build out another Nexus 9K DC and use Cisco OTV to stitch them together.  This will give me an opportunity to learn some Datacenter Internconnect (DCI) technologies in the process.


References

Here's a collection of some really good Nexus 9K VXLAN EVPN configuration documentation. These articles, documents and presentations were instrumental during the study process and would like to personally thank everyone for sharing their experience.


VXLAN EVPN Tutorials

INE Introduction to VXLAN EVPN

Cisco: Deploy a VXLAN Network with an MP-BGP EVPN Control Plane White Paper

Personally this Cisco Live 2016 (Berlin) presentation was really good and full of useful information. I recommend anyone to watch this.

Cisco Live: BRKDCT-3378 - Building DataCenter networks with VXLAN BGP-EVPN (2016 Berlin)


VXLAN EVPN Configuration

Cisco: Configuration and Verification VXLAN with MP-BGP EVPN Control Plane


Cisco Nexus 9000 Series NX-OS VXLAN Configuration Guide, Release 7.x


VXLAN BGP EVPN Challenge Lab


VXLAN EVPN


VXLAN Fabric using EVPN with Cisco Nexus 9000 Switches


VXLAN MP-BGP EVPN Overview


VxLAN BGP EVPN on Nexus 9000v – Part 1


Lab Setup Caveats

  • ESXi standard vSwitch setup caveat
    • Make sure MTU is set to 9000 for the additional VXLAN overhead.
    • Make sure the following security settings are to accept, otherwise traffic won’t pass over the vSwitch.


  • Cisco NX-OSv 9000 Setup Information
  • Serial port access on Cisco NX-OSv 9000 on VMWare ESXi
    • Note: An important note here is that in order to configure the virtual serial port, you need Enterprise or better licensing.  


    • A standalone ESXi install will allow you to create a remote serial port in the VM, however it simply will not work.  The VMWare logs will show this…

2017-06-29T01:07:33.786Z| vcpu-0| I125: [msg.serial.uri.nolicense.basic] The currently available license does not permit the use of remote serial ports.

2017-06-29T01:07:33.786Z| vcpu-0| I125: [msg.device.startdisconnected] Virtual device 'serial0' will start disconnected.

    • If you only have a standalone version of ESXi, getting the console to work on the NX-OSv was kind of tricky.  I first had to deploy the VM in an ESXi environment with Enterprise licensing first (having all the vnics in an isolated vSwitch for safety) then configure the remote serial port per the Cisco document. Once you have console access to the VM, assign an IP address to the “mgmt0” interface.  Make sure that interface is not in shutdown, add a default route into the “management” vrf context, enable the telnet service (feature telnet) and configure a local username and password.  Save the configuration and power off the VM.  At this point, you can put the mgmt0 interface into a management vSwitch and can then telnet from a management station in the same subnet.  You can then clone the VM for multiple instances and change the management IP as necessary.


1 comment: