MPLS L2VPN (EVPN VPWS)

Ethernet VPN or EVPN is considered the next generation of L2VPNs.  It is far different from the existing L2VPN technology as it no longer uses the concept of pseudowires and MAC flooding & learning in the data plane.  In the EVPN world, a control plane protocol is introduced and MAC learning & advertising is accomplished via MP-BGP.  This technology is an overhaul to L2VPNs, so interestingly enough I had to get my hands on it to fully understand it.

Fortunately, the CSR1000v with IOS XE 03.18 or newer has support for EVPN VPWS, so I decided to give that a try.  The lab was configured with a single homed connection to the CE, so unfortunately testing some of EVPN’s new features, such as all-active redundancy mode was not possible.  I believe this was an IOS XE limitation since I couldn't find any documentation on a multi-homed EVPN setup.  However I did see some articles in regards to IOS XR, so I may try that next.

Lab Environment

Note: These resources were appended to the existing IP/MPLS core network. 

• (2) CSR1000v: Version 03.16.05.S for CE’s
• (2) CSR1000v: Version 16.4.2 for PE’s
• IOS XE 03.18 at a minimum was required for EVPN VPWS support

Diagram & Topology

Two additional PE’s (PE25 and PE26) were added to existing IP/MPLS core network and two additional CE’s (CE23 and CE24) were attached to PE25 and PE26 respectively.  The VPWS was then created between PE25 and PE26 and using the CE devices as end hosts.

• Infrastructure Information
• PE25 and PE26 iBGP to Route Reflector (R11) using address family L2VPN EVPN
• CUSTOMER E
• CE IP Address: 10.1.1.0/24
• VPWS Name: EVPN_VPWS_CUSTOMER_E
• VPN ID (EVI): 30
• CE VLAN: 333
• CE AC Interface: Gig1
• PE AC Interface: Gig2
• PE AC Service Instance: 33
• PE25 VPWS Service Instance ID: 25
• PE26 VPWS Service Instance ID: 26

Technology Overview

Here were some good informational slides I found on the technology.  It will cover common terms and acronyms that are used throughout this post. 

EVPN Summary and Benefits

EVPN Service Requirements

• EVI or VPN ID to define the customer VPN
• Local & Remote AC identifiers (VPWS Service Instance ID or Ethernet Tags)

https://www.cisco.com/c/en/us/td/docs/iosxr/ncs5500/vpn/61x/b-ncs5500-l2vpn-configuration-guide-61x/b-ncs5500-l2vpn-configuration-guide-61x_chapter_01001.html

EVPN Instance (EVI)

An EVPN Instance identifies the VPN in the IP/MPLS network and spans on all participating PE routers.  This is analogous to a VRF used in a L3VPN.

https://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-bgp-multihoming-overview.html

Ethernet Segment ID (ESI)

The Ethernet Segment ID identifies each "leg" into the network.  A single homed CE will have a ESI of 0 but a multi-homed CE will need an unique 10 byte ESI configured.

EVPN BGP Route Type 1, Ethernet Auto-Discovery (EAD)

As stated earlier, the EVPN VPWS solution utilizes the BGP Type 1 EAD (per EVI) route to advertise its reachability and capabilities to other PEs.  A type 1 route is advertised by each PE's endpoint (AC) in the EVI with the associated MPLS label.  The receiving PE then enters reachability information into its L2 RIB.

https://kb.wisc.edu/uwsysnet/page.php?id=56133

EVPN BGP Route Type 1 EAD per EVI Route Format

https://kb.wisc.edu/uwsysnet/page.php?id=56133

VPWS Service Instance ID’s

The VPWS Service Instance ID is another word for the local/remote AC identifier.

https://www.jnpr.net/documentation/en_US/junos/topics/reference/configuration-statement/vpws-service-id-edit-routing-instance-instance-type-protocols-evpn-interface-interface-name.html

BGP will advertise these Service Instance ID in the Type 1 EAD per EVI route.

https://pike.lysator.liu.se/docs/ietf/rfc/82/rfc8214.xml

The Service Instance ID is also known as the Ethernet Tag (per the RFC).  However in the actual configuration, it is referenced as the Service Targets (Confusing I know!).

https://tools.ietf.org/html/rfc8214#section-4

Configurations

Configurations below are from the two new PEs and CEs.  The MPLS core configurations has not changed.

PE_R25

version 16.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname PE_R25
!
boot-start-marker
boot-end-marker
!
!
logging buffered 16000
no logging console
enable secret <removed>
!
no aaa new-model
!
ip vrf MGMT
!
!
!
!
!
!
!
!
!

no ip domain lookup
ip domain name domain.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
mpls label range 2500 2599
mpls label protocol ldp
mpls traffic-eng tunnels
!
!
multilink bundle-name authenticated
l2vpn evpn logging vc-state
!
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn <removed>
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!         
username admin secret <removed>
!
redundancy
!
!
!
!
!
cdp run
!
! 
!
!
!
!
!
!
!
!
!
!
!
!
! 
! 
! 
! 
! 
! 
!
!
interface Loopback0
 description RID
 ip address 25.25.25.25 255.255.255.255
 ip ospf 100 area 0
!
interface GigabitEthernet1
 mtu 9216
 bandwidth 100
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 ip rsvp bandwidth
!
interface GigabitEthernet1.34
 description TO P_R03
 bandwidth 100
 encapsulation dot1Q 34
 ip address 10.1.1.42 255.255.255.252
 ip ospf network point-to-point
 ip ospf 100 area 0
 ip ospf cost 2
 mpls ip
 mpls traffic-eng tunnels
 ip rsvp bandwidth 100
!
interface GigabitEthernet1.35
 description TO P_R04
 bandwidth 100
 encapsulation dot1Q 35
 ip address 10.1.1.46 255.255.255.252
 ip ospf network point-to-point
 ip ospf 100 area 0
 ip ospf cost 4
 mpls ip
 mpls traffic-eng tunnels
 ip rsvp bandwidth 100
!
interface GigabitEthernet2
 description TO CE_R23 - EVPN VPWS
 mtu 9216
 bandwidth 100
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 33 ethernet
  encapsulation dot1q 333
  rewrite ingress tag pop 1 symmetric
 !
!
interface GigabitEthernet3
 no ip address
 shutdown
 negotiation auto
 no mop enabled
 no mop sysid
!
interface GigabitEthernet4
 description MANAGEMENT
 ip vrf forwarding MGMT
 ip address 10.1.1.245 255.255.255.0
 negotiation auto
 no mop enabled
 no mop sysid
!
router ospf 100
 router-id 25.25.25.25
 passive-interface default
 no passive-interface GigabitEthernet1.34
 no passive-interface GigabitEthernet1.35
 mpls traffic-eng router-id Loopback0
 mpls traffic-eng area 0
!
router bgp 2345
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor RR peer-group
 neighbor RR remote-as 2345
 neighbor RR description ROUTE REFLECTOR - R11
 neighbor RR update-source Loopback0
 neighbor RR timers 7 21
 neighbor 11.11.11.11 peer-group RR
 !
 address-family ipv4
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor RR send-community both
  neighbor 11.11.11.11 activate
 exit-address-family
!
!
virtual-service csr_mgmt
!
no ip forward-protocol nd
no ip http server
ip http authentication local
no ip http secure-server
!
ip bgp-community new-format
ip tftp source-interface GigabitEthernet4
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.1.1.254 name MGMT_DEFAULT
!
!
!
mpls ldp router-id Loopback0
l2vpn evpn instance 30 point-to-point
 !
 vpws context EVPN_VPWS_CUSTOMER_E
  service target 26 source 25
  member GigabitEthernet2 service-instance 33
!
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!         
!
line con 0
 logging synchronous
 stopbits 1
line vty 0 4
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
!
ntp server pool.ntp.org
!
!
!
!
!
end

PE_R26

version 16.4
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console auto
!
hostname PE_R26
!
boot-start-marker
boot-end-marker
!
!
logging buffered 16000
no logging console
enable secret <removed>
!
no aaa new-model
!
ip vrf MGMT
!
!
!
!
!
!
!
!
!
no ip domain lookup
ip domain name domain.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
mpls label range 2600 2699
mpls label protocol ldp
mpls traffic-eng tunnels
!
!
multilink bundle-name authenticated
l2vpn evpn logging vc-state
!
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn <removed>
diagnostic bootup level minimal
!
spanning-tree extend system-id
!
!         
username admin secret <removed>
!
redundancy
!
!
!
!
!
!
! 
!
!
!
!
!
!
!
!
!
!
!
!
! 
! 
! 
! 
! 
! 
!
!
interface Loopback0
 description RID
 ip address 26.26.26.26 255.255.255.255
 ip ospf 100 area 0
!
interface GigabitEthernet1
 mtu 9216
 bandwidth 100
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 ip rsvp bandwidth
!
interface GigabitEthernet1.36
 description TO P_R02
 bandwidth 100
 encapsulation dot1Q 36
 ip address 10.1.1.50 255.255.255.252
 ip ospf network point-to-point
 ip ospf 100 area 0
 ip ospf cost 1
 mpls ip
 mpls traffic-eng tunnels
 ip rsvp bandwidth 100
!
interface GigabitEthernet1.37
 description TO P_R04
 bandwidth 100
 encapsulation dot1Q 37
 ip address 10.1.1.54 255.255.255.252
 ip ospf network point-to-point
 ip ospf 100 area 0
 ip ospf cost 4
 mpls ip
 mpls traffic-eng tunnels
 ip rsvp bandwidth 100
!
interface GigabitEthernet2
 description TO CE_R24 - EVPN VPWS
 mtu 9216
 bandwidth 100
 no ip address
 negotiation auto
 no mop enabled
 no mop sysid
 service instance 33 ethernet
  encapsulation dot1q 333
  rewrite ingress tag pop 1 symmetric
 !
!
interface GigabitEthernet3
 no ip address
 shutdown
 negotiation auto
 no mop enabled
 no mop sysid
!
interface GigabitEthernet4
 description MANAGEMENT
 ip vrf forwarding MGMT
 ip address 10.1.1.246 255.255.255.0
 negotiation auto
 no mop enabled
 no mop sysid
!
router ospf 100
 router-id 26.26.26.26
 passive-interface default
 no passive-interface GigabitEthernet1.36
 no passive-interface GigabitEthernet1.37
 mpls traffic-eng router-id Loopback0
 mpls traffic-eng area 0
!
router bgp 2345
 bgp log-neighbor-changes
 no bgp default ipv4-unicast
 neighbor RR peer-group
 neighbor RR remote-as 2345
 neighbor RR description ROUTE REFLECTOR - R11
 neighbor RR update-source Loopback0
 neighbor RR timers 7 21
 neighbor 11.11.11.11 peer-group RR
 !
 address-family ipv4
 exit-address-family
 !
 address-family l2vpn evpn
  neighbor RR send-community both
  neighbor 11.11.11.11 activate
 exit-address-family
!
!
virtual-service csr_mgmt
!
no ip forward-protocol nd
no ip http server
ip http authentication local
no ip http secure-server
!
ip bgp-community new-format
ip tftp source-interface GigabitEthernet4
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.1.1.254 name MGMT_DEFAULT
!
!
!
mpls ldp router-id Loopback0
l2vpn evpn instance 30 point-to-point
 !
 vpws context EVPN_VPWS_CUSTOMER_E
  service target 25 source 26
  member GigabitEthernet2 service-instance 33
!
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!
!
line con 0
 logging synchronous
 stopbits 1
line vty 0 4
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
!
ntp server pool.ntp.org
!
!
!
!
!
end

CE_R23

version 15.5
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CE_R23
!
boot-start-marker
boot-end-marker
!
!
logging buffered 16000
no logging console
enable secret <removed>
!
no aaa new-model
!
ip vrf MGMT
!
!
!
!
!
!
!
!
!
!
!
no ip domain lookup
ip domain name domain.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn <removed>
!
spanning-tree extend system-id
!
username admin secret <removed>
!
redundancy
!
!         
!
!
!
!
! 
!
!
!
!
!
!
!
!
!
!
!
!
! 
! 
! 
! 
! 
! 
!
!
interface GigabitEthernet1
 description TO PE_R25
 mtu 9216
 no ip address
 negotiation auto
!
interface GigabitEthernet1.333
 description TO PE_R25 - EVPN VPWS
 encapsulation dot1Q 333
 ip address 10.1.1.1 255.255.255.0
!
interface GigabitEthernet2
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet3
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet4
 description MANAGEMENT
 ip vrf forwarding MGMT
 ip address 10.1.1.243 255.255.255.0
 negotiation auto
!
!
virtual-service csr_mgmt
!
no ip forward-protocol nd
!
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet4
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.1.1.254 name MGMT_DEFAULT
ip ssh time-out 60
!
!
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!
!
line con 0
 logging synchronous
 stopbits 1
line vty 0 4
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
!
ntp server pool.ntp.org
!
end

CE_R24

version 15.5
no service pad
service tcp-keepalives-in
service tcp-keepalives-out
service timestamps debug datetime localtime show-timezone year
service timestamps log datetime localtime show-timezone year
service password-encryption
no platform punt-keepalive disable-kernel-core
platform console virtual
!
hostname CE_R24
!
boot-start-marker
boot-end-marker
!
!
logging buffered 16000
no logging console
enable secret <removed>
!
no aaa new-model
!
ip vrf MGMT
!
!
!
!
!
!
!
!
!
!
!
no ip domain lookup
ip domain name domain.local
!
!
!
!
!
!
!
!
!
!
subscriber templating
!
multilink bundle-name authenticated
!
!
!
!
!
!
!
!
!
!
!
!
!
license udi pid CSR1000V sn <removed>
!
spanning-tree extend system-id
!
username admin secret <removed>
!
redundancy
!
!         
!
!
!
!
! 
!
!
!
!
!
!
!
!
!
!
!
!
! 
! 
! 
! 
! 
! 
!
!
interface GigabitEthernet1
 description TO PE_R26
 mtu 9216
 no ip address
 negotiation auto
!
interface GigabitEthernet1.333
 description TO PE_R26 - EVPN VPWS
 encapsulation dot1Q 333
 ip address 10.1.1.2 255.255.255.0
!
interface GigabitEthernet2
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet3
 no ip address
 shutdown
 negotiation auto
!
interface GigabitEthernet4
 description MANAGEMENT
 ip vrf forwarding MGMT
 ip address 10.1.1.244 255.255.255.0
 negotiation auto
!
!
virtual-service csr_mgmt
!
no ip forward-protocol nd
!
no ip http server
no ip http secure-server
ip tftp source-interface GigabitEthernet4
ip route vrf MGMT 0.0.0.0 0.0.0.0 10.1.1.254 name MGMT_DEFAULT
ip ssh time-out 60
!
!
!
!
control-plane
!
 !
 !
 !
 !
!
!
!
!
!         
line con 0
 logging synchronous
 stopbits 1
line vty 0 4
 exec-timeout 60 0
 logging synchronous
 login local
 transport input telnet ssh
!
ntp server pool.ntp.org
!
end

EVPN VPWS Specific Configurations

Configuration snippets below were taken only from PE_R25 and CE_R23 for brevity.

Attachment Circuit

The interface facing the CE defines the attachment circuit (AC) using an Ethernet Flow Point (EFP) configuration.  In this example we match VLAN 333 to classify the traffic to the service instance (33).  The rewrite ingress command is told to remove the outermost Dot1q tag (pop 1) as it enters the MPLS network.  The Dot1q header is irrelevant for MPLS switching so its told to remove it as it enters the MPLS domain.  However the "symmetric" keyword ensures the Dot1q header is added back on the egress destination end.

interface GigabitEthernet2  description TO CE_R23 - EVPN VPWS  mtu 9216  bandwidth 100  no ip address  negotiation auto  service instance 33 ethernet   encapsulation dot1q 333   rewrite ingress tag pop 1 symmetric

L2VPN EVPN Instance

This command builds the EVPN instance by defining the EVI and type (point to point), VPWS Service Instance ID’s/Ethernet Tags (service targets) and binding the AC (interface and service instance number).

l2vpn evpn instance 30 point-to-point  !  vpws context EVPN_VPWS_CUSTOMER_E   service target 26 source 25   member GigabitEthernet2 service-instance 33

MP-BGP

An BGP peering is configured to a Route Reflector using AFI 25 (L2VPN) & SAFI 70 (EVPN).

PE25

router bgp 2345  bgp log-neighbor-changes  no bgp default ipv4-unicast  neighbor RR peer-group  neighbor RR remote-as 2345  neighbor RR description ROUTE REFLECTOR - R11  neighbor RR update-source Loopback0  neighbor RR timers 7 21  neighbor 11.11.11.11 peer-group RR  !  address-family ipv4  exit-address-family  !  address-family l2vpn evpn   neighbor RR send-community both   neighbor 11.11.11.11 activate  exit-address-family

Route Reflector (R11)

router bgp 2345  bgp log-neighbor-changes  no bgp default ipv4-unicast  neighbor PE_R25 peer-group  neighbor PE_R25 remote-as 2345  neighbor PE_R25 description TO PE R25  neighbor PE_R25 update-source Loopback0  neighbor PE_R25 timers 7 21  neighbor PE_R26 peer-group  neighbor PE_R26 remote-as 2345  neighbor PE_R26 description TO PE R26  neighbor PE_R26 update-source Loopback0  neighbor PE_R26 timers 7 21  neighbor 25.25.25.25 peer-group PE_R25  neighbor 26.26.26.26 peer-group PE_R26  !  address-family ipv4  exit-address-family  !  address-family l2vpn evpn   neighbor PE_R25 send-community both   neighbor PE_R25 route-reflector-client   neighbor PE_R26 send-community both   neighbor PE_R26 route-reflector-client   neighbor 25.25.25.25 activate   neighbor 26.26.26.26 activate  exit-address-family

CE Interface

The CE's AC interfaces uses a customer VLAN of 333.

interface GigabitEthernet1.333  description TO PE_R25 - EVPN VPWS  encapsulation dot1Q 333  ip address 10.1.1.1 255.255.255.0

Testing Results

The testing results were simple pings between CE’s to verify full L3 reachability.  ARP table lookup  verifed L2 reachability.

CE23 to CE24

CE_R23#ping 10.1.1.2 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.1.1.2, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 6/11/24 ms

CE23’s ARP Table

CE_R23#sh arp Protocol  Address          Age (min)  Hardware Addr   Type   Interface Internet  10.1.1.1                -   000c.2936.16b8  ARPA   GigabitEthernet1.333 Internet  10.1.1.2              229   000c.290f.bf15  ARPA   GigabitEthernet1.333

CE24 to CE23

CE_R24#ping 10.1.1.1 Type escape sequence to abort. Sending 5, 100-byte ICMP Echos to 10.1.1.1, timeout is 2 seconds: !!!!! Success rate is 100 percent (5/5), round-trip min/avg/max = 7/10/23 ms

CE24’s ARP Table

CE_R24#sh arp Protocol  Address          Age (min)  Hardware Addr   Type   Interface Internet  10.1.1.1              229   000c.2936.16b8  ARPA   GigabitEthernet1.333 Internet  10.1.1.2                -   000c.290f.bf15  ARPA   GigabitEthernet1.333

Verification Tasks

Below is a list of commands used to verify the operational status of the L2VPN.

• L2VPN EVPN Summary
• L2VPN EVPN EVI Detail
• L2VPN EVPN VC All
• L2VPN EVPN VC All Detail
• L2VPN EVPN RIB EAD EVI
• BGP L2VPN EVPN Summary
• BGP L2VPN EVPN
• BGP L2VPN EVPN Detail

L2VPN EVPN Summary

The EVPN summary command displays a summary of the virtual circuit and its operational state.

PE_R25#sh l2vpn evpn summary L2VPN EVPN VPWS:   EVIs (point-to-point): 1   Total VCs: 1     1 up, 0 down, 0 admin-down, 0 hot-standby, 0 other   Total EVPN EAD routes: 2     1 local, 1 remote   Total EVI EAD routes: 2     1 local, 1 remote (1 in-use)   BGP: ASN 2345, address-family l2vpn evpn configured   Router ID: 25.25.25.25

L2VPN EVPN EVI Detail

This command displays EVPN instance details, such as RD and RT’s.

PE_R25#sh l2vpn evpn evi 30 detail EVPN instance: 30 (point-to-point)   RD: 25.25.25.25:30 (auto)   Import-RTs: 2345:30   Export-RTs: 2345:30   Total VCs: 1     1 up, 0 down, 0 admin-down, 0 hot-standby, 0 other   Total EAD routes: 2     1 local, 1 remote (1 in-use)

L2VPN EVPN VC All

This command displays EVPN virtual circuit information, such as EVI, Service Instance ID’s and AC interface/Service Instance.

PE_R25#sh l2vpn evpn vc all  EVPN ID Source   Target   Type   Name/Interface                   Status ------- -------- -------- ------ -------------------------------- ---------- 30      25       26       p2p    EVPN_VPWS_CUSTOMER_E             up                                  Gi2:33                           up

L2VPN EVPN VC All Detail

This command displays EVPN virtual circuit in more detail, including MPLS VPN labels.

PE_R25#sh l2vpn evpn vc all detail EVPN name: EVPN_VPWS_CUSTOMER_E, state: up, type: point-to-point   EVPN ID: 30   VPWS Service Instance ID: Source 25, Target 26   Labels: Local 2500, Remote 2600   Next Hop Address: 26.26.26.26   Associated member interface Gi2 up, Gi2:33 status is up   Dataplane:     SSM segment/switch IDs: 4105/4096 (used), PWID: 1   Rx Counters     5 input transit packets, 666 bytes     0 drops   Tx Counters     5 output transit packets, 686 bytes     0 drops   5 VC FSM state transitions, Last 5 shown     Prov: Idle -> Prov, Sat Oct 14 20:51:08.743 (11:00:11 ago)     AdmUp: Prov -> LocWait, Sat Oct 14 20:51:08.772 (11:00:11 ago)     LocUp: LocWait -> RemWait, Sat Oct 14 20:51:15.266 (11:00:04 ago)     RemUp: RemWait -> Act, Sat Oct 14 20:51:43.274 (10:59:36 ago)     DpUp: Act -> Est, Sat Oct 14 20:52:11.339 (10:59:08 ago)

L2VPN EVPN RIB EAD EVI

This command displays the L2 Routing Information Base (L2 RIB) learned from the BGP EVPN EAD route for each PE.  This output shows the next-hop for this point to point EVC and the local/remote MPLS label.  The output also shows the Service Instance ID's as Ethernet Tags.

PE_R25#sh l2vpn evpn rib ead evi +- Origin of entry                                (i=iBGP/e=eBGP/L=Local) | +- Provisioned                                  (Yes/No)? | | +- Best path                                  (Yes/No)? | | | v v v O P B  EVI    Ethernet Segment Id    Eth Tag     Next Hop      Label  -+-+-+-----+------------------------+--------+---------------+-------- i Y Y 30    0000.0000.0000.0000.0000 26       26.26.26.26     2600    L - - 30    0000.0000.0000.0000.0000 25                       2500

PE_R26#sh l2vpn evpn rib ead evi +- Origin of entry                                (i=iBGP/e=eBGP/L=Local) | +- Provisioned                                  (Yes/No)? | | +- Best path                                  (Yes/No)? | | | v v v O P B  EVI    Ethernet Segment Id    Eth Tag     Next Hop      Label   -+-+-+-----+------------------------+--------+---------------+-------- i Y Y 30    0000.0000.0000.0000.0000 25       25.25.25.25     2500     L - - 30    0000.0000.0000.0000.0000 26                       2600

BGP L2VPN EVPN Summary

This command displays BGP peering and prefixes received for the L2VPN/EVPN address family.  Here we see that one route was learned from neighbor 11.11.11.11 (Route Reflector).

PE_R25#sh ip bgp l2vpn evpn summary BGP router identifier 25.25.25.25, local AS number 2345 BGP table version is 4, main routing table version 4 2 network entries using 688 bytes of memory 2 path entries using 400 bytes of memory 2/2 BGP path/bestpath attribute entries using 544 bytes of memory 1 BGP rrinfo entries using 40 bytes of memory 1 BGP extended community entries using 24 bytes of memory 0 BGP route-map cache entries using 0 bytes of memory 0 BGP filter-list cache entries using 0 bytes of memory BGP using 1696 total bytes of memory BGP activity 2/0 prefixes, 2/0 paths, scan interval 60 secs Neighbor        V           AS MsgRcvd MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd 11.11.11.11     4         2345    5866    5862        4    0    0 11:01:46        1

BGP L2VPN EVPN

This command displays the BGP table for the address family.  One BGP EAD type 1 route was learned from Route Reflector.

PE_R25#sh ip bgp l2vpn evpn         BGP table version is 4, local router ID is 25.25.25.25 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,               r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,               x best-external, a additional-path, c RIB-compressed,               t secondary path, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found      Network          Next Hop            Metric LocPrf Weight Path Route Distinguisher: 25.25.25.25:30  *>   [1][25.25.25.25:30][00000000000000000000][25]/23                       ::                                 32768 ? Route Distinguisher: 26.26.26.26:30  *>i  [1][26.26.26.26:30][00000000000000000000][26]/23                       26.26.26.26              0    100      0 ?

This command displays the BGP table from Route Reflector’s perspective.  Here we see two BGP EAD type 1 routes that were learned, one from each PE.

RR_R11#sh ip bgp l2vpn evpn BGP table version is 5, local router ID is 11.11.11.11 Status codes: s suppressed, d damped, h history, * valid, > best, i - internal,               r RIB-failure, S Stale, m multipath, b backup-path, f RT-Filter,               x best-external, a additional-path, c RIB-compressed, Origin codes: i - IGP, e - EGP, ? - incomplete RPKI validation codes: V valid, I invalid, N Not found      Network          Next Hop            Metric LocPrf Weight Path Route Distinguisher: 25.25.25.25:30  *>i [1][25.25.25.25:30][00000000000000000000][25][2500]/25                        25.25.25.25              0    100      0 ? Route Distinguisher: 26.26.26.26:30  *>i [1][26.26.26.26:30][00000000000000000000][26][2600]/25                        26.26.26.26              0    100      0 ?

BGP L2VPN EVPN Detail

This command displays the EVPN BGP Type 1 NLRI in more detail.

• BGP Route Type
• RD
• Ethernet Segment ID (ESI)
• VPWS Service Instance ID (or Ethernet Tag)
• Route Target
• VPN Labels

PE_R25#sh ip bgp l2vpn evpn detail Route Distinguisher: 25.25.25.25:30 BGP routing table entry for [1][25.25.25.25:30][00000000000000000000][25]/23, version 2   Paths: (1 available, best #1, table EVPN-BGP-Table)   Advertised to update-groups:      1           Refresh Epoch 1   Local     :: (via default) from 0.0.0.0 (25.25.25.25)       Origin incomplete, localpref 100, weight 32768, valid, sourced, local, best       Rcvd Label: None, Local Label: 2500       Extended Community: RT:2345:30       rx pathid: 0, tx pathid: 0x0 Route Distinguisher: 26.26.26.26:30 BGP routing table entry for [1][26.26.26.26:30][00000000000000000000][26]/23, version 4   Paths: (1 available, best #1, table EVPN-BGP-Table)   Not advertised to any peer   Refresh Epoch 1   Local     26.26.26.26 (metric 7) (via default) from 11.11.11.11 (11.11.11.11)       Origin incomplete, metric 0, localpref 100, valid, internal, best       Rcvd Label: 2600, Local Label: None       Extended Community: RT:2345:30       Originator: 26.26.26.26, Cluster list: 11.11.11.11       rx pathid: 0, tx pathid: 0x0

MPLS Forwarding Trace

This is a look into each device's MPLS switching operation for traffic sent from CE23 to CE24.

For the virtual circuit between PE25 to PE26 (EVI 30), a packet was imposed a VC label of 2600 and forwarded to the next hop of 26.26.26.26.

PE_R25#sh l2vpn evpn rib ead evi +- Origin of entry                                (i=iBGP/e=eBGP/L=Local) | +- Provisioned                                  (Yes/No)? | | +- Best path                                  (Yes/No)? | | | v v v O P B  EVI    Ethernet Segment Id    Eth Tag     Next Hop      Label  -+-+-+-----+------------------------+--------+---------------+-------- i Y Y 30    0000.0000.0000.0000.0000 26       26.26.26.26     2600    L - - 30    0000.0000.0000.0000.0000 25                       2500

To reach destination 26.26.26.26, a packet was imposed an outer LSP label of 330 and forwarded to P3.

PE_R25#sh mpls forwarding-table 26.26.26.26 detail       Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              2535       330        26.26.26.26/32   0             Gi1.34     10.1.1.41           MAC/Encaps=18/22, MRU=9216, Label Stack{330}         000C291EB42A000C297D84BA810000228847 0014A000         No output feature configured

P3 received the packet, swapped label 330 to 230 and forwarded to P2.

P_R03#sh mpls forwarding-table labels 330 detail Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              330        230        26.26.26.26/32   1250          Gi1.14     10.0.0.17           MAC/Encaps=18/22, MRU=9216, Label Stack{230}         000C294A72B4000C291EB42A8100000E8847 000E6000         No output feature configured

P2 received the packet, popped the label and forwarded to PE26.

P_R02#sh mpls forwarding-table labels 230 detail Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              230        Pop Label  26.26.26.26/32   2722100       Gi1.36     10.1.1.50           MAC/Encaps=18/18, MRU=9220, Label Stack{}         000C29432B51000C294A72B4810000248847         No output feature configured

PE26 looked up label of 2600, which identified the VC and forwarded the packet out its local AC, interface Gig2.

PE_R26#sh mpls forwarding-table labels 2600 detail Local      Outgoing   Prefix           Bytes Label   Outgoing   Next Hop    Label      Label      or Tunnel Id     Switched      interface              2600       No Label   l2ckt(1)         1206          Gi2        point2point         MAC/Encaps=0/0, MRU=0, Label Stack{}         No output feature configured

References

https://www.cisco.com/c/en/us/td/docs/routers/asr920/configuration/guide/mpls/mp-l2-vpns-xe-3s-asr920-book/mp-l2-vpns-xe-3s-asr920-book_chapter_01000.pdf

https://tools.ietf.org/html/rfc8214

Generic EVPN Information

http://packetpushers.net/evpn-introduction-next-generation-l2vpn/

http://www.bgphelp.com/2017/06/12/evpn-faq/

http://wiki.kemot-net.com/vpn-evpn-bgp

http://www.xceptional.com/2017/03/29/4200/