EVPN Single-Active Multi-Homing – Load Balancing Traffic Analysis

Here are my traffic analysis notes for the Single-Active lab.  After doing a deep dive on the All-Active traffic, I felt this final step was necessary to get a detailed understanding of both multi-homing redundancy modes.  The test plan was identical to the All-Active traffic analysis and the Wireshark traces were performed on the same backbone links.

Test Plan

• Simulate traffic flow from CE_R27 to CE_R29
• Observe interface statistics to determine traffic routing
• Observe BGP routing and label exchange
• Perform targeted packet captures on PE links
• Examine packet captures for label usage

Test Traffic

Continuous ping from host CE_R27 to CE_R29.

CE_R27#ping 172.16.50.2 rep 2147483647 Type escape sequence to abort. Sending 2147483647, 100-byte ICMP Echos to 172.16.50.2, timeout is 2 seconds: !!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!! .. snip ..

Traffic Observation

CE_R27’s Traffic Distribution

CE_R27’s G1 and G2 Bridge-Domain interface.  The input/output rate were equal and normal.

CE_R27#sh int bdi50 | in rate   Queueing strategy: fifo   30 second input rate 38000 bits/sec, 42 packets/sec   30 second output rate 38000 bits/sec, 42 packets/sec

CE_R27’s G1 Ethernet interface to PE_MXR01.  The primary forwarding interface’s input/output rates were in-line with the BDI interface.

CE_R27#sh int g1 | in rate     Queueing strategy: fifo   30 second input rate 38000 bits/sec, 42 packets/sec   30 second output rate 38000 bits/sec, 42 packets/sec

CE_R27’s G2 Ethernet interface to PE_MXR03.  The standby interface’s has no traffic.

CE_R27#sh int g2 | in rate   Queueing strategy: fifo   30 second input rate 0 bits/sec, 0 packets/sec   30 second output rate 0 bits/sec, 0 packets/sec

PE_MXR01's Traffic Distribution

PE_MXR01’s AC interface to CE_R27 (Gig1).  The active AC interface’s input/output rates were in-line with the CE.

admin@PE_MXR01> show interfaces ge-0/0/2.500 detail |grep pps         Input  packets:                19049                   42 pps      Output packets:                19046                   42 pps

PE_MXR01 MPLS interface to P_R03.  The link towards the MPLS core saw all traffic to P3.

admin@PE_MXR01> show interfaces ge-0/0/1.44 detail |grep pps          Input  packets:              6172520                   42 pps      Output packets:              7677321                   42 pps

PE_MXR01 MPLS interface to P_R01.  The link towards the MPLS core saw no traffic to P1.

admin@PE_MXR01> show interfaces ge-0/0/1.45 detail |grep pps         Input  packets:                    3                    0 pps      Output packets:                    1                    0 pps

PE_MXR02's Traffic Distribution

PE_MXR02’s AC interface to CE_R28.  The destination AC interface’s input/output rates were in-line with the CE.

admin@PE_MXR02> show interfaces ge-0/0/2.500 detail |grep pps                                                                          Input  packets:              6473475                   42 pps      Output packets:              6914532                   42 pps

PE_MXR02’s MPLS interface to P_R03.  The link towards the MPLS core saw all traffic to P3.

admin@PE_MXR02> show interfaces ge-0/0/1.46 detail |grep pps         Input  packets:              6914474                   42 pps      Output packets:             12623504                   42 pps

PE_MXR02’s MPLS interface to P_R04.  The link towards the MPLS core saw no traffic to P4.

admin@PE_MXR02> show interfaces ge-0/0/1.47 detail |grep pps         Input  packets:                    2                    0 pps      Output packets:                    0                    0 pps

PE_MXR03's Traffic Distribution

PE_MXR03’s AC interface to CE_R27 (Gig2).  The redundant AC interface, configured in standby mode, saw no traffic.

admin@PE_MXR03> show interfaces ge-0/0/3.500 detail |grep pps       Input  packets:                    0                    0 pps      Output packets:                    0                    0 pps

PE_MXR03’s MPLS interface to P_R02.  No traffic was forwarded to this PE under normal operating conditions.

admin@PE_MXR03> show interfaces ge-0/0/1.48 detail |grep pps          Input  packets:              7514322                    0 pps      Output packets:               299204                    0 pps

PE_MXR03’s MPLS interface to P_R01.  No traffic was forwarded to this PE under normal operating conditions.

admin@PE_MXR03> show interfaces ge-0/0/1.49 detail |grep pps         Input  packets:                    4                    0 pps      Output packets:                   27                    0 pps

Wireshark Traffic Analysis

Capture 1 – Link between PE_MXR01 and P_R03

Capture 1, Frame 2 was the ICMP request from CE_R27 to CE_R28 as seen between PE_MXR01 and P_R03.  This request was sent to CE_R28 with destination MAC address 00:0c:29:49:aa:8c.  

Based on the Type 2 MAC route lookup, the zero ESI indicated that the destination host was single-homed and a Type 1 aliasing route lookup wasn’t necessary.  As seen from the Wireshark capture, its label usage (bottom label 299776, top label 328) was consistent with the route lookup.

admin@PE_MXR01> show route table bgp.evpn.0 extensive               bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) .. snip .. 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP (1 entry, 0 announced)         *BGP    Preference: 170/-101                 Route Distinguisher: 112.112.112.112:50                 Next hop type: Indirect, Next hop index: 0                 Address: 0xb7a3a90                 Next-hop reference count: 8                 Source: 112.112.112.112                 Protocol next hop: 112.112.112.112                 Indirect next hop: 0x2 no-forward INH Session ID: 0x0                 State: <Active Int Ext>                 Local AS:  2345 Peer AS:  2345                 Age: 17:57      Metric2: 1                 Validation State: unverified                 Task: BGP_2345.112.112.112.112                 AS path: I                 Communities: target:2345:50                 Import Accepted                 Route Label: 299776                 ESI: 00:00:00:00:00:00:00:00:00:00                 Localpref: 100                 Router ID: 112.112.112.112                 Secondary Tables: EVPN_CUSTOMER_G_ELAN_500.evpn.0                 Indirect next hops: 1                         Protocol next hop: 112.112.112.112 Metric: 1                         Indirect next hop: 0x2 no-forward INH Session ID: 0x0                         Indirect path forwarding next hops: 1                                 Next hop type: Router                                 Next hop: 10.1.1.81 via ge-0/0/1.44                                 Session Id: 0x0                         112.112.112.112/32 Originating RIB: inet.3                           Metric: 1                       Node path count: 1                           Forwarding nexthops: 1                                 Nexthop: 10.1.1.81 via ge-0/0/1.44

admin@PE_MXR01> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both .. snip .. 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:17:37, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328

Capture 2 – Link between P_R03 and PE_MXR02

Capture 2, Frame 1 was the ICMP request from CE_R27 to CE_R28 as seen from P_R03 and PE_MXR02.  This request was sent to CE_R28's MAC address of 00:0c:29:49:aa:8c.  

As P_R03 forwarded the packet to PE_MXR02, it popped the top label of 328.  The VPN label of 299776 was looked up, associated to the EVPN instance (via Ingress-MAC since the Type 2 route’s label was used) and packet delivered to CE_R28.  At this point everything looked normal.

admin@PE_MXR02> show route table mpls.0 label 299776 mpls.0: 52 destinations, 52 routes (52 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 299776             *[EVPN/7] 00:24:26, routing-instance EVPN_CUSTOMER_G_ELAN_500, route-type Ingress-MAC, vlan-id 500                       to table EVPN_CUSTOMER_G_ELAN_500.evpn-mac.0

Capture 2, Frame 2 was the ICMP reply from CE_R28 to CE_R27 as seen from PE_MXR02 to P_R03.  According to Wireshark’s analysis (arrows), this was Frame 1’s corresponding reply.  

PE_MXR02 initially performed a Type 2 MAC route lookup for the destination MAC address and discovered this host was multi-homed (from non-zero ESI).  A Type 1 EAD per EVI route lookup was subsequently performed and its aliasing label was then used to construct the label stack of 306800 and 329.

In single-active mode, a Type 1 EAD per EVI route was only advertised from the primary forwarding PE.  In this case, this route was only received from 111.111.111.111 (PE_MXR01), therefore PE_MXR02 would forward any traffic to 111.111.111.111.

admin@PE_MXR02> show route table bgp.evpn.0 extensive    bgp.evpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) .. snip .. 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI (1 entry, 0 announced)         *BGP    Preference: 170/-101                 Route Distinguisher: 111.111.111.111:50                 Next hop type: Indirect, Next hop index: 0                 Address: 0xb7a5950                 Next-hop reference count: 10                 Source: 111.111.111.111                 Protocol next hop: 111.111.111.111                 Indirect next hop: 0x2 no-forward INH Session ID: 0x0                 State: <Active Int Ext>                 Local AS:  2345 Peer AS:  2345                 Age: 25:22      Metric2: 1                 Validation State: unverified                 Task: BGP_2345.111.111.111.111                 AS path: I                 Communities: target:2345:50                 Import Accepted                 Route Label: 306800                 Localpref: 100                 Router ID: 111.111.111.111                 Secondary Tables: EVPN_CUSTOMER_G_ELAN_500.evpn.0                 Indirect next hops: 1                         Protocol next hop: 111.111.111.111 Metric: 1                         Indirect next hop: 0x2 no-forward INH Session ID: 0x0                         Indirect path forwarding next hops: 1                                 Next hop type: Router                                 Next hop: 10.1.1.89 via ge-0/0/1.46                                 Session Id: 0x0                         111.111.111.111/32 Originating RIB: inet.3                           Metric: 1                       Node path count: 1                           Forwarding nexthops: 1                                 Nexthop: 10.1.1.89 via ge-0/0/1.46 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 0 announced)         *BGP    Preference: 170/-101                 Route Distinguisher: 111.111.111.111:50                 Next hop type: Indirect, Next hop index: 0                 Address: 0xb7a5950                 Next-hop reference count: 10                 Source: 111.111.111.111                 Protocol next hop: 111.111.111.111                 Indirect next hop: 0x2 no-forward INH Session ID: 0x0                 State: <Active Int Ext>                 Local AS:  2345 Peer AS:  2345                 Age: 24:47      Metric2: 1                 Validation State: unverified                 Task: BGP_2345.111.111.111.111                 AS path: I                 Communities: target:2345:50                 Import Accepted                 Route Label: 306800                 ESI: 00:11:22:33:44:55:66:77:88:99                 Localpref: 100                 Router ID: 111.111.111.111                 Secondary Tables: EVPN_CUSTOMER_G_ELAN_500.evpn.0                 Indirect next hops: 1                         Protocol next hop: 111.111.111.111 Metric: 1                         Indirect next hop: 0x2 no-forward INH Session ID: 0x0                         Indirect path forwarding next hops: 1                                 Next hop type: Router                                 Next hop: 10.1.1.89 via ge-0/0/1.46                                 Session Id: 0x0                         111.111.111.111/32 Originating RIB: inet.3                           Metric: 1                       Node path count: 1                           Forwarding nexthops: 1                                 Nexthop: 10.1.1.89 via ge-0/0/1.46

admin@PE_MXR02> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both .. snip .. 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:24:56, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 00:24:21, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329

Capture 1 – Link between P_R03 and PE_MXR01

Capture 1, Frame 4 was the ICMP reply from CE_R28 to CE_R27 as seen from P_R03 and PE_MXR01.  According to Wireshark’s analysis (arrows), this was Frame 2’s corresponding reply.  

As P_R03 forwarded the packet to PE_MXR01, it popped the top label of 329.  The VPN label of 306800 was looked up, associated to the EVPN instance (via Ingress-Aliasing since Type 1 route’s label was used) and packet delivered to CE_R27.  Again, traffic forwarding in this direction looked normal.

admin@PE_MXR01> show route table mpls.0 label 306800 mpls.0: 44 destinations, 44 routes (44 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 306800             *[EVPN/7] 00:19:49, routing-instance EVPN_CUSTOMER_G_ELAN_500, route-type Ingress-Aliasing                       to table EVPN_CUSTOMER_G_ELAN_500.evpn-mac.0                     [EVPN/7] 00:19:15, routing-instance EVPN_CUSTOMER_G_ELAN_500, route-type Ingress-MAC, vlan-id 500                       to table EVPN_CUSTOMER_G_ELAN_500.evpn-mac.0

Conclusion

From my observation and analysis in this lab, traffic forwarding in single-active redundancy mode operated as expected per my understanding of EVPN multi-homing.  I didn’t experience any odd behaviors in regards to routing exchange and label usage.

MPLS L2VPN (EVPN on JunOS with Single-Active Multi-Homing)

Exploring EVPN’s multi-homing capabilities wouldn’t be complete without a detailed look into Single-Active mode.  To finish off my testing, I modified my previous All-Active lab to Single-Active to observe the differences between the two redundancy modes.

I was also motivated to understand MPLS label usage, as I experienced some inconsistent behavior with label usage during the All-Active lab.  My results and documentation can be found here.  Hopefully this lab will reinforce my understanding of multi-homing operation in EVPN.

Lab Environment

Note: No additional devices were added or changed.

Diagram & Topology

CE_R27’s G1 and G2 interfaces were reconfigured from an Ether-Channel to an L2 bridge domain with a BDI acting as the IP enabled interface (highlighted).

Configurations Changes

Minor configuration changes were only made to PE_MXR01, PE_MXR03 and CE_R27.  No changes to PE_MXR02 were made.

Attachment Circuit

Redundancy mode was changed from all-active to single-active.

PE_MXR01

interfaces {     ge-0/0/2 {         description "TO CUSTOMER_G ELAN 500 VSWITCH_24";         flexible-vlan-tagging;         encapsulation flexible-ethernet-services;         esi {             00:11:22:33:44:55:66:77:88:99;             single-active;         }         unit 500 {             family bridge {                 interface-mode trunk;                 vlan-id-list [ 500 501 ];             }         }     }

PE_MXR03

interfaces {     ge-0/0/3 {         description "TO CUSTOMER_G ELAN 500 VSWITCH_27";         flexible-vlan-tagging;         encapsulation flexible-ethernet-services;         esi {             00:11:22:33:44:55:66:77:88:99;             single-active;         }         unit 500 {             family bridge {                 interface-mode trunk;                 vlan-id-list [ 500 501 ];             }         }     }

CE_R27

The CE’s Ethernet interfaces facing the PEs were re-configured to be in a L2 bridge domain.

! bridge-domain 50  mac aging-time 30 ! interface GigabitEthernet1  description TO PE_MXR01  no ip address  load-interval 30  negotiation auto  no mop enabled  no mop sysid  service instance 50 ethernet   encapsulation untagged   bridge-domain 50  ! ! interface GigabitEthernet2  description TO PE_MXR03  no ip address  load-interval 30  negotiation auto  no mop enabled  no mop sysid  service instance 50 ethernet   encapsulation untagged   bridge-domain 50  ! ! interface BDI50  description BD TO PE_MXR01 & PE_MXR03  ip address 172.16.50.1 255.255.255.0  load-interval 30  no mop enabled  no mop sysid !

Verification Tasks

The list below outlines commands used to verify EVPN operation in a single-active configuration during normal operating conditions.

• Show EVPN Instance Extensive
• Show Route Advertising-Protocol BGP
• Show Route Table EVPN

EVPN Instance

The outputs below displays the EVPN operational information from all PEs during normal conditions.  In single-active redundancy mode, DF and BDF election determines which PE is the primary and backup PE for the ES (as explained in RFC 7432 Sec. 14.1.1).

PE_MXR01 is the DF and is the primary PE.  The remote PE 113.113.113.113 (PE_MXR03) is the backup.  This PE will advertise its aliasing label of 306800 (via the Type 1 EAD per EVI route) to signal other PEs to only forward traffic to this PE for the ES.  

admin@PE_MXR01> show evpn instance extensive    Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 111.111.111.111:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   307248          Disabled     501                  1    1                Extended         Enabled   307264          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         1         0         0         2         0     113.113.113.113         0         0         1         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by IFL ge-0/0/2.500       Local interface: ge-0/0/2.500, Status: Up/Forwarding       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     0               single-active       Designated forwarder: 111.111.111.111       Backup forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 19:06:20       Advertised MAC label: 306800       Advertised aliasing label: 306800       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 111.111.111.111:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         0         0         0         0         1

PE_MXR02 shows its received the aliasing label from 111.111.111.111 but not from 113.113.113.113.  This would indicate that 111.111.111.111 is the primary PE and any traffic to the ES should be forwarded there. 

admin@PE_MXR02> show evpn instance extensive    Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 112.112.112.112:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       2     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   299840          Disabled     501                  1    1                Extended         Enabled   299856          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         1         0         2         2         0     113.113.113.113         1         0         1         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048576       Number of remote PEs connected: 2         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     0               single-active         111.111.111.111  306800     306800          single-active Instance: __default_evpn__   Route Distinguisher: 112.112.112.112:0   Number of bridge domains: 0   Number of neighbors: 0

PE_MXR03 is the backup PE for the ES based on DF/BDF election and is in the blocking state.  Egress traffic from this PE to the ES will take the path towards 111.111.111.111 via its received aliasing label.

admin@PE_MXR03> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 113.113.113.113:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       0       3     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 2 (2 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root     ge-0/0/3.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  2    2                Extended         Enabled   300528          Disabled     501                  2    2                Extended         Enabled   300544          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         1         0         2         2         0     112.112.112.112         2         0         0         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048578       Local interface: ge-0/0/3.500, Status: Up/Blocking       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         111.111.111.111  306800     306800          single-active       Designated forwarder: 111.111.111.111       Backup forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:21:38       Advertised MAC label: 304176       Advertised aliasing label: 304176       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 113.113.113.113:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         0         0         0         0         1

Route Advertising

With the redundancy mode changed to single-active, Type 1 EAD per EVI route export behave slightly differently than in an all-active configuration.  This route type is responsible for advertising the aliasing label used to reach a multi-homed ES.  Since one PE is forwarding at any given time, the designated backup PE will withhold its Type 1 EAD per EVI route advertisement.  This method is used to signal other PEs to only forward to the primary PE.  The Type 4 ES route exchange must still take place to elect the DF and BDF.

PE_MXR01, being the primary PE, will advertise its Type 1 EVI route with the aliasing label.  The Type 4 route is still advertised for DF/BDF purposes.

admin@PE_MXR01> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:111.111.111.111:50::500::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307248      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111 * 3:111.111.111.111:50::501::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307264      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR01> show route advertising-protocol bgp 113.113.113.113 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:111.111.111.111:50::500::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307248      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111 * 3:111.111.111.111:50::501::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307264      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

PE_MXR03, being the backup PE, will not advertise its Type 1 EVI and not seen here.  The Type 4 route is still advertised for DF/BDF purposes.

admin@PE_MXR03> show route advertising-protocol bgp 111.111.111.111 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR03> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

Route Table EVPN

The EVPN route table will show the full route table for all PEs during normal operation. 

PE_MXR01, being the primary PE for the ES, will not receive any Type 1 EVI routes from the other PEs.  The Type 4 route is still imported from 113.113.113.113 for DF/BDF purposes.

admin@PE_MXR01> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:19:10, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:17:37, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES                               *[BGP/170] 00:19:10, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330

PE_MXR02 imports the Type 1 EVI route from the primary PE (111.111.111.111).  Traffic to the ES egressing this PE will forward only to 111.111.111.111.  Since this PE contains no multi-homed hosts, no type 4 routes were imported.

admin@PE_MXR02> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:24:58, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:24:56, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:24:55, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 00:24:21, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:111.111.111.111:50::500::111.111.111.111/248 IM                               *[BGP/170] 1d 00:58:24, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:111.111.111.111:50::501::111.111.111.111/248 IM                               *[BGP/170] 1d 00:58:24, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:15:47, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:15:47, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330

PE_MXR03, being the backup PE, receives the Type 1 EVI route from 111.111.111.111.  The Type 4 route is still imported from 111.111.111.111 for DF/BDF purposes.

admin@PE_MXR03> show route table bgp.evpn.0                            bgp.evpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 1d 04:27:03, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 1d 04:27:01, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 05:08:49, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 05:08:49, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:111.111.111.111:50::500::111.111.111.111/248 IM                               *[BGP/170] 1d 04:27:02, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 3:111.111.111.111:50::501::111.111.111.111/248 IM                               *[BGP/170] 1d 04:27:02, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 2d 23:02:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 2d 23:02:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES                               *[BGP/170] 1d 04:27:04, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229

Verification Tasks (During Failure)

The CE’s AC interfaces towards PE_MXR01 were shutdown to simulate an ES failure.

Simulated Failure of an Ethernet Segment on PE_MXR01

CE_R27’s G1 interface was shutdown to simulate an Ethernet segment failure to PE_MXR01.

CE_R27#conf t Enter configuration commands, one per line.  End with CNTL/Z. CE_R27(config)#int g1 CE_R27(config-if)#shut CE_R27(config-if)#end

CE_R27#sh int desc Interface                      Status         Protocol Description Gi1                            admin down     down     TO PE_MXR01 Gi2                            up             up       TO PE_MXR03 Gi3                            admin down     down     Gi4                            up             up       MANAGEMENT BD50                           up             up       BD TO PE_MXR01 & PE_MXR03

PE_MXR01’s AC interface towards CE_R27’s G1 interface was shutdown to ensure a failure at both ends.

admin@PE_MXR01> configure Entering configuration mode [edit] admin@PE_MXR01# set interfaces ge-0/0/2 disable [edit] admin@PE_MXR01# commit commit complete

admin@PE_MXR01# run show interfaces terse ge-0/0/2  Interface               Admin Link Proto    Local                 Remote ge-0/0/2                down  down ge-0/0/2.500            up    down bridge  ge-0/0/2.32767          up    down multiservice

The list below outlines commands used to verify EVPN operation in a single-active configuration during a failure condition.

• Show EVPN Instance Extensive
• Show Route Advertising-Protocol BGP
• Show Route Table EVPN

EVPN Instance (During Failure)

The outputs below will display the EVPN operational information from all PEs during an ES failure.

During PE_MXR01’s local ES failure, the DF switches to PE_MXR03.  The redundant ES is now reachable via the backup PE.

admin@PE_MXR01> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 111.111.111.111:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       0       3     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (0 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:11:22:33:44:55:66:77:88:99  single-active    Down       Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    0                Extended         Enabled                   Disabled     501                  1    0                Extended         Enabled                   Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         2         0         0         2         0     113.113.113.113         1         0         2         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048578       Local interface: ge-0/0/2.500, Status: Down       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     304176          single-active       Designated forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:54:29       Advertised MAC label: 306800       Advertised aliasing label: 306800       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 111.111.111.111:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         0         0         0         0         1

From PE_MXR02, the ES is now reachable via the backup PE 113.113.113.113 (PE_MXR03).

admin@PE_MXR02> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 112.112.112.112:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   299840          Disabled     501                  1    1                Extended         Enabled   299856          Disabled   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         1         0         2         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048576       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     304176          single-active Instance: __default_evpn__   Route Distinguisher: 112.112.112.112:0   Number of bridge domains: 0   Number of neighbors: 0

This PE is now the forwarding PE and the ES is reachable via its local interface.

admin@PE_MXR03> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 113.113.113.113:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 2 (2 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root     ge-0/0/3.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  2    2                Extended         Enabled   300528          Disabled     501                  2    2                Extended         Enabled   300544          Disabled   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         1         0         0         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by IFL ge-0/0/3.500       Local interface: ge-0/0/3.500, Status: Up/Forwarding       Designated forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:54:29       Advertised MAC label: 304176       Advertised aliasing label: 304176       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 113.113.113.113:0   Number of bridge domains: 0   Number of neighbors: 0

Route Advertising

With PE_MXR01’s ES down, it withdraws all its routes to signal other PEs of the failure.

admin@PE_MXR01> show route advertising-protocol bgp 112.112.112.112    admin@PE_MXR01>

admin@PE_MXR01> show route advertising-protocol bgp 113.113.113.113    admin@PE_MXR01>

PE_MXR03 now advertises its Type 1 EAD per EVI route (with aliasing label) to advertise its reachability to this ES.

admin@PE_MXR03> show route advertising-protocol bgp 111.111.111.111 extensive EVPN_CUSTOMER_G_ELAN_500.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) * 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:0c:29:e4:3e:cd/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 299776      ESI: 00:00:00:00:00:00:00:00:00:00      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR03> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) * 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:0c:29:e4:3e:cd/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 299776      ESI: 00:00:00:00:00:00:00:00:00:00      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

Route Table EVPN (During Failure)

The outputs below will observe the route exchange from all PEs during an ES failure.

With PE_MXR01’s ES down, it imports the Type 1 EAD per EVI (aliasing) route from the backup PE 113.113.113.113 (PE_MXR03).  Traffic egressing this PE will forward to 113.113.113.113.

admin@PE_MXR01> show route table bgp.evpn.0 bgp.evpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:44:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:11:27, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:42:50, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                            *[BGP/170] 00:11:24, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:112.112.112.112:50::500::112.112.112.112/248 IM                                *[BGP/170] 1d 01:17:52, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES                               *[BGP/170] 00:44:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330

PE_MXR02 also receives the Type 1 EAD per EVI route from the backup PE 113.113.113.113 (PE_MXR03) and will now have an aliasing label to forward traffic to it.

admin@PE_MXR02> show route table bgp.evpn.0 bgp.evpn.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:47:31, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:14:36, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 00:14:33, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:38:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:38:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330

With PE_MXR01’s ES down, no routes are received from 111.111.111.111.

admin@PE_MXR03> show route table bgp.evpn.0    bgp.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:50:19, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 1d 01:42:43, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 01:42:43, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 22

References

EVPN Single-Active Multi-Homing

https://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-bgp-multihoming-overview.html

https://www.juniper.net/documentation/en_US/junos/topics/example/example-evpn-a-s-basic.html

Other Blogs and Labs

https://tgregory.org/2016/06/16/evpn-single-active-redundancy/