MPLS L2VPN (EVPN on JunOS with Single-Active Multi-Homing)

Exploring EVPN’s multi-homing capabilities wouldn’t be complete without a detailed look into Single-Active mode.  To finish off my testing, I modified my previous All-Active lab to Single-Active to observe the differences between the two redundancy modes.

I was also motivated to understand MPLS label usage, as I experienced some inconsistent behavior with label usage during the All-Active lab.  My results and documentation can be found here.  Hopefully this lab will reinforce my understanding of multi-homing operation in EVPN.

Lab Environment

Note: No additional devices were added or changed.

Diagram & Topology

CE_R27’s G1 and G2 interfaces were reconfigured from an Ether-Channel to an L2 bridge domain with a BDI acting as the IP enabled interface (highlighted).

Configurations Changes

Minor configuration changes were only made to PE_MXR01, PE_MXR03 and CE_R27.  No changes to PE_MXR02 were made.

Attachment Circuit

Redundancy mode was changed from all-active to single-active.

PE_MXR01

interfaces {     ge-0/0/2 {         description "TO CUSTOMER_G ELAN 500 VSWITCH_24";         flexible-vlan-tagging;         encapsulation flexible-ethernet-services;         esi {             00:11:22:33:44:55:66:77:88:99;             single-active;         }         unit 500 {             family bridge {                 interface-mode trunk;                 vlan-id-list [ 500 501 ];             }         }     }

PE_MXR03

interfaces {     ge-0/0/3 {         description "TO CUSTOMER_G ELAN 500 VSWITCH_27";         flexible-vlan-tagging;         encapsulation flexible-ethernet-services;         esi {             00:11:22:33:44:55:66:77:88:99;             single-active;         }         unit 500 {             family bridge {                 interface-mode trunk;                 vlan-id-list [ 500 501 ];             }         }     }

CE_R27

The CE’s Ethernet interfaces facing the PEs were re-configured to be in a L2 bridge domain.

! bridge-domain 50  mac aging-time 30 ! interface GigabitEthernet1  description TO PE_MXR01  no ip address  load-interval 30  negotiation auto  no mop enabled  no mop sysid  service instance 50 ethernet   encapsulation untagged   bridge-domain 50  ! ! interface GigabitEthernet2  description TO PE_MXR03  no ip address  load-interval 30  negotiation auto  no mop enabled  no mop sysid  service instance 50 ethernet   encapsulation untagged   bridge-domain 50  ! ! interface BDI50  description BD TO PE_MXR01 & PE_MXR03  ip address 172.16.50.1 255.255.255.0  load-interval 30  no mop enabled  no mop sysid !

Verification Tasks

The list below outlines commands used to verify EVPN operation in a single-active configuration during normal operating conditions.

• Show EVPN Instance Extensive
• Show Route Advertising-Protocol BGP
• Show Route Table EVPN

EVPN Instance

The outputs below displays the EVPN operational information from all PEs during normal conditions.  In single-active redundancy mode, DF and BDF election determines which PE is the primary and backup PE for the ES (as explained in RFC 7432 Sec. 14.1.1).

PE_MXR01 is the DF and is the primary PE.  The remote PE 113.113.113.113 (PE_MXR03) is the backup.  This PE will advertise its aliasing label of 306800 (via the Type 1 EAD per EVI route) to signal other PEs to only forward traffic to this PE for the ES.  

admin@PE_MXR01> show evpn instance extensive    Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 111.111.111.111:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   307248          Disabled     501                  1    1                Extended         Enabled   307264          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         1         0         0         2         0     113.113.113.113         0         0         1         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by IFL ge-0/0/2.500       Local interface: ge-0/0/2.500, Status: Up/Forwarding       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     0               single-active       Designated forwarder: 111.111.111.111       Backup forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 19:06:20       Advertised MAC label: 306800       Advertised aliasing label: 306800       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 111.111.111.111:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         0         0         0         0         1

PE_MXR02 shows its received the aliasing label from 111.111.111.111 but not from 113.113.113.113.  This would indicate that 111.111.111.111 is the primary PE and any traffic to the ES should be forwarded there. 

admin@PE_MXR02> show evpn instance extensive    Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 112.112.112.112:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       2     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   299840          Disabled     501                  1    1                Extended         Enabled   299856          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         1         0         2         2         0     113.113.113.113         1         0         1         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048576       Number of remote PEs connected: 2         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     0               single-active         111.111.111.111  306800     306800          single-active Instance: __default_evpn__   Route Distinguisher: 112.112.112.112:0   Number of bridge domains: 0   Number of neighbors: 0

PE_MXR03 is the backup PE for the ES based on DF/BDF election and is in the blocking state.  Egress traffic from this PE to the ES will take the path towards 111.111.111.111 via its received aliasing label.

admin@PE_MXR03> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 113.113.113.113:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       0       3     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 2 (2 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root     ge-0/0/3.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  2    2                Extended         Enabled   300528          Disabled     501                  2    2                Extended         Enabled   300544          Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         1         0         2         2         0     112.112.112.112         2         0         0         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048578       Local interface: ge-0/0/3.500, Status: Up/Blocking       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         111.111.111.111  306800     306800          single-active       Designated forwarder: 111.111.111.111       Backup forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:21:38       Advertised MAC label: 304176       Advertised aliasing label: 304176       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 113.113.113.113:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     111.111.111.111         0         0         0         0         1

Route Advertising

With the redundancy mode changed to single-active, Type 1 EAD per EVI route export behave slightly differently than in an all-active configuration.  This route type is responsible for advertising the aliasing label used to reach a multi-homed ES.  Since one PE is forwarding at any given time, the designated backup PE will withhold its Type 1 EAD per EVI route advertisement.  This method is used to signal other PEs to only forward to the primary PE.  The Type 4 ES route exchange must still take place to elect the DF and BDF.

PE_MXR01, being the primary PE, will advertise its Type 1 EVI route with the aliasing label.  The Type 4 route is still advertised for DF/BDF purposes.

admin@PE_MXR01> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:111.111.111.111:50::500::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307248      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111 * 3:111.111.111.111:50::501::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307264      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR01> show route advertising-protocol bgp 113.113.113.113 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 306800      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:111.111.111.111:50::500::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307248      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307248: Type INGRESS-REPLICATION 111.111.111.111 * 3:111.111.111.111:50::501::111.111.111.111/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:50      Route Label: 307264      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 307264: Type INGRESS-REPLICATION 111.111.111.111 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 111.111.111.111:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

PE_MXR03, being the backup PE, will not advertise its Type 1 EVI and not seen here.  The Type 4 route is still advertised for DF/BDF purposes.

admin@PE_MXR03> show route advertising-protocol bgp 111.111.111.111 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR03> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 10 destinations, 10 routes (10 active, 0 holddown, 0 hidden) * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

Route Table EVPN

The EVPN route table will show the full route table for all PEs during normal operation. 

PE_MXR01, being the primary PE for the ES, will not receive any Type 1 EVI routes from the other PEs.  The Type 4 route is still imported from 113.113.113.113 for DF/BDF purposes.

admin@PE_MXR01> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:19:10, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:17:37, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 00:52:39, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES                               *[BGP/170] 00:19:10, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330

PE_MXR02 imports the Type 1 EVI route from the primary PE (111.111.111.111).  Traffic to the ES egressing this PE will forward only to 111.111.111.111.  Since this PE contains no multi-homed hosts, no type 4 routes were imported.

admin@PE_MXR02> show route table bgp.evpn.0 bgp.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:24:58, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:24:56, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:24:55, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 00:24:21, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:111.111.111.111:50::500::111.111.111.111/248 IM                               *[BGP/170] 1d 00:58:24, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:111.111.111.111:50::501::111.111.111.111/248 IM                               *[BGP/170] 1d 00:58:24, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 329 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:15:47, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:15:47, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330

PE_MXR03, being the backup PE, receives the Type 1 EVI route from 111.111.111.111.  The Type 4 route is still imported from 111.111.111.111 for DF/BDF purposes.

admin@PE_MXR03> show route table bgp.evpn.0                            bgp.evpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:111.111.111.111:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 1d 04:27:03, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 1:111.111.111.111:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 1d 04:27:01, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 2:111.111.111.111:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 05:08:49, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 05:08:49, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:111.111.111.111:50::500::111.111.111.111/248 IM                               *[BGP/170] 1d 04:27:02, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 3:111.111.111.111:50::501::111.111.111.111/248 IM                               *[BGP/170] 1d 04:27:02, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 2d 23:02:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 2d 23:02:39, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 4:111.111.111.111:0::112233445566778899:111.111.111.111/296 ES                               *[BGP/170] 1d 04:27:04, localpref 100, from 111.111.111.111                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 229

Verification Tasks (During Failure)

The CE’s AC interfaces towards PE_MXR01 were shutdown to simulate an ES failure.

Simulated Failure of an Ethernet Segment on PE_MXR01

CE_R27’s G1 interface was shutdown to simulate an Ethernet segment failure to PE_MXR01.

CE_R27#conf t Enter configuration commands, one per line.  End with CNTL/Z. CE_R27(config)#int g1 CE_R27(config-if)#shut CE_R27(config-if)#end

CE_R27#sh int desc Interface                      Status         Protocol Description Gi1                            admin down     down     TO PE_MXR01 Gi2                            up             up       TO PE_MXR03 Gi3                            admin down     down     Gi4                            up             up       MANAGEMENT BD50                           up             up       BD TO PE_MXR01 & PE_MXR03

PE_MXR01’s AC interface towards CE_R27’s G1 interface was shutdown to ensure a failure at both ends.

admin@PE_MXR01> configure Entering configuration mode [edit] admin@PE_MXR01# set interfaces ge-0/0/2 disable [edit] admin@PE_MXR01# commit commit complete

admin@PE_MXR01# run show interfaces terse ge-0/0/2  Interface               Admin Link Proto    Local                 Remote ge-0/0/2                down  down ge-0/0/2.500            up    down bridge  ge-0/0/2.32767          up    down multiservice

The list below outlines commands used to verify EVPN operation in a single-active configuration during a failure condition.

• Show EVPN Instance Extensive
• Show Route Advertising-Protocol BGP
• Show Route Table EVPN

EVPN Instance (During Failure)

The outputs below will display the EVPN operational information from all PEs during an ES failure.

During PE_MXR01’s local ES failure, the DF switches to PE_MXR03.  The redundant ES is now reachable via the backup PE.

admin@PE_MXR01> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 111.111.111.111:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       0       3     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (0 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:11:22:33:44:55:66:77:88:99  single-active    Down       Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    0                Extended         Enabled                   Disabled     501                  1    0                Extended         Enabled                   Disabled   Number of neighbors: 2     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         2         0         0         2         0     113.113.113.113         1         0         2         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048578       Local interface: ge-0/0/2.500, Status: Down       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     304176          single-active       Designated forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:54:29       Advertised MAC label: 306800       Advertised aliasing label: 306800       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 111.111.111.111:0   Number of bridge domains: 0   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         0         0         0         0         1

From PE_MXR02, the ES is now reachable via the backup PE 113.113.113.113 (PE_MXR03).

admin@PE_MXR02> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 112.112.112.112:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 1 (1 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  1    1                Extended         Enabled   299840          Disabled     501                  1    1                Extended         Enabled   299856          Disabled   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     113.113.113.113         1         0         2         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by NH 1048576       Number of remote PEs connected: 1         Remote PE        MAC label  Aliasing label  Mode         113.113.113.113  304176     304176          single-active Instance: __default_evpn__   Route Distinguisher: 112.112.112.112:0   Number of bridge domains: 0   Number of neighbors: 0

This PE is now the forwarding PE and the ES is reachable via its local interface.

admin@PE_MXR03> show evpn instance extensive Instance: EVPN_CUSTOMER_G_ELAN_500   Route Distinguisher: 113.113.113.113:50   Per-instance MAC route label: 299776   MAC database status                     Local  Remote     MAC advertisements:                       1       1     MAC+IP advertisements:                    0       0     Default gateway MAC advertisements:       0       0   Number of local interfaces: 2 (2 up)     Interface name  ESI                            Mode             Status     AC-Role     ge-0/0/2.500    00:00:00:00:00:00:00:00:00:00  single-homed     Up         Root     ge-0/0/3.500    00:11:22:33:44:55:66:77:88:99  single-active    Up         Root   Number of IRB interfaces: 0 (0 up)   Number of bridge domains: 2     VLAN  Domain ID   Intfs / up    IRB intf   Mode             MAC sync  IM route label  SG sync  IM core nexthop     500                  2    2                Extended         Enabled   300528          Disabled     501                  2    2                Extended         Enabled   300544          Disabled   Number of neighbors: 1     Address               MAC    MAC+IP        AD        IM        ES Leaf-label     112.112.112.112         1         0         0         2         0   Number of ethernet segments: 1     ESI: 00:11:22:33:44:55:66:77:88:99       Status: Resolved by IFL ge-0/0/3.500       Local interface: ge-0/0/3.500, Status: Up/Forwarding       Designated forwarder: 113.113.113.113       Last designated forwarder update: Feb 01 01:54:29       Advertised MAC label: 304176       Advertised aliasing label: 304176       Advertised split horizon label: 0 Instance: __default_evpn__   Route Distinguisher: 113.113.113.113:0   Number of bridge domains: 0   Number of neighbors: 0

Route Advertising

With PE_MXR01’s ES down, it withdraws all its routes to signal other PEs of the failure.

admin@PE_MXR01> show route advertising-protocol bgp 112.112.112.112    admin@PE_MXR01>

admin@PE_MXR01> show route advertising-protocol bgp 113.113.113.113    admin@PE_MXR01>

PE_MXR03 now advertises its Type 1 EAD per EVI route (with aliasing label) to advertise its reachability to this ES.

admin@PE_MXR03> show route advertising-protocol bgp 111.111.111.111 extensive EVPN_CUSTOMER_G_ELAN_500.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) * 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:0c:29:e4:3e:cd/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 299776      ESI: 00:00:00:00:00:00:00:00:00:00      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

admin@PE_MXR03> show route advertising-protocol bgp 112.112.112.112 extensive    EVPN_CUSTOMER_G_ELAN_500.evpn.0: 8 destinations, 8 routes (8 active, 0 holddown, 0 hidden) * 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:0c:29:e4:3e:cd/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 299776      ESI: 00:00:00:00:00:00:00:00:00:00      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 304176      ESI: 00:11:22:33:44:55:66:77:88:99      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 * 3:113.113.113.113:50::500::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300528      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300528: Type INGRESS-REPLICATION 113.113.113.113 * 3:113.113.113.113:50::501::113.113.113.113/248 IM (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:50      Route Label: 300544      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50      PMSI: Flags 0x0: Label 300544: Type INGRESS-REPLICATION 113.113.113.113 __default_evpn__.evpn.0: 2 destinations, 2 routes (2 active, 0 holddown, 0 hidden) * 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: target:2345:50 esi-label:0x1:single-active (label 0) * 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES (1 entry, 1 announced)  BGP group PE type Internal      Route Distinguisher: 113.113.113.113:0      Nexthop: Self      Flags: Nexthop Change      Localpref: 100      AS path: [2345] I      Communities: es-import-target:11-22-33-44-55-66

Route Table EVPN (During Failure)

The outputs below will observe the route exchange from all PEs during an ES failure.

With PE_MXR01’s ES down, it imports the Type 1 EAD per EVI (aliasing) route from the backup PE 113.113.113.113 (PE_MXR03).  Traffic egressing this PE will forward to 113.113.113.113.

admin@PE_MXR01> show route table bgp.evpn.0 bgp.evpn.0: 9 destinations, 9 routes (9 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:44:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:11:27, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:42:50, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                            *[BGP/170] 00:11:24, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:112.112.112.112:50::500::112.112.112.112/248 IM                                *[BGP/170] 1d 01:17:52, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 328 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:17:52, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330 4:113.113.113.113:0::112233445566778899:113.113.113.113/296 ES                               *[BGP/170] 00:44:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.81 via ge-0/0/1.44, Push 330

PE_MXR02 also receives the Type 1 EAD per EVI route from the backup PE 113.113.113.113 (PE_MXR03) and will now have an aliasing label to forward traffic to it.

admin@PE_MXR02> show route table bgp.evpn.0 bgp.evpn.0: 5 destinations, 5 routes (5 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 1:113.113.113.113:0::112233445566778899::FFFF:FFFF/192 AD/ESI                           *[BGP/170] 00:47:31, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 1:113.113.113.113:50::112233445566778899::0/192 AD/EVI                           *[BGP/170] 00:14:36, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 2:113.113.113.113:50::500::00:1e:e5:c8:0f:bf/304 MAC/IP                           *[BGP/170] 00:14:33, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::500::113.113.113.113/248 IM                               *[BGP/170] 1d 01:38:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330 3:113.113.113.113:50::501::113.113.113.113/248 IM                               *[BGP/170] 1d 01:38:23, localpref 100, from 113.113.113.113                       AS path: I, validation-state: unverified                     > to 10.1.1.89 via ge-0/0/1.46, Push 330

With PE_MXR01’s ES down, no routes are received from 111.111.111.111.

admin@PE_MXR03> show route table bgp.evpn.0    bgp.evpn.0: 3 destinations, 3 routes (3 active, 0 holddown, 0 hidden) + = Active Route, - = Last Active, * = Both 2:112.112.112.112:50::500::00:0c:29:49:aa:8c/304 MAC/IP                           *[BGP/170] 00:50:19, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::500::112.112.112.112/248 IM                               *[BGP/170] 1d 01:42:43, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 228 3:112.112.112.112:50::501::112.112.112.112/248 IM                               *[BGP/170] 1d 01:42:43, localpref 100, from 112.112.112.112                       AS path: I, validation-state: unverified                     > to 10.1.1.97 via ge-0/0/1.48, Push 22

References

EVPN Single-Active Multi-Homing

https://www.juniper.net/documentation/en_US/junos/topics/concept/evpn-bgp-multihoming-overview.html

https://www.juniper.net/documentation/en_US/junos/topics/example/example-evpn-a-s-basic.html

Other Blogs and Labs

https://tgregory.org/2016/06/16/evpn-single-active-redundancy/