Aldrin IsaacCo-author RFC7432Juniper Networks

EVPN VXLANDemystified

Infrastructure Subsystems -- Requirements

Infrastructure as a Service

Infrastructure Resource Optimization

Physical Infrastructure

Network Subsystems -- Requirements

Network Virtualization

Traffic Optimization

WAN FabricLAN Fabric

LAN WAN

Workload Optimization

Network Virtualization

Network Subsystems -- Solutions

EVPN + Overlay Controller

MPLE TE + MPLS TE Controller

IP/MPLS BackboneIP Fabric

LAN WAN

Workload Controller

Network Modularity and EVPN

EVPN

IP Network

BGP

IGP

BGP

Network Modularity and EVPN

Why EVPN for Network Virtualization?

• Most pragmatic reason ⇒ there’s no better open alternative at this time• Gathering strong interest and adoption across multiple domains

• Built on top of trusted robust technologies – BGP and IP networks

• Simplified end-to-end network using a common technology across all multiple domains.• Translates to talent availability and fungibility, common tools, predictability, lower cost, etc

• Robust machinery for control and data plane scaling with minimum blast radius

• Seamless integration between IP and Ethernet forwarding paradigms with flexible overlay types and topologies

• All-active Ethernet multi-homing with end-to-end traffic load balancing

• Standards-based control plane based federation between autonomous systems

• Standards-based and operator-friendly network virtualization platform for SDN

8

A Simple Network Virtualization Overlay Reference Model

Tenant 1

VLAN2VLAN1

E2E1E4E3

Tenant 2

VLAN4VLAN3

E5 E6E7 E8

• For this talk, “tenants” are groups of location-independent endpoints where:

• Groups manifest as subnets that are routed to other groups of the same tenant (i.e. east-west) via a distributed routing function

• Tenants are routed to other tenants and to external destinations (i.e. north-south) through service function chains

• Tenants and groups are implemented as IP and Ethernet overlay virtual networks

• The network virtualization edge (NVE) function may be implemented on

• ToR switch: to support physical end-systems

• Virtual routers: to support virtual end-points

• Note: NVE are also referred to as PE in SP networks, or VTEP in VXLAN networks.

SF

VRF2 VRF1 VRF2

E4

VRF1 VRF2

E3 E8E7E6E5

VRF1

E1 E2

OverlayEdge

VXLAN overlay data plane

BGP Route Reflectors

“NVE”“VTEP”“PE”

VLAN2 VLAN3 VLAN4VLAN1VLAN3 VLAN4 VLAN1 VLAN2

IP EVPN

Broadcast DomainEVPN Tag

VXLAN VNI

Broadcast DomainEVPN Tag

VXLAN VNI

Ethernet EVPNaka EVI zz

Virtual Switchaka MAC-VRF

Physical Switch

Physical Switch Physical Switch

EVPN Parallels with Classical Networks

VTEPNVE / PE

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

VLAN Table

Virtual Routeraka VRF

Physical Router

Physical Router

Physical Router

EVPN Network Classical Network

IRB Interfaces

IP Fabric

Multi-Tenant Single-Tenant

MP-BGP Route Reflector

VTEP 2

VTEP 3

BGP-based VPNs Overview

VTEP 1MP-BGP

EVPN

IPVPN-A

Broadcast DomainEVPN Tag

VXLAN VNI

Broadcast DomainEVPN Tag

VXLAN VNI

EVI-A

Export local routes with Route Target 1111:1111

Import remote routes with Route Target 1111:1111

Export local routes with Route Target 2222:2222

Import remote routes with Route Target 2222:2222

MAC-VRF-A BGP Policy

VRF-A BGP Policy

MAC-VRF-A

VRF-A

VLAN 10EVPN Tag 100

VXLAN VNI 100

VLAN 20EVPN Tag 200

VXLAN VNI 200

IP Fabric

Tunnels

L3 Routes

L2 Routes

L1 Routes

L1: Ethernet Multi-Homing • Type-4 Ethernet Segment (ES) Route

• Designated Forwarder (DF) election

• Type-1 Ethernet A-D Route• Per ES

• Split horizon, Fast convergence• Per EVI (ES:Tag)

• Aliasing

• Type-7 Multicast Join Sync Route• Selective IP multicast support

• Type-8 Multicast Leave Sync Route• Selective IP multicast support

L2: Ethernet Bridging• Type-2 MAC/IP route

• MAC-Only• MAC unicast forwarding

• MAC + IP• ARP Proxy

• Type-3 Inclusive Multicast Ethernet Tag (IMET) Route• BUM forwarding

• Type-6 Selective Multicast Ethernet Tag (SMET) Route• Selective IP multicast forwarding

EVPN Route Types – Modularity by OSI Layer

L3: IP Routing• Type-5 IP Prefix Route

• IP Unicast Forwarding

Unicast• L1: Type-1 Ethernet A-D Route per ES

• Fast convergence

• L1: Type-1 Ethernet A-D Route per EVI• Aliasing

• L2: Type-2 MAC/IP route• MAC unicast forwarding, ARP Proxy **

• L3: Type-5 Prefix Route Route• IP forwarding

EVPN Route Types – Modularity by Unicast / Multicast

BUM and IP Multicast• L1: Type-1 Ethernet A-D Route per ES

• Split horizon

• L1: Type-4 Ethernet Segment (ES) Route• Designated Forwarder (DF) election

• L1: Type-7 Multicast Join Sync Route• Selective IP multicast support

• L1: Type-8 Multicast Leave Sync Route• Selective IP multicast support

• L2: Type-3 Inclusive Multicast Ethernet Tag (IMET) Route• BUM forwarding

• L2: Type-6 Selective Multicast Ethernet Tag (SMET) Route **• Selective IP multicast forwarding

MAC-VRF-TMAC-VRF-T

Pure Bridging Overlay

VTEP1 VTEP2

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

Type-2 MAC, Type-3 IMET

← MAC →

VTEP3

VTEP1 VTEP2← MAC/IP →

← MAC/IP →

MAC-VRF-T

VLAN1 VLAN2

VRF-T

← MAC/IP →

MAC-VRF-TMAC-VRF-T

Centrally Routed Bridging Overlay (CRB)

VLAN1 VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

Type-2 MAC, Type-3 IMET

Type-2 MAC, MAC+IP

Type-2 MAC, MAC+IP

CRB Access

CRB Access

CRB Gateway

← MAC/IP →

MAC-VRF-TMAC-VRF-T

VLAN1 VLAN2 VLAN1VLAN2

Edge Routed Bridging Overlay (ERB)

VTEP1 VTEP2

← Host IP →

IP EVPNVRF-T VRF-T

L2 EVPN

H2 H3H1 H4

Type-2 MAC, Type-3 IMET

Local

Type-5 IP Host

, Type-2 MAC, MAC+IP

IP Routed Overlay -- Full Mesh Topology

VTEP3

VTEP1 VTEP2

Tk →

VRF-T VRF-T← TjT i →

← Tj

VRF-T

Import RT-TExport RT-T

Import RT-TExport RT-T

Import RT-TExport RT-T

← T k

Ti →

Type-5 IP Prefix

H1 H2

IP EVPN

VTEP3

← Aggregates

IP Routed Overlay -- Hub-and-spoke/Directional Topology

VTEP1 VTEP2

G →

VRF-X VRF-X

← G

Xi →← Xj

VRF-G

Import RT-GExport RT-X

Import RT-GExport RT-X

Import RT-XExport RT-G

Type-5 IP PrefixBorder

IP EVPN

MAC-VRF-TMAC-VRF-T

Edge Routed Bridging with IP Border Gateway

VTEP3

VTEP1 VTEP2

← Host IP →

Default →

IP EVPN

VLAN1

VRF-T VRF-T

VLAN2

L2 EVPN

VLAN1VLAN2

H2 H3H1 H4

← D

efault

Host IP

→← Host IP

Type-2 MAC, Type-3 IMET

Local

Type-5 IP Host

Type-5 IP Prefix← Aggregates VRF-G

← MAC/IP →

Border

VR

F-L9

BD-L2-1

BD-L2-2

Tenant-L2

VR

F-L8

BD-L1-1

BD-L1-2

Tenant-L1

BD-R

9-2

Tenant-R2

VR

F-R

8 BD

-R8-

1BD

-R8-

2

Tenant-R1

BD-R

8-FW

1

ERB CRB

Bridged

VR

F-SF1-L

SF1-L

BD-SF1-L

ERBH & S L3VN

ServiceFunction

GW1aL3

GW1bL3

Gateway

SF1aL3

SF1bL3

VR

F-SF1-R

BD-SF1-R

SF1-R

VR

F-SF1-L

SF2-L

IPERB H & SL3VN

ServiceFunction

SF2aL1

SF2bL1

SF2-R

VR

F-SF1-R

VR

F-SF1-L

BD-SF1-L

GW1-L

IP H & SL3VN

ERBERB

Service Chaining with these Building Blocks

Ethernet Multihoming

• EVPN supports N-way Ethernet multihoming where N can be greater than 2

• No ICL link required

• Multi-homed end-systems are identified in the overlay by unique Ethernet Segment ID (ESI).

• EVPN Auto-ESI -- ESI generated automatically from LACP system-id or from BPDU root bridge snooping

LAG Trunk

VLAN1 VLAN2 VLAN1 VLAN2 VLAN1 VLAN2

VLAN1 VLAN2

ESI-2ESI-1 ESI-2ESI-1 ESI-2ESI-1

LAG

VTEP3VTEP2VTEP1

BD1

BD1

VTEP 1

VTEP 2

Source

Receivers

BD1

VTEP 3

Receivers

“Stateless” IP Core

• Overlay replication uses “over-the-top” signaling• No hop-by-hop per-flow or per-group multicast

signaling or BUM state in underlay• No traditional underlay multicast protocols

translates to lean core network design• Multicast convergence “same as” unicast

convergence on transit link or node failure• Selective IP multicast replication capability allows

for IP multicast flow to only be replicated by an ingress VTEP only to egress VTEP that have at least one active receiver for that flow

• Further optimizations possible with Assisted Replication (AR) and Optimized Inter-Subnet Replication (OISM)

Pure Overlay BUM Replication

EVPN ARP Proxy -- Synchronization and Suppression

• ARP synchronization keeps the per-subnet ARP tables of tenant VRFs synchronized

• MAC-to-IP bindings are learned by Access VTEP from the Sender field of local ARP request and reply packets and advertised as Type-2 MAC+IP routes

• With distributed ARP broadcast suppression, Leaf VTEP will proxy respond to local ARP requests using the same synchronized MAC-to-IP bindings

• Reduces the impact of ARP broadcast on routers and hosts

ARP Suppression

OriginalARP response

GeneratedARP response

GeneratedARP response

VTEP3

H2 H3

ARP request ARP request

1 3

4

VTEP2

H1

VTEP1

Subnet 1 Subnet 1 Subnet 1

MAC/IP Route2

Flow 2

ARP Synchronization

VRF1VRF1

ARP request

ARP response

Flow 1

15

3

MAC/IP Route

24

Leaf

Gateway

Closing Thoughts

• EVPN based networks are only as complex as they need to be• Most use cases can be satisfied with only a few key building blocks • Complexity is proportional to the functionality required

• EVPN VXLAN is an open standard. Equivalent proprietary technology is not any simpler.

23

Thank You