Arista Networks VXLAN Overvie · 2/21/2017 · Data Center Interconnect VM Mobility Naked STP EoMPLS VPLS OTV VXLAN EVPN. Conﬁdential. ... Industry Overview ... 10 GbE 40 GbE 1

Confidential. Copyright © Arista 2016. All rights reserved.

Alternate text color: Hex color# 112346

ARISTA color Palette


Arista Networks

Chris Kane, [email protected], CCIE #14430 @ccie14430 Columbus, OH (OH, MI, KY)

VXLAN – Overview

Agenda

2

What problem are we trying to solve? Data Center Interconnect VM Mobility Naked STP EoMPLS VPLS OTV VXLAN EVPN


Industry Overview

Disclaimer

All opinions expressed in this presentation are mine (Chris Kane) and do not necessarily reflect those of my

current or previous employers

3


Industry OverviewWhat Problem Are We Trying To Solve

4

One Data Center Redundancy Needed

Second Data Center (ex. Sungard)

Wasted Resources and Purchase Priority Rights

Own Second Data Center Wasted Resources if Active/DR and long recovery

Active/Active Mobility (ex. Re-IP for vMotion)

DCI (Data Center Interconnect) STP issues propagate quickly

I want Active/Active, no re-IP, no STP eruptions and it must be easy to manage


Naked STP

•  “We’ll just extend Layer 2 from Data Center 1 to Data Center 2” -  STP Loop Prevention kicks in and blocks some of our paths ≫  Less bandwidth

≫  Longer convergence events (failure detection, MAC address table learning, etc…)

-  But what about First Hop Redundancy Protocols? -  Um….we had an STP issue in Data Center 1 that in a heartbeat just wiped out Data Center 2

5

Layer 2

Layer 2 Layer 2

Layer 2

Layer 2STP BPDUs

STP BPDUsLayer 2

STPBlocking

Loop Prevention

STPBlocking

Loop Prevention

STPBlocking

Loop Prevention


EoMPLS

•  AToM = Any Transport over MPLS

•  RFC 4448 (April 2006) Updated by RFC 5462 (February 2009)

•  To provide coverage for various use cases and leverage existing backbone MPLS; carriers offered both Layer 2 VPNs and Layer 3 VPNs

•  Ethernet is another service over MPLS (Also included ATM and Frame Relay)

•  A Pseudowire is a single point-to-point Layer 2 connection provided over an MPLS backbone

6


EoMPLS

•  Could buy Pseudowire service from a provider

•  Or could roll your own Pseudowire if you have the appropriate equipment

-  Vendors often required very specific hardware components and software versions

-  Much like VRF in that only the most basic elements of an MPLS network are needed

•  Two modes for Ethernet

-  One option is to map Pseudowires on a per-VLAN basis (VLAN mode)

-  Another associates the entire Ethernet interface to the Pseudowire (Port mode)

7


EoMPLS – Example

8

PE/P

PE/P PE/P

PE/P

Layer 2

Layer 2 Layer 2

Layer 2

Layer 3

Layer 3

LoopbackCable

LoopbackCable

LoopbackCable

LoopbackCable

EoMPLS

EoMPLS


EoMPLS – Example Configuration

•  On the PE router (Port mode) config t interface Gig1/0 xconnect x.x.x.x 10 encapsulation mpls

-  And on the P routers including the appropriate signaling protocol and label switch mapping of Tunnel label and inner VC label

9


EoMPLS

10

•  Pros -  Layer 2 extension over Layer 3 -  Service Provider managed

•  Cons -  Service Provider managed ≫  MRC ≫  Change Management

≫  Change Costs

-  No switching behaviors incorporated

-  If running your own; requires specific hardware options and software versions

-  High CAPEX -  High OPEX


VPLS

•  Virtual Private LAN Service

•  Where EoMPLS is often point-to-point (Pseudowires), VPLS provides an Ethernet multipoint service

•  Does more to take advantage of some switching features -  Data Plane learning ≫ MAC address learning and aging out are part of the service

•  While VPLS had been widely deployed by Service Providers, several short comings led to the development of other solutions (VXLAN -> EVPN) -  Lack of multi-site Active/Active forwarding support -  Limited Multicast support

11


VPLS - Example

12

CE

CE

CE

PE

PE

PE

P

Data EthernetHeader Data Ethernet

HeaderAToMHeader Data Ethernet

Header

Data 802.1QHeader Data 802.1Q

HeaderAToMHeader Data 802.1Q

Header

CE

AccessPort

AccessPort

TrunkPort

TrunkPort


VPLS – Configuration Example

PE example config t mpls label protocol ldp mpls ldp discovery targeted-hello accept mpls ldp router-id loopback0 force interface Gi1/0 ip address x.x.x.x/30 mpls ip I2 vfi <name> vpn id <oui:vpn-index> neighbor <remote-PE-loopback> encapsulation mpls interface vlan xxx xconnect vfi <vfi name>

13


VPLS

•  Pros -  Service Provider managed -  Multi-homing is supported

14

•  Cons -  Service Provider managed ≫  MRC ≫  Change Management

≫  Change Costs

-  Provider to Provider interconnects, like the last mile, may conflict (read Battle Scar)

-  If running your own; required specific hardware options and software versions

-  High CAPEX -  High OPEX -  Multi-homing with all active

forwarding is not supported -  Lacks Multicast support


OTV

•  Overlay Transport Virtualization

-  OTV is a MAC-in-IP method of extending Layer 2 over a Layer 3 infrastructure

-  Underlying routing protocol mechanism is IS-IS -  Supports being deployed as either Multicast-based or Unicast-based

•  Terminology -  Join Interface – WAN Layer 3 interface -  Overlay Interface – Virtual interface to perform encap/decap functions -  Internal Interface – Where Layer 2 adjacency to internal data center LAN

occurs -  Overhead = 42 byte header (needs Jumbo frames enabled on all possible

paths) -  Sets the DF bit on all packets

15

16

OTV Example

Layer 3 WAN

Layer 3 WAN

AED for Odd VLANs AED for Odd VLANs

AED for Even VLANs AED for Even VLANs

OTV

OTV

Data Center 1 Data Center 2

AED = Authoritative Edge Device

OTVVDC

OTVVDC

OTVVDC

OTVVDC

17

OTV – Multicast Configuration Example

config t feature otv otv site-vlan 101 otv site-identifier 0000.0000.0001 interface overlay1 otv join-interface port-channel 10 otv control-group 239.1.1.1 otv data-group 232.1.1.0/24 otv extend-vlan 2-20 interface port-channel 10 mtu 9216 ip address x.x.x.x/30 ip igmp v3

18

OTV – Unicast Configuration Example

config t feature otv otv site-vlan 101 otv site-identifier 0000.0000.0001 interface overlay1 otv join-interface port-channel 10 otv adjacency-server unicast-only otv use-adjacency-server x.x.x.x unicast-only otv extend-vlan 2-20 interface port-channel 10 mtu 9216 ip address x.x.x.x/30 ip igmp v3


OTV

•  Pros -  Layer 2 extension over Layer 3 -  N7K or ASR1K -  VLAN translation -  Supports First Hop Redundancy

•  Notables -  Jumbo Frames on all possible

paths

19

•  Cons -  AED Odd and Even ownership -  Specific hardware required -  Proprietary -  Not the go-forward strategy (VXLAN

and/or EVPN) -  PIM requirement for Multicast

deployments -  Join interface can’t have PIM enabled

because OTV sets that interface as an IGMP host

-  ISSU not available on devices running OTV. OTV must be shutdown


VXLAN

20


VXLAN

•  Virtual Extensible LAN

•  RFC 7348 -  Initial proposal August 2011, fully published in 2014

•  Industry Standard

•  UDP is used as a tunneling mechanism, wrapped around the MAC addresses

•  Used for intra-data center Layer 2 over Layer 3

•  Used for inter-data center Layer 2 over Layer 3 (most common in my patch)

21


VXLAN

•  Virtual Extensible LAN

•  Scalable for large throughput numbers -  Real world example; an IXC pushes 350 Gbps as an average run rate

•  More in the reading and due diligence than in the configuration and operation

•  Industry standard provides interoperability -  Either Hardware VTEP to Hardware VTEP -  Software VTEP to Software VTEP -  Or Software VTEP to Hardware VTEP

22


VXLAN - Header

23

24

VXLAN – Intra Data Center Example

Compute

Compute

LACP

LACP

Compute

Compute

LACP

LACP

Compute

Compute

LACP

LACP

MLAG

eBGP

Layer 3 Layer 3

VLANs 2 - 9 VLANs 10 - 19 VLANs 20 - 29

VTEP VTEP VTEP VTEP VTEP VTEP

Spine

Leaf

MLAGMLAG

10 GbE

40 GbE

1 GbE

100 GbE

Firewalls

Controllers

LACP

LACP

VTEP VTEP

Compute Leaf Compute Leaf Storage Leaf Services LeafVLANs 2 - 29

MLAG

VXLAN

25

VXLAN – Inter Data Center Example

Compute Compute

Compute

Layer 3 WAN

Layer 3 WAN

Compute

LACP

LACP

LACP

LACP

MLAG MLAG MLAGMLAG

MLAG MLAG

MLAG MLAG

VTEP

Compute

Compute

SVI

Jumbo Frames

Jumbo Frames

VXLAN

VXLAN

Compute

Compute

West Data Center East Data Center

Primary FW Secondary FW

10 GbE

40 GbE

1 GbE

Design Notes

One EOS image for all switch modelsLine rate forwarding on all interfacesActive/Active Forwarding for both Layer 2 and Layer 3Industry Standard VXLAN

Spine Spine

LeafLeaf

LeafLeaf

LeafLeaf

LeafLeaf

SpineSpine

VTEPVTEP

VTEPMLAG

MLAG

LACP

LACPLACP LACP

LACP

LACP

MLAG

MLAG

100 GbE

VXLAN

26

VXLAN – Inter Data Center Production Example


VXLAN – Configuration Example

Manual Flood List interface Vxlan1 vxlan source-interface Loopback1 vxlan udp-port 4789 vxlan flood vtep 161.1.1.4 161.1.1.5 vxlan vlan 101 vni 20101 vxlan vlan 102 vni 20102 vxlan vlan 103 vni 20103 vxlan vlan 104 vni 20104 vxlan vlan 105 vni 20105

27

Dynamic Flood List (CVX) interface Vxlan1 vxlan source-interface Loopback1 vxlan controller-client vxlan udp-port 4789 vxlan vlan 101 vni 20101 vxlan vlan 102 vni 20102 vxlan vlan 103 vni 20103 vxlan vlan 104 vni 20104 vxlan vlan 105 vni 20105


VXLAN – Example ‘show’ commands

28


VXLAN – CVX with BYOC

29

Overlay Controller

Network Layer

Controller Layer

OVSDB

Topology/Device Dependent

Traditional Approach


VXLAN – CVX with BYOC

30

OVSDB

Overlay Controller

Network Layer

Controller Layer

10x Improvement

SysDB State Sync

Topology/Device Abstraction

CloudVision Approach


VXLAN – CVX with NSX

31


VXLAN – Integration Example

32


VXLAN

•  Pros -  Industry Standard -  Scalability (from a traffic volume

perspective) -  Maturity ≫ MCAST, UNICAST

-  Widely deployed -  Intra-data center -  Inter-data center -  VLAN translation -  Supports First Hop Redundancy

Notables -  Jumbo Frames required on all

possible paths

33

•  Cons -  Lack of Control Plane (in defined

standard) -  Some interoperability

considerations ≫ MCAST v UNICAST

Documents

Arista Networks VXLAN Overvie · 2/21/2017 · Data Center Interconnect VM Mobility Naked STP EoMPLS VPLS OTV VXLAN EVPN. Conﬁdential. ... Industry Overview ... 10 GbE 40 GbE 1