MPLS in Data Center Fabrics

Alexander VirilinHead of Network Infrastructure

1

Cloud fabric

Leaf

Spine

Leaf

Spine

Leaf

Spine

Leaf

Spine

cE cE cE cE

PoD

Edge PoD

2

Label-switched path to the internet

cGW FabricSwitch cE cC cC Border

Router

IP L IP L L IPIP L IP L IP L L+ IP L

PUSH SWAP SWAP PUSH SWAP POP

+

POP

PE P P PE P PE PE

LDP/RSVP LSPBGP LU LSP

Operation

Router Type

SWAP

RRBGP BGP

3

4

IGPLDP

RSVPBGP for IPv4/IPv6BGP LU for MPLS

Fabric control plane

Network appliances

• Cloud Gateways (cGW)• Load Balancers (based on cGW)

• Route Reflectors• Route Injectors (FlowSpec, Host Routes)• Filter Nodes• Management Nodes. . .

5

Cloud “connector”

cC MPLS Backbone cC

LURR

cEcE cEcE

IGP/LDP IGP/LDP

eBGP LU Loopbacks Loopbacks eBGP LU

AZ-a AZ-b

MP-BGP

iBGPCloud Loopbacks

iBGPCloud Loopbacks

6

Backbone

PoP PoP

PoP

AZ-a AZ-b AZ-c

MPLS Backbone

7

Routing information distribution

cGW

eBGP LU

AZ PoP

Full-View

8

Fabric/Leaf

MPLS Backbone

cC

RR

LB

cC

Border

LURR

Border

LoLo

Lo

Lo Lo

LoLo

iBGP LU

eBGP LUIGP/LDP IGP/LDP

Cloud prefixes

Anycast

Regional Load BalancerBorderRouter

LB LB LB LB

AZ-a AZ-b

VM VM VM VM VM VM

Regional Load BalancerAnycast (/32, /128) VIP

ECMP

PoP

9

LSPLSP

Regional Load Balancer EnhancedBorderRouter

LB LB LB LB

AZ-a AZ-b

VM VM VM VM VM VM

Regional Load BalancerAnycast (/32, /128) VIP

ECMP

PoP

Pre-LB

Pre-LB

Pre-LB

Pre-LB

PoD

Edge PoD

10

LSPLSP

Cloud interconnect service

cGW Fabric cE cP cP Borderrouter

IP L L

PUSH SWAP SWAP SWAP POP

+

POP

PE P P PE P PE PE

LDP/RSVP LSPBGP LU LSP

Custon-

premise

L IP LL IP LL IP LL LIP LL IP LL IP L IP

L

L L

Service label

Transport labels

VPNRRMP-BGP MP-BGP

SWAPPUSH

11

Bare metal case

Leaf/PE cGW

Borderrouter

Custon-

premise

Leaf/PE

VPNRR

VM VM VM

Fabric

OverlayNetwork

LSPMP-BGP

MP-BGP

BGP

UnderlayNetwork

MP-BGP

LSP

12

LSP

Point of delivery

13

14

Eat your own dogfood

Leaf switch

Compute node

Route Reflector

Cloud Gateway

Compute node

Management host

15

Leaf switch

Compute node

Route Reflector

Filter Node

Compute node

Leaf switch

Compute node

Route Reflector

Compute node

Load Balancer

Rack layout

AZ-c

Compute node Compute node

… … …

AZ-a AZ-b

x3

Implicit-null

16

Explicit-null

17

Conclusion

“It’s much harder (or rather more expensive) to obtain MPLS support in enterprise

or data center class equipment, as MPLS is seen as a “transit service provider

solution.” Sometimes our own preconceptions can cause us to choose solutions

based on what they’re “meant for,” rather than what they’re really useful for; in the end, this can make our networks more, rather than less, complex.” –

Navigating Network Complexity

by Russ White, Jeff (Evgeny) Tantsura

18

Thank you! Questions?Alexander VirilinHead of Network InfrastructureYandex Cloud📧 axscrew@yandex-team.ru

19