Deploying MPLS Traffic Engineering - Cisco€¦ · Introduction • MPLS-TE was designed to move traffic along a path other than the IGP shortest path Bring ATM/FR traffic engineering

1© 2002, Cisco Systems, Inc. All rights reserved.

RST-2515465_05_2002_c1

Deploying MPLS Traffic Engineering

James Moffat

Consulting Systems Engineer

[email protected]


RST-2515465_05_2002_c1

Agenda

• Prerequisites

• Introduction

• How MPLS-TE Works

• Fast ReRoute

• Design


RST-2515465_05_2002_c1

Prerequisites

• Must know how to configure a router!

…Preferably Cisco…

• Basic knowledge of MPLS forwarding

push, pop, swap, etc.

• Some exposure to MPLS-TE helps

Not much time spent on basic configs


RST-2515465_05_2002_c1

Agenda

• Prerequisites

• Introduction


• Fast ReRoute

• Design


RST-2515465_05_2002_c1

MPLS Is Key technology for delivery of L2& L3 services

IPServices

IPServices

ATMServices

ATMServices

IP+ATM SwitchIP+ATM Switch

PNNIPNNI MPLSMPLS

IPIP

IPServices

IPServices

OpticalServicesOptical

Services

IP+Optical SwitchIP+Optical Switch

O-UNIO-UNI MPLSMPLS

IPIP

FrameRelay

ATM

FrameRelay

IP+ATM Integration

MPLS VPNs: Scalable Network based VPNs

Traffic Engineering: Optimization forAdditional traffic =>$$

Protection solutionReduction in CAPEX & OPEX

IP+Optical Integration

Layer 2 Integration forA single convergedNetwork Infrastructure


RST-2515465_05_2002_c1

Introduction

• MPLS-TE was designed to move traffic along a path other than the IGP shortest path

Bring ATM/FR traffic engineering abilities to an IP networkAvoid full IGP mesh and O(N2) floodingBandwidth-aware connection setup

• Fast ReRoute (FRR) is emerging as another application of MPLS-TE

O(msec) of packet loss when a link goes downReplace expensive SONET gear with routersCan be used in conjunction with MPLS-TE for primary paths, can also be used standalone

• Diffserv Aware Traffic EngineeringDelivering strict QOS guarantees through the integration of MPLS-TE and advanced QOS techniques


RST-2515465_05_2002_c1

Router F

The Problem with Shortest-Path

• Changing to A->C->D->E won’t help

Router C Router D

Router G

80Mb Traffic

80Mb Traffic

35Mb Drops!

35Mb Drops!Router A

Router B

NodeNode Next-HopNext-Hop CostCostBB 1010BB

FF 3030BB

CC 1010CCDD 2020CCEE 2020BB

GG 3030BB

OC-3

OC-3

DS3

DS3

DS3OC-3

OC-3

• Some links are DS3, some are OC-3

• Router A has 40Mb of traffic for Route F, 40Mb of traffic for Router G

• Massive (44%) packet loss at Router B->Router E!

Router E


RST-2515465_05_2002_c1

Router F

40Mb40Mb

What MPLS-TE Address

• Router A sees all links

• Router A computes paths on properties other than just shortest cost

• No link oversubscribed!

OC-3

OC-3

DS3

DS3

DS3OC-3

Router C

Router E

Router D

Router G

Router A

Router B

40Mb40Mb


F 30Tunnel 0


G 30Tunnel 1

OC-3


RST-2515465_05_2002_c1

Fast ReRoute

• FRR: A mechanism to minimize packet loss during a failure

• Pre-provision protection tunnels that carry traffic when a protected resource (link/node) goes down

• Use MPLS-TE to signal the FRR protection tunnels, taking advantage of the fact that MPLS-TE traffic doesn’t have to follow the IGP shortest path

• Can protect MPLS traffic or IP traffic, depends on the type of protection

• See later slides on FRR


RST-2515465_05_2002_c1

Agenda

• Prerequisites

• Introduction


• Fast ReRoute

• Design


RST-2515465_05_2002_c1

How MPLS-TE Works

• Information distribution

• Path calculation

• Path setup

• Forwarding traffic down tunnels


RST-2515465_05_2002_c1

Information Distribution

• Need to tell the network about per-link resources (mostly available bandwidth)

• This is done using extensions to IGP (OSPF, ISIS)

• EIGRP, RIP not supported for MPLS-TE

EIGRP, RIP will work for other MPLS applications (like VPNs!), just not for TE.


RST-2515465_05_2002_c1


• OSPFUses type 10 (opaque area—local) lSAs

See draft-katz-yeung-ospf-traffic

router ospf 1mpls traffic-eng area <x>mpls traffic-eng router-id Loopback0


RST-2515465_05_2002_c1


• IS-ISUses Type 22 TLVs

See draft-ietf-isis-traffic

router isis foompls traffic-eng level-1|level-2mpls traffic-eng router-id Loopback0metric-style wide


RST-2515465_05_2002_c1

Path Calculation

• Modified Dijkstra at tunnel head-end

• Often referred to as CSPF

Constrained SPF

• …or PCALC (path calculation)


RST-2515465_05_2002_c1

Path Calculation

• What if there’s more than one path that meets the minimum requirements (bandwidth, etc.)?

• PCALC algorithm:Find all paths with the lowest IGP cost

Then pick the path with the highest minimum bandwidth along the path

Then pick the path with the lowest hop count (not IGP cost, but hop count)

Then just pick one path at random


RST-2515465_05_2002_c1

Path Calculation

“What’s the Shortest Path to All Routers?”

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

• Normal SPF—Find shortest path across all links

• See Perlman (2nd ed), Moy, etc. for explanation of SPF


RST-2515465_05_2002_c1

Path Calculation

RtrA





RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC RtrD


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC

RtrE

RtrD


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation




RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

• Constrained SPF—Find shortest path to a specific node

• Consider more than just link cost!

OC3

OC3

DS3

DS3

DS3

OC3

OC3

“What’s the Shortest Path to Router F with 40Mb Available?”

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

RtrA





RST-2515465_05_2002_c1

Path Calculation

OC3

OC3




RtrA

RtrB

RtrC


RST-2515465_05_2002_c1

Path Calculation

OC3

OC3

DS3




RtrA

RtrB

RtrC RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

OC3

DS3

DS3




RtrA

RtrB

RtrC

RtrE

RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

OC3

DS3

DS3

OC3

OC3




RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

OC3

OC3

DS3

DS3

OC3

OC3





RST-2515465_05_2002_c1

Path Calculation

OC3

DS3OC3




RtrA

RtrB

RtrE

RtrF


RST-2515465_05_2002_c1

Path Calculation

• “But wait! There’s nothing different between the two SPF results!”

• …but…


RST-2515465_05_2002_c1

Path Calculation

• What about the2nd path?

• Available bandwidth has changed!

OC3

5MB

DS3

DS3OC3

115MB115MB

“What’s the Shortest Path to Router G with 40Mb Available?”

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

RtrA





RST-2515465_05_2002_c1

Path Calculation

OC3

115MB




RtrA

RtrB

RtrC


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3

115MB




RtrA

RtrB

RtrC RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

5MB

DS3

115MB




RtrA

RtrB

RtrC

RtrE

RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

5MB

DS3

115MB




RtrA

RtrB

RtrC

RtrE

RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3

115MB




RtrA

RtrB

RtrC RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3




RtrA

RtrC RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3

DS3




RtrA

RtrC

RtrE

RtrD


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3

DS3OC3

115MB




RtrA

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

OC3

DS3

DS3OC3




RtrA

RtrC

RtrE

RtrD

RtrG


RST-2515465_05_2002_c1

Path Calculation

• End result:Bandwidth used efficiently!

30Tunnel1G

30Tunnel0F

20BE

20CD

10CC

10BB

CostNext-HopNode

OC3

OC3

DS3

DS3

DS3

OC3

OC3

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


RST-2515465_05_2002_c1

Path Calculation

• What if there’s more than one path that meets the minimum requirements (bandwidth, etc.)?

• PCALC algorithm:Find all paths with the lowest IGP costThen pick the path with the highest minimum bandwidth along the pathThen pick the path with the lowest hop count (not IGP cost, but hop count)Then just pick one path at “random” (take the top path on the TENT list)


RST-2515465_05_2002_c1

Path Calculation

All Left-side LinksAre {10,100M}

All Right-side LinksAre {5,150M}

{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}

{10,100M}

{8,80M}

What’s the BestPath from A to Z with BW of 20M?

Path Has Cost of 25, Not the

Lowest Cost!


RST-2515465_05_2002_c1

Path Calculation

{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}

{8,80M}

Path Min BW Is Lower than the

Other Paths!





RST-2515465_05_2002_c1

Path Calculation

{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}

Hop Count Is 5, Other Paths

Are 4!





RST-2515465_05_2002_c1

Path Calculation

{cost,available BW}

RtrA RtrZ

{8,90M}

Pick a Path at Random!

{8,90M}All Left-side LinksAre {10,100M}




RST-2515465_05_2002_c1

Path Calculation

{cost,available BW}

RtrA RtrZ

{8,90M}





RST-2515465_05_2002_c1

Path Setup

• Cisco MPLS-TE uses RSVP• RFC2205 (base RSVP), RFC 3209 (TE extensions

for RSVP)• No CR-LDP; no plans for CR-LDP

• Once the path is calculated, it is handed to RSVP• RSVP uses PATH and RESV messages to request

an LSP along the calculated path


RST-2515465_05_2002_c1

Router F

Path Setup

• PATH message: “Can I have 40Mb along this path?”

• RESV message: “Yes, and here’s the label to use”

• LFIB is set up along each hop

Router B

Router C

Router E

Router D

Router G

Router A

= PATH Messages= RESV Messages


RST-2515465_05_2002_c1

Router F

Path Setup

• PATH message: “Can I have 40Mb along this path?”

• RESV message: “Yes, and here’s the label to use”

• LFIB is set up along each hop

Router B

Router C

Router E

Router D

Router G

Router A

= PATH Messages= RESV Messages

IMPLICITNULL23

39

15


RST-2515465_05_2002_c1

Path Setup

• Errors along the way will trigger RSVP errors

• May also trigger re-flooding of TE information if appropriate


RST-2515465_05_2002_c1

Forwarding Traffic Down a Tunnel

• There are four ways traffic can be forwarded down a TE tunnel

Static routes

Policy routing

Auto-route

Forwarding-adjacency

• With all but PBR, MPLS-TE gets you unequal cost load balancing


RST-2515465_05_2002_c1

Static Routing

RtrA(config)#ip route H.H.H.H 255.255.255.255 Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router 1

Tunnel1


RST-2515465_05_2002_c1

Tunnel1

Static Routing

• Router H is known via the tunnel

• Router G is not routed to over the tunnel, even though it’s the tunnel tail!

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router 1


FF 3030BB


GG 3030BBHH 4040Tunnel 1Tunnel 1II 4040BB


RST-2515465_05_2002_c1

Policy Routing

RtrA(config-if)#ip policy route-map set-tunnel

RtrA(config)#route-map set-tunnel

RtrA(config-route-map)#match ip address 101

RtrA(config-route-map)#set interface Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router 1

Tunnel1


RST-2515465_05_2002_c1

Policy Routing

• Routing table isn’t affected by policy routing

• Need (12.0(16)ST or 12.2T) or higher for ‘set interface tunnel’ to work (CSCdp54178)

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router 1


FF 3030BB


GG 3030BBHH 4040BBII 4040BB

Tunnel1Tunnel1


RST-2515465_05_2002_c1

Auto-Route

• Auto-route = “Use the tunnel as a directly connected link for SPF purposes”

• This is not the CSPF (for path determination), but the regular IGP SPF (route determination)

• Behavior is intuitive, operation can be confusing


RST-2515465_05_2002_c1

Auto-Route

This Is the Physical Topology

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


RST-2515465_05_2002_c1

Auto-Route

• This is Router A’s logical topology

• By default, other routers don’t see the tunnel!

Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


RST-2515465_05_2002_c1

Auto-Route

• Router A’s routing table, built via auto-route

• Everything “behind” the tunnel is routed via the tunnel

Tunnel1

Router FRouter H

Router B

Router C

Router E

Router D

Router G

Router A

Router I


FF 3030BB


GG 3030Tunnel 1Tunnel 1HH 4040Tunnel 1Tunnel 1II 4040Tunnel 1Tunnel 1


RST-2515465_05_2002_c1

Auto-Route

• How does autoroute avoid loops?

Process SPF with physical links, replaceOIF with tunnel for tail and all subsequent neighbors


RST-2515465_05_2002_c1

Forwarding Adjacency

• Autoroute metric change is purely local to the headend

• This makes MPLS TE different from TE with ATM

In ATM TE, the TE link (PVC) has its cost and neighbor advertised into the network

In MPLS TE, no such thing is done—Until FA


RST-2515465_05_2002_c1

ATM Model

• Cost of ATM links (blue) is unknown to routers• A sees two links in IGP—E->H and B->D• A can load-share between B and E

A I

E

BC

D

F GH


RST-2515465_05_2002_c1

Before FA

• All links have cost of 10

• A’s shortest path to I is A->B->C->D->I

• A doesn’t see TE tunnels on {E,B}, alternate path never gets used!

• Changing link costs is undesirable, can have strangeadverse effects

A I

E

B C D

F GH


RST-2515465_05_2002_c1

F-A Advertises TE Tunnels in the IGP

• With forwarding-adjacency, A can see the TE tunnels as links

• A can then send traffic across both paths

• This is desirable in some topologies (looks just like ATM did, same methodologies can be applied)

A I

E

B C D

F GH


RST-2515465_05_2002_c1

F-A Issues

• In order for A to use F-A links, they need to be the best cost IGP path

Otherwise the physical topo gets used

• F-A configured withtunnel mpls traffic-eng forwarding-adjacencyisis metric <x> level-<y>


RST-2515465_05_2002_c1

F-A Issues

• Only ISIS supports F-A

OSPF support coming RSN

• F-A must be bidirectional

• IGP adjacency still not run over TE tunnel

• F-A cost should probably be lower than lowest possible IGP path from head to tail, otherwise it might not always get used


RST-2515465_05_2002_c1

Unequal Cost Load Balancing

• IP routing has equal-cost load balancing, but not unequal cost*

• Unequal cost load balancing difficult to do while guaranteeing a loop-free topology

*EIGRP Has ‘Variance’, but That’s Not as Flexible


RST-2515465_05_2002_c1

Unequal Cost Load Balancing

• Since MPLS doesn’t forward based on IP header, permanent IGP routing loops don’t happen with unequal cost

• 16 hash buckets for next-hop, shared in rough proportion to configured tunnel bandwidth or load-share value


RST-2515465_05_2002_c1

Unequal Cost: Example 1

Router A Router E

Router F

Router G

gsr1#show ip route 192.168.1.8Routing entry for 192.168.1.8/32

Known via "isis", distance 115, metric 83, type level-2Redistributing via isisLast update from 192.168.1.8 on Tunnel0, 00:00:21 agoRouting Descriptor Blocks:* 192.168.1.8, from 192.168.1.8, via Tunnel0

Route metric is 83, traffic share count is 2192.168.1.8, from 192.168.1.8, via Tunnel1

Route metric is 83, traffic share count is 1

40MB

20MB


RST-2515465_05_2002_c1


Note That the Load Distribution Is 11:5—Very Close to 2:1, but Not Quite!

gsr1#sh ip cef 192.168.1.8 internal………Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1)Hash OK Interface Address Packets Tags imposed1 Y Tunnel0 point2point 0 {23}2 Y Tunnel1 point2point 0 {34}

………

Router A 40MB

20MBRouter G

Router E

Router F


RST-2515465_05_2002_c1


Q: How Does 100:10:1 Fit into a 16-Deep Hash?

gsr1#sh ip rou 192.168.1.8Routing entry for 192.168.1.8/32Known via "isis", distance 115, metric 83, type level-2Redistributing via isisLast update from 192.168.1.8 on Tunnel2, 00:00:08 agoRouting Descriptor Blocks:* 192.168.1.8, from 192.168.1.8, via Tunnel0

Route metric is 83, traffic share count is 100192.168.1.8, from 192.168.1.8, via Tunnel1Route metric is 83, traffic share count is 10

192.168.1.8, from 192.168.1.8, via Tunnel2Route metric is 83, traffic share count is 1

100MB10MB1MB

Router A

Router G

Router E

Router F


RST-2515465_05_2002_c1


A: Any Way It Wants to! 15:1, 14:2, 13:2:1, It Depends on the Order the Tunnels Come Up

Deployment Guideline: Don’t Use Tunnel Metrics That Don’t Reduce to 16 Buckets!

gsr1#sh ip cef 192.168.1.8 internal………Load distribution: 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (refcount 1)

Hash OK Interface Address Packets Tags imposed1 Y Tunnel0 point2point 0 {36}2 Y Tunnel1 point2point 0 {37}

………

100MB10MB1MB

Router A

Router G

Router E

Router F


RST-2515465_05_2002_c1

Forwarding Traffic down a Tunnel

• You can use any combination of auto-route, forwarding-adjacency, static routes, or PBR

• …but simple is better unless you have a good reason

• Recommendation: autoroute, forwarding-adjacency, or statics to BGP next-hops, depending on your needs


RST-2515465_05_2002_c1

Agenda

• Prerequisites

• Introduction


• Fast ReRoute

• Design


RST-2515465_05_2002_c1

Fast ReRoute

• FRR: A mechanism to minimize packet loss during a failure

• Pre-provision protection tunnels that carrytraffic when a protected resource (link/node) goes down

• Use MPLS-TE to signal the FRR protection tunnels, taking advantage of the fact thatMPLS-TE traffic doesn’t have to follow the IGP shortest path

• Can protect MPLS traffic or IP traffic, depends on the type of protection


RST-2515465_05_2002_c1

Fast Reroute

• In an IP network, a link failure causes several seconds of outage

Link Failure DetectionLink Failure Detection

Information PropagationInformation Propagation

Route RecalculationRoute Recalculation

ThingThing

IGP Timers, NetworkSize, Collective

Router Load


Router Load

Media- and Platform-specific

Media- and Platform-specific ~µsecs (POS + APS)~µsecs (POS + APS)

~5–30 sec~5–30 sec

LSDB Size, CPU Load LSDB Size, CPU Load ~1–3 sec~1–3 sec

DependencyDependency TimeTime


RST-2515465_05_2002_c1

Fast Reroute

• In an MPLS network, there’s more work to be done, so a (slightly) longer outage happens


Route RecalculationRoute Recalculation

ThingThing

~Usecs (POS + APS)~Usecs (POS + APS)

~5–30 sec~5–30 sec

LSDB Size, CPU Load LSDB Size, CPU Load ~1–3 sec~1–3 sec


New LSP SetupNew LSP SetupNetwork Size,

CPU Load Network Size,

CPU Load ~5–10 sec~5–10 sec



Router Load


Router Load




RST-2515465_05_2002_c1

Three Kinds of Fast Reroute

• Link protection

Implemented today

• Node protection

Implemented today

• Path protection

On development radar


RST-2515465_05_2002_c1

Link Protection

• TE Tunnel A -> B -> D -> E

Router DRouter B

Router C

Router ERouter A10 34 POP


RST-2515465_05_2002_c1

Link Protection

• B has a pre-provisioned backup tunnel to the other end of the protected link (Router D)

• B relies on the fact that D is using globallabel space

Router D

Router C

Router A Router B Router E10 34 POP

27 POP

PLR MPNHOP

NHOPBackupTunnel

ProtectedLink


RST-2515465_05_2002_c1

Link Protection

• B -> D link fails, A -> E tunnel is encapsulated in B -> D tunnel

• Backup tunnel is used until A can re-compute tunnel path as A -> B -> C -> D -> E (10–30 seconds or so)

Router C

Router DRouter A Router B Router E10 POP

27, 34 34


RST-2515465_05_2002_c1

Link Protection

• On tunnel head-end: tunnel mpls traffic-eng fast-reroute

• On protected link:mpls traffic-eng backup-path <backup-tunnel>

Router DRouter B Router ERouter ERouter A


RST-2515465_05_2002_c1

Node Protection

• Router A has a tunnel A -> B -> D -> E -> F

• Router B has a protect tunnel B -> C -> E -> D

Router D Router FRouter B Router ERouter A


RST-2515465_05_2002_c1

Node Protection

• Link protection is OK if the B -> D link goes down

• What if Router D goes away?



RST-2515465_05_2002_c1

Node Protection

• Solution: Protect tunnel to the hop past the protected link

Router D Router FRouter B Router ERouter A10 34 POP

27 POP

22

PLR MPNNHOP

NNHOPBackupTunnel

NHOP

ProtectedNode


RST-2515465_05_2002_c1

Node Protection

• Node protection still has the same convergence properties as link protection

• Deciding where to place your backup tunnels is a much harder to problem to solve large-scale

• For small-scale protection, link may be better

• Cisco has developed tools to solve these hard problems for you (Tunnel Builder Pro)


RST-2515465_05_2002_c1

Path Protection

• Path protection: Multiple tunnels from TE head to tail, across diverse paths



RST-2515465_05_2002_c1

Path Protection

• Path protection: Least scalable, most resource-consuming, slowest convergence of all 3 protection schemes

• Path protection is useful in two places:

1. When you have more links than tunnels

2. When you need to protect links not using global label space


RST-2515465_05_2002_c1

Path Protection

Head-end Switch-overto Protect LSP

Head-end Switch-overto Protect LSP

Network Size, CPU Load

Network Size, CPU Load

~Msec~Msec

Path vs. Local Protection

Local (Link/Node) Protection


Local Switch-over toProtect Tunnel


ThingThing

RP-> Communication Time



Media- and Platform-specific ~Usecs (POS + APS)~Usecs (POS + APS)

~Few msec or Less~Few msec or Less



ThingThing






Router Load


Router Load~5–30+ sec~5–30+ sec


RST-2515465_05_2002_c1

APS Protection

IGP Reconvergeson New Link

IGP Reconvergeson New Link

IGP Timers,IGP Size, CPU Load, Etc.

IGP Timers,IGP Size, CPU Load, Etc.

Seconds or LessSeconds or Less

Local Protection vs. APS Protection

Local (Link/Node) Protection




ThingThing





~Few msec or Less~Few msec or Less



ThingThing

Media-and Platform-specific

Media-and Platform-specific ~Usecs (POS + APS)~Usecs (POS + APS)


APS/MSP CutoverAPS/MSP Cutover Generally a Fixed TimeGenerally a Fixed Time <50ms, per spec<50ms, per spec


RST-2515465_05_2002_c1

Agenda

• Prerequisites

• Introduction


• Fast ReRoute

• Design


RST-2515465_05_2002_c1

Design

• Two ways to deploy MPLS-TE

As needed to clear up congestion—Tactical

Full mesh between a set of routers—Strategic

• Strategic can be online or offline path calculation

• Both methods are valid, both have their pros and cons


RST-2515465_05_2002_c1

Tactical

• All links are OC12

• A has consistent ±700MB to send to C

• ~100MB constantly dropped!

Case Study: A Large US ISP

Router A

Router B

Router D Router E

Router C


RST-2515465_05_2002_c1

Tactical

• Solution: Multiple tunnels, unequal cost load sharing!

• Tunnels with bandwidth in 3:1 (12:4) ratio

• 25% of traffic sent the long way

• 75% sent the short way

• No out-of-order packet issues—CEF’s normal per-flow hashing is used!

Router A

Router B

Router D Router E

Router C


RST-2515465_05_2002_c1

Tactical

• From Router A’s perspective, topology is:

Router A

Router B

Router D Router E

Router C


RST-2515465_05_2002_c1

Tactical

• As needed—Easy, quick, but hard to track over time

• Easy to forget why a tunnel is in place

• Inter-node BW requirements may change, tunnels may be working around issues that no longer exist

• Link protection pretty straightforward, node protection harder to track


RST-2515465_05_2002_c1

Strategic

• Put a full mesh of TE tunnels between routers

• Initially deploy tunnels with 0 bandwidth

• Watch tunnel interface statistics, see how much bandwidth you are using between router pairs

Tunnels are interfaces—Use IF-MIB!

Make sure that Σtunnel <= Σnetwork BW


RST-2515465_05_2002_c1

Strategic

• Some folks deploy full mesh just to get router-to-router (pop-to-pop) traffic matrix

• Largest TE network ~80 routers full mesh (~6400 tunnels)

• As tunnel bandwidth is changed, tunnels will find the best path across your network


RST-2515465_05_2002_c1

Strategic

• Physical topology is:

Router A

Router B

Router D Router E

Router C


RST-2515465_05_2002_c1

Strategic• Logical topology is*

*Each link is actually 2 unidirectional tunnels

• Total of 20 tunnels in this networkRouter A

Router B

Router D Router E

Router C


RST-2515465_05_2002_c1

Strategic

• Things to remember with full mesh

N routers, N*(N-1) tunnels

Routing protocols not run over TE tunnels—Unlike an ATM/FR full mesh!

Tunnels are unidirectional—This is a good thing

…Can have different bandwidth reservations in two different directions


RST-2515465_05_2002_c1

Strategic

• Best practices for full mesh:

Periodically re-optimize tunnels based on need (just like an ATM network)

TE was designed to be a combination of online (router-based) and offline (NMS) calculation

Node protection more practical in a full-mesh, offline-generate TE topography


RST-2515465_05_2002_c1

Recommended Reading

• Traffic Engineering with MPLS

ISBN: 1-58705-031-5


RST-2515465_05_2002_c1


RST-2515465_05_2002_c1

Scalability

• Tests were done on a GSR

• RSP4, RSP8, VXR300, VXR400 will be similar

• 10,000 tunnels come up in 3-5 minutes

How Many Tunnels on a Router?

Number of Head-End

Tunnels

Number of Head-End

Tunnels

Number ofTunnel TailsNumber of

Tunnel TailsNumber

of Mid-PointsNumber

of Mid-PointsCodeCode

12.0ST12.0ST 600600 10,00010,000 5,0005,000


RST-2515465_05_2002_c1

Scalability

• Largest TE network today = 80 routers, ~6400 tunnels full mesh

• 12.0ST—600 head-ends, 360,000 tunnels full mesh with 10,000 tunnels per midpoint

• 600 = 80*7.50

Or (360,000=6400*56) if you’re in marketing

• Bottom line: MPLS-TE is not a gating factor in scaling most networks!


RST-2515465_05_2002_c1

Scalability

• Or just search CCO for “Scalability Enhancements for MPLS Traffic Engineering”

http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120st/120st14/scalable.htm

Documents

Deploying MPLS Traffic Engineering - Cisco€¦ · Introduction • MPLS-TE was designed to move traffic along a path other than the IGP shortest path Bring ATM/FR traffic engineering