1 © 2001, Cisco Systems, Inc. Course Number Presentation_ID MPLS TE TOI [email protected]

1© 2001, Cisco Systems, Inc.

Course NumberPresentation_ID

MPLS TE TOIMPLS TE TOI

[email protected]@cisco.com

2Presentation_ID © 2001, Cisco Systems, Inc.

AgendaAgenda

• How MPLS TE works

• What Code Is MPLS TE In?

• Platform Issues in Implementation

• Lab Demo - config


How MPLS TE WorksHow MPLS TE Works

• Prerequisites

• How MPLS-TE Works

• Basic Configuration

• Knobs! Knobs! Knobs!

• Deploying and Designing


PrerequisitesPrerequisites

You should already understand…

• How to configure a Cisco router

• Basic MPLS concepts like push/pop/swap, EXP, and LFIB

• How a link-state routing protocol works

• Basic QoS mechanisms like MDRR and LLQ


AgendaAgenda

• Prerequisites




• Deploying and Desiginig


How MPLS-TE WorksHow MPLS-TE Works


-What good is MPLS-TE?

-Information Distribution

-Path Calculation

-Path Setup

-Forwarding Traffic Down A Tunnel


What Good Is MPLS-TE?What Good Is MPLS-TE?

• There are two kinds of networks

1. Those that have plenty of bandwidth everywhere

2. Those with congestion in some places, but not in others

• The first kind always evolve into the second kind!



• MPLS-TE introduces a 3rd kind:1. Those that have plenty of bandwidth everywhere

2. Those with congestion in some places, but not in others

3. Those that use all of their bandwidth to its maximum efficiency, regardless of shortest-path routing!



MultiProtocolLabelSwitching -TrafficEngineering

MagicProblem-solvingLaborSubstitute which isTotallyEffortless

This stuff takes work, but it’s worth it!!!

What is MPLS-TE? What is it not?


Information DistributionInformation Distribution

• You need a link-state protocol as your IGP

IS-IS or OSPF

• Link-state requirement is only for MPLS-TE!

Not a requirement for VPNs, etc!


Need for a Link-State ProtocolNeed for a Link-State Protocol

• Why do I need a link-state protocol?

1. To make sure info gets flooded

2. To build a picture of the entire network


Need for a Link-State ProtocolNeed for a Link-State Protocol

Consider the following network:

- All links have a cost of 10

- RtrA’s path to RtrE is A->B->E, cost 20

- All traffic from A to {E,F,G} goes A->B->E

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


What a DV Protocol SeesWhat a DV Protocol Sees

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G B 30

• RtrA doesn’t see all the links

• RtrA only knows about the shortest path

• This is by design

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


What a LS Protocol SeesWhat a LS Protocol Sees

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G B 30

• RtrA sees all links

• RtrA only computes the shortest path

• Routing table doesn’t change

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


The Problem With Shortest-PathThe Problem With Shortest-Path

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G B 30

• Some links are DS3, some are OC3

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

OC3

OC3

DS3

DS3

DS3

OC3

OC3

• RtrA has 40Mb of traffic for RtrF, 40Mb of traffic for RtrG

• Massive (44%) packet loss at RtrB->RtrE!

• Changing to A->C->D->E won’t help


What MPLS-TE AddrsWhat MPLS-TE Addrs

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F Tunnel0 30

G Tunnel1 30

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

OC3

OC3

DS3

DS3

DS3

OC3

OC3

• RtrA sees all links

• RtrA computes paths on properties other than just shortest cost

• No congestion!






-Path Calculation

-Path Setup




• OSPF

-Uses Type 10 (Opaque Area-Local) LSAs

-See draft-katz-yeung-ospf-traffic



• IS-IS

-Uses Type 22 TLVs

-See draft-ietf-isis-traffic



• IS-IS and OSPF propagate the same information!

-Link identification

-TE Metric

-Bandwidth info (max physical, max reservable, available per-class)

-Attribute flags



• TE flooding is local to a single {area|level}

• Inter-{area|level} TE harder, but possible (think PNNI)






-Path Calculation

-Path Setup



Path CalculationPath Calculation

• Modified Dijkstra at tunnel head-end

• Often referred to as CSPF

Constrained SPF

• …or PCALC (path calculation)



• Normal SPF – find shortest path across all links

• See Perlman (2nd ed), Moy, etc. for explanation of SPF

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

“what’s the shortest path to all routers?”





RtrA






RtrA

RtrB

RtrC






RtrA

RtrB

RtrC RtrD






RtrA

RtrB

RtrC

RtrE

RtrD






RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG






RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG






RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG






RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG




• Constrained SPF – find shortest path to a specific node

• Consider more than just link cost!

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

“what’s the shortest path to router F with 40Mb available??”

OC3

OC3

DS3

DS3

DS3

OC3

OC3





RtrA






RtrA

RtrB

RtrC


OC3

OC3





RtrA

RtrB

RtrC RtrD


OC3

OC3

DS3





RtrA

RtrB

RtrC

RtrE

RtrD


OC3

OC3

DS3

DS3





RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


OC3

OC3

DS3

DS3

OC3

OC3





RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


OC3

OC3

DS3

DS3

DS3

OC3

OC3





RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


OC3

OC3

DS3

DS3

DS3

OC3

OC3





RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG


OC3

OC3

DS3

DS3

OC3

OC3





RtrA

RtrB

RtrC

RtrE

RtrD

RtrF


OC3

OC3

DS3

DS3

OC3





RtrA

RtrB

RtrE

RtrF


OC3

DS3OC3



• “But Wait! There’s nothing different between the two SPF results!”

• ….but….



• What about the 2nd path?

• Available bandwidth has changed!

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

“what’s the shortest path to router G with 40Mb available??”

OC3

OC3

5MB

DS3

DS3

OC3

OC3





RtrA






RtrA

RtrB

RtrC


OC3

OC3





RtrA

RtrB

RtrC RtrD


OC3

OC3

DS3





RtrA

RtrB

RtrC

RtrE

RtrD


OC3

OC3

5MB

DS3





RtrA

RtrB

RtrC

RtrE

RtrD


OC3

OC3

5MB

DS3





RtrA

RtrB

RtrC RtrD


OC3

OC3

DS3





RtrA

RtrC RtrD


OC3

DS3





RtrA

RtrC

RtrE

RtrD


OC3

DS3

DS3





RtrA

RtrC

RtrE

RtrD

RtrF

RtrG


OC3

DS3

DS3

OC3

OC3





RtrA

RtrC

RtrE

RtrD

RtrG


OC3

DS3

DS3OC3



Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F Tunnel0 30

G Tunnel1 30

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

OC3

OC3

DS3

DS3

DS3

OC3

OC3

• End result:

-bandwidth used efficiently!



• Happy! Happy!

• Joy! Joy!



• What if there’s more than one path that meets the minimum requirements (BW, etc)?

• PCALC algorithm:

1. find all paths with the lowest IGP cost

2. then pick the path with the highest minimum bandwidth along the path

3. then pick the path with the lowest hop count (not IGP cost, just hop count)

4. then just pick one path at random



all left-side linksare {10,100M}

all right-side linksare {5,50M}

{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}

{10,100M}

{8,80M}

What’s the bestpath from A to Z with BW of 20M?

Path has cost of 25, not the

lowest cost!





{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}

{8,80M}


Path min BW is lower than the other paths!





{cost,available BW}

RtrA RtrZ

{8,90M}

{8,90M}

{4,90M}


Hop count is 5, other paths are

4!





{cost,available BW}

RtrA RtrZ

{8,90M}


Pick a path at random!

{8,90M}





{cost,available BW}

RtrA RtrZ

{8,90M}







-Path Calculation

-Path Setup



Path SetupPath Setup

• Cisco MPLS-TE uses RSVP

• RFC2205, plus draft-ietf-mpls-rsvp-lsp-tunnel



• Once the path is calculated, it is handed to RSVP

• RSVP uses PATH and RESV messages to request an LSP along the calculated path



• PATH message: “Can I have 40Mb along this path?”

• RESV message: “Yes, and here’s the label to use.”

• LFIB is set up along each hop

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

= PATH messages

= RESV messages



• Errors along the way will trigger RSVP errors

• May also trigger re-flooding of TE info if appropriate






-Path Calculation

-Path Setup



Forwarding Traffic Down a TunnelForwarding Traffic Down a Tunnel

• There are three ways traffic can be forwarded down a TE tunnel

-Autoroute

-Static routes

-Policy routing

• For the first two, MPLS-TE gets you unequal-cost load-balancing.


AutorouteAutoroute

• Autoroute = “use the tunnel as a directly connected link for SPF purposes”

• This is not the CSPF (for path determination), but the regular IGP SPF (route determination)

• Behavior is intuitive, operation can be confusing


AutorouteAutoroute

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

This is the physical topology

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrI


AutorouteAutoroute

This is RtrA’s logical topology

Other routers don’t see the tunnel!

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

Tunnel1

RtrI


AutorouteAutoroute

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

Tunnel1

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G Tunnel1 30

H Tunnel1 40

I Tunnel1 40

Router A’s routing table, built via autoroute.

Everything “behind” the tunnel is routed via the tunnel.

RtrI


Static routingStatic routing

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

RtrA(config)#ip route H.H.H.H 255.255.255.255 Tunnel1

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

RtrI

Tunnel1


Static routingStatic routing

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrHRtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

RtrI

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G B 30

H Tunnel1 40

I B 40

RtrH is known via the tunnel.

RtrG is not routed to over the tunnel, even though it’s the tunnel tail!

Tunnel1


Unequal-Cost Load BalancingUnequal-Cost Load Balancing

• IP routing has equal-cost load-balancing, but not unequal-cost*

• Unequal-cost load balancing difficult to do while guaranteeing a loop-free topology

*EIGRP has ‘variance’, but that’s not as flexible, and besides, MPLS-TE and EIGRP are two different things


Unequal-Cost Load BalancingUnequal-Cost Load Balancing

• Since MPLS doesn’t forward based on IP header, permanent routing loops don’t happen.

• 16 hash buckets for next-hop, shared in rough proportion to tunnel BW


Unequal-cost, Example 1Unequal-cost, Example 1

RtrA RtrE

RtrF

RtrG

40MB

20MB

gsr1#show ip route 192.168.1.8Routing entry for 192.168.1.8/32 Known via "isis", distance 115, metric 83, type level-2 Redistributing via isis Last update from 192.168.1.8 on Tunnel0, 00:00:21 ago Routing Descriptor Blocks: * 192.168.1.8, from 192.168.1.8, via Tunnel0 Route metric is 83, traffic share count is 2 192.168.1.8, from 192.168.1.8, via Tunnel1 Route metric is 83, traffic share count is 1



RtrA RtrE

RtrF

RtrG

40MB

20MB

Note that the load distribution is 11:5 – very close to 2:1, but not quite!

gsr1#sh ip cef 192.168.1.8 int………Load distribution: 0 1 0 1 0 1 0 1 0 1 0 0 0 0 0 0 (refcount 1) Hash OK Interface Address Packets Tags imposed

1 Y Tunnel0 point2point 0 {23} 2 Y Tunnel1 point2point 0 {34}………



RtrA RtrE

RtrF

RtrG

100MB10MB

Q:How does 100:10:1 fit into a 16-deep bucket?

1MB

gsr1#sh ip rou 192.168.1.8Routing entry for 192.168.1.8/32 Known via "isis", distance 115, metric 83, type level-2 Redistributing via isis Last update from 192.168.1.8 on Tunnel2, 00:00:08 ago Routing Descriptor Blocks: * 192.168.1.8, from 192.168.1.8, via Tunnel0 Route metric is 83, traffic share count is 100 192.168.1.8, from 192.168.1.8, via Tunnel1 Route metric is 83, traffic share count is 10 192.168.1.8, from 192.168.1.8, via Tunnel2 Route metric is 83, traffic share count is 1



RtrA RtrE

RtrF

RtrG

100MB10MB

A:Any way it wants to! 15:1, 14:2, 13:2:1, it depends on the order the tunnels come up.Deployment guideline: don’t use tunnel metrics that don’t reduce to 16 buckets!

1MB

gsr1#sh ip cef 192.168.1.8 internal

………

Load distribution: 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 (refcount 1)

Hash OK Interface Address Packets Tags imposed

1 Y Tunnel0 point2point 0 {36}

2 Y Tunnel1 point2point 0 {37}

………


Policy routingPolicy routing

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

RtrI

Tunnel1

RtrA(config-if)#ip policy route-map set-tunnel

RtrA(config)#route-map set-tunnel

RtrA(config-route-map)#match ip address 101

RtrA(config-route-map)#set interface Tunnel1


Policy routingPolicy routing

Node Next-Hop Cost

B B 10

C C 10

D C 20

E B 20

F B 30

G B 30

H B 40

I B 40

Routing table isn’t affected by policy routing.

Need (12.0(16)ST or 12.2T) or higher for ‘set int Tunnel’ to work (CSCdp54178)

RtrA

RtrB

RtrC

RtrE

RtrD

RtrF

RtrG

RtrH

RtrI

Tunnel1


Forwarding Traffic Down a TunnelForwarding Traffic Down a Tunnel

• You can use any combination of autoroute, static routes, or PBR.

• …but simple is better unless you have a good reason.

• Recommendation: either autoroute or statics to BGP next-hops, depending on your needs.


AgendaAgenda

• Prerequisites






Basic ConfigurationBasic Configuration


-Basic Midpoint/Tail Config

-Basic Headend Config

-Path-option

-Bandwidth


Basic Midpoint/Tail ConfigBasic Midpoint/Tail Config

(globally)

ip cef {distributed}

mpls traffic-eng tunnels



(per interface)

mpls traffic-eng tunnels



(if IGP == OSPF)

router ospf <x>

mpls traffic-eng router-id Loopback0

mpls traffic-eng area <y>



(if IGP == OSPF)

• MPLS TE is a single area only (usually area 0)

• RID must be set (unlike OSPF RID)

It’s a Very Very Good idea to make it a /32 loopback.



(if IGP == IS-IS)

router isis <x>

mpls traffic-eng router-id Loopback0

mpls traffic-eng level-{1,2}

metric-style wide



(if IGP == IS-IS)

• MPLS TE is a single level only

• RID must be set (unlike OSPF RID)

It’s a Very Very Good idea to make it a /32 loopback.



‘metric-style wide’ - ???

• IS-IS must have wide metrics enabled

• This is discussed in more detail later in this presentation; see also www.cisco.com.



• Total config tally so far:

1 line globally

1 line per interface

2 lines if OSPF

3 lines if IS-IS


Basic Headend ConfigBasic Headend Config

• Headend needs the 4-5 ‘mid/tail’ lines

• But wait – there’s more!



• Create the tunnel interfaceinterface Tunnel0

ip unnumbered Loopback0

tunnel mode mpls traffic-eng

tunnel source Loopback0

tunnel destination <tunnel endpoint>

tunnel mpls traffic-eng autoroute

tunnel mpls traffic-eng path-option 10 dynamic

unnumbered to Loop0

path-option tells the tunnel how to get to tail’10’ is the priority of the path-option

there are other options besides dynamic

autoroute is not strictly necessary, but is useful



• Total config tally:

1 line globally

1 line per interface

2 lines if OSPF

3 lines if IS-IS

7 lines per tunnel at headend


AgendaAgenda

• Prerequisites






Knobs! Knobs! Knobs!Knobs! Knobs! Knobs!

• Influencing the Path Selection

• Auto-Bandwidth

• Fast Reroute

• DiffServ-Aware Traffic Engineering




Bandwidth

Priority

Administrative Weight

Attributes & Affinity


BandwidthBandwidth

ip rsvp bandwidth <x> <y>

• Per-physical-interface command

• X = amount of reservable BW, in K

• Y = not used by MPLS-TE

• default: X==Y==75% of link bandwidth


PriorityPriority

tunnel mpls traffic-eng <S> {H}

• Configured on tunnel inteface

• S = setup priority (0-7)

• H = holding priority (0-7)

• lower number is more important, or better.


PriorityPriority

• New tunnel with better setup priority will force teardown of already-established tunnel with worse holding priority

• Configuring S<H is illegal

• Default is S=7,H=7


PriorityPriority

RtrB

RtrA

RtrC RtrD45MB

45MB

45MB

= 40MB tunnel with S=7, H=7



PriorityPriority

RtrB

RtrA

RtrC RtrD45MB

45MB

45MB



ResvTear

• RtrC sends ResvTear to RtrA, tunnel is torn down.


PriorityPriority

“Should I ever set S != H?”

No. Not unless you know you have a good reason to.


Administrative WeightAdministrative Weight

mpls traffic-eng administrative-weight <X>


• X = 0-(232 –1)

• gives a metric that be considered for use instead of the IGP metric

• this can be used as a per-tunnel delay-sensitive metric for doing VoIP TE



tunnel mpls traffic-eng path-selection metric {te|igp}

• Per-tunnel command

• default is ‘igp’

• ‘te’ uses the configured administrative-weight to determine shortest cost

• use this as a delay-sensitve metric


Delay-Sensitve Metric with Delay-Sensitve Metric with Adminastrative WeightAdminastrative Weight

tunnel mpls traffic-eng path-selection metric {te|igp}

mpls traffic-eng administrative-weight <x>

• configure admin weight == interface delay

• configure VoIP tunnels to use TE metric to calculate the path

• delay-sensitive metric!


Attributes & AffinityAttributes & Affinity

• Link attribute – 32 separate link properties

• Tunnel affinity – desire for links to have certain properties set

• Invent your own property meanings



mpls traffic-eng attribute-flags <0x0-0xFFFFFFFF>




tunnel mpls traffic-eng affinity <0x0-0xFFFFFFFF> {mask <0x0-

0xFFFFFFFF>}


• Mask is a collection of do-care bits

• ‘affinity 0x2 mask 0xA’ means ‘I care about bits 2 and 8; bit 2 must be set, bit 8 must be 0’



• Q: How should I use admin-weight?

• A: To exclude some links from consideration by some tunnels

• …so give a satellite link an attribute of 0x2, and any VoIP tunnels can be configured with ‘affinity 0x0 mask 0x2’




• Auto-Bandwidth

• Fast Reroute



Auto-BandwidthAuto-Bandwidth

tunnel mpls traffic-eng auto-bw ? collect-bw Just collect Bandwidth info on this tunnel frequency Frequency to change tunnel BW max-bw Set the Maximum Bandwidth for auto-bw on this tunnel min-bw Set the Minimum Bandwidth for auto-bw on this tunnel <cr>


• Periodically changes tunnel BW reservation based on traffic out tunnel

• Timers are tunable to make auto-bw more or less sensitive



tunnel mpls traffic-eng auto-bw ? collect-bw Just collect Bandwidth info on this tunnel frequency Frequency to change tunnel BW max-bw Set the Maximum Bandwidth for auto-bw on this tunnel min-bw Set the Minimum Bandwidth for auto-bw on this tunnel <cr>


• Periodically changes tunnel BW reservation based on traffic out tunnel

• Timers are tunable to make auto-bw more or less sensitive

tradeoff: quicker reaction vs. more churn



gsr1#sh mpls traffic-eng tunnels t0…Config Parameters:… auto-bw: (86400/86259) 0 Bandwidth Requested: 100

• 86400 = reoptimization time (default 24h)tunnel mpls traffic-eng auto-bw frequency <x>

• 86259 = time left to reoptimization

• 0 = BW measured at end of last reopt interval

• bw requested = signalled tunnel BWtunnel mpls traffic-eng {max-bw|min-bw} <bw>




• Auto-Bandwidth

• Fast Reroute



Fast RerouteFast Reroute

• In an IP network, a link failure causes several seconds of outageThing Dependency TimeLink failure detection

Media- and platform-specific

~usecs (POS + APS)

Info propagation IGP timers, network size, collective router load

~5-30sec

Route recalculation LSDB size, CPU load ~1-2sec


Fast RerouteFast Reroute

• In an MPLS network, there’s more work to be done, so a (slightly) longer outage happensThing Dependency TimeLink failure detection

Media- and platform-specific

~usecs (POS + APS)


~5-30sec

Route recalculation LSDB size, CPU load ~1-2sec

New LSP setup network size, CPU load

~5-10sec


Three Kinds of FRRThree Kinds of FRR

• Link Protection

the only scheme implemented today

• Node Protection

on the way

• Path Protection

on development radar


Link ProtectionLink Protection

• TE tunnel A->B->D->E

RtrDRtrB

RtrC

RtrERtrA



• B has a pre-provisioned backup tunnel to the other end of the protected link (RtrD)

• B relies on the fact that D is using global label space

RtrDRtrB

RtrC

RtrERtrA



• B->D link fails, A->E tunnel is encapsulated in B->D tunnel

• Backup tunnel is used until A can recompute tunel path as A->B->C->D->E (so 10-30sec or so)

RtrC

RtrERtrA RtrDRtrB



• On tunnel headend:

tunnel mpls traffic-eng fast-reroute

RtrC

RtrERtrA RtrDRtrB

• On protected link:

mpls traffic-eng backup-path <backup-tunnel>


Node ProtectionNode Protection

RtrA

RtrC

RtrERtrDRtrB RtrF

•RtrA has a tunnel A->B->D->E->F

•RtrB has a protect tunnel B->C->E->D



RtrA

RtrC

RtrERtrDRtrB RtrF

• Link protection is OK if the B->D link goes down

• What if Router D goes away?



RtrA

RtrC

RtrERtrDRtrB RtrF

• Solution: protect tunnel to the hop past the protected link



• Node protection still has the same convergence properties as link protection

• Deciding where to place your backup tunnels is a much harder to problem to solve large-scale

…turns out it’s an NP-complete problem.

• For small-scale protection, link may be better

• Cisco is developing tools to solve these hard problems for you (see TunnelVision, later)


Path ProtectionPath Protection

RtrA

RtrC

RtrERtrDRtrB RtrF

• Path Protection: multiple tunnels from TE head to tail, across diverse paths


Path ProtectionPath Protection

• Path Protection: least scalable, most resource-consuming, slowest convergence of all 3 protection schemes

• Path protection is useful in two places:

1) when you have more links than tunnels

2) when you need to protect links not using global label space


Path vs. Local ProtectionPath vs. Local Protection

Thing Dependency TimeLink failure detection Media- and platform-

specific~usecs (POS + APS)

Local switchover to protect tunnel

RP->IPC communication time

~few msec or less

Thing Dependency TimeLink failure detection Media- and platform-

specific~usecs (POS + APS)


~5-30sec

Headend switchover to protect LSP

network size, CPU load ~msec

Local (link/node) Protection

Path Protection



How Many Backup Tunels Are Required?

• consider 3 LSPs: A->J, B->J, C->

• how can we protect against a failure of RtrF?

RtrB RtrD

RtrA

RtrCRtrE

RtrF

RtrH

RtrG RtrI RtrJ



Protection Scheme 1 tunnel per…Link protection Protected link (since all protected links are p2p)

Number of Backup Tunnels Required

RtrB

RtrA

RtrCRtrE

RtrH

RtrG RtrI RtrJ

Protecting the D->F linkProtect LSP carries 2 LSPs inside it

RtrD RtrF

= protecting B,G



Protection Scheme 1 tunnel per…Node protection Next-next-hop


RtrB RtrD

RtrA

RtrCRtrE

RtrF

RtrH

RtrG RtrI RtrJ

Protecting Router F= protecting R

= protecting B,G



Protection Scheme 1 tunnel per…Path protection LSP


RtrB RtrD

RtrA

RtrCRtrE

RtrF

RtrH

RtrG RtrI RtrJ

Protecting Each LSPR and R’ have mutually exlusive reservations!



Protection Scheme 1 tunnel per…Link protection Protected link (since all protected links are p2p)

Node protection Next-next-hop

Path protection LSP


• So is Path Protection evil?

No. But it has some scalability limits.




• Auto-Bandwidth

• Fast Reroute



Diffserv-Aware Traffic Diffserv-Aware Traffic EngineeringEngineering

• MPLS can advertise and reserve bandwidth on a link

• Works great, but what if you send a mix of LLQ and BE traffic down a TE tunnel?

• Need some way to differentiate and reserve LLQ bandwidth on a link.



• 2 tunnels across C<->E link

• 40MB each tunnel

• 100MB reservable on C<->E, with a 30MB LLQ

• What happens when both tunnels send 20MB of VoIP traffic?

RtrA

RtrB

RtrC

RtrE

RtrD RtrF

RtrG



• Problem: only one pool on an interface, no way to differentiate what types of traffic are carried!

• Solution: advertise more than one pool!

RtrA

RtrB

RtrC

RtrE

RtrD RtrF

RtrG

30MB LLQ+40MB LLQ traffic = 10MB not LLQ’d!



ip rsvp bandwidth <x> sub-pool <y>

• ‘this link has available bandwidth of X, Y of which is in a sub-pool’

• Not quite two pools, really – no sense in witholding bandwidth from global availabilty if it’s not in use

• …which means sub-pool tunnels need to have a better priority than non-sub-pool tunnels.



tunnel mpls traffic-eng bandwidth <x> sub-pool

• ‘this tunnel wants to reserve X Kbps from a sub-pool’

• sub-pool BW is looked at instead of global pool BW

• if sub-pool BW is not available, tunnel won’t come up


AgendaAgenda

• Prerequisites




• Deploying and Designing


Deploying and DesigningDeploying and Designing

• Deployment Methodologies

• Scalability

• Management

• Security


Deployment MethodologiesDeployment Methodologies

• Two ways to deploy MPLS-TE

-as needed to clear up congestion

-full mesh between a set of routers

• Both methods are valid, both have their pros and cons


As NeededAs Needed

• Case study: a large US ISP

RtrA

RtrB

RtrD RtrE

RtrC

•All links are OC12•A has consistent 700MB to send to C•~100MB constantly dropped!


As NeededAs Needed

• Solution: multiple tunnels, unequal-cost load sharing!

RtrB

RtrA

RtrD RtrE

RtrC

•Tunnels with bandwidth in 3:1 ratio•175MB sent the long way•525MB sent the short way•No out-of-order packet issues –

CEF’s normal per-flow hashing is used!


As NeededAs Needed

• From RtrA’s perspective, topo is:

RtrB

RtrA

RtrD RtrE

RtrC


As NeededAs Needed

• As Needed: easy, quick, but hard to track over time.

• Easy to forget why a tunnel is in place

• Inter-node BW requirements may change, tunnels may be working around issues that no longer exist

• Link protection pretty straightforward, node protection much harder to track


Full MeshFull Mesh

• Put a full mesh of TE tunnels between routers

• Initially deploy tunnels with 0 BW

• Watch Tunnel inteface stats, see how much BW you are using between router pairs

-Tunnels are intefaces – use IF-MIB!

-Make sure that tunnel bw <= network bw


Full MeshFull Mesh

• Some folks deploy full mesh just to get router-to-router (pop-to-pop) traffic matrix

• Largest TE network ~80 routers full mesh (~6400 tunnels)

• As tunnel BW is changed, tunnels will find the best path across your network


Full MeshFull Mesh

RtrA

RtrB

RtrD RtrE

RtrC

• Physical topology is:


Full MeshFull Mesh

RtrA

RtrB

RtrD RtrE

RtrC

• Logical topology is:


Full MeshFull Mesh

• Things to remember with full mesh

-N routers, N*(N-1) tunnels

-Routing protocols not run over TE tunnels – unlike an ATM full mesh!

-Tunnels are unidirectional – this is a Good Thing

…can have different BW reservations in two different directions


Full MeshFull Mesh

• Best practices for full mesh:

-periodically reoptimize tunnels based on need (just like an ATM network)

-TE was always designed to be a combination of online (router-based) and offline (NMS) calculation

-Node protection more practical in a full-mesh, offline-generate TE topo




• Scalability

• Management

• Security


ScalabilityScalability

• How many tunnels on a router?

Code # headend tunnels

# of midpoints

12.0S 300 10,00012.0ST 600 10,000

• Tests were done on a GSR.

• RSP4, RSP8, VXR300, VXR400 will be similar



• 300 headends = ~90,000 tunnels

• 600 headends = ~360,000 tunnels

• Largest TE network today = ~6400 tunnels

• 90,000 tunnels = 6400*14

• 360,000 tunnels = 6400*56

• There are other factors to consider

-IGP scaling, BGP, etc

• …but MPLS-TE is not the gating factor in network scaling!



• Largest TE network today = ~6400 tunnels

• 80 routers, ~6400 tunnels full mesh

• 12.0S scales to 300 headends, ~90,000 tunnels full mesh

• 12.0ST – 600 headends, 360,000 tunnels full mesh

• 300=80*3.75

..or (90,000=6400*14) if you’re in marketing

• 600=80*7.50

… or (360,000=6400*56)

• Bottom line: MPLS-TE is not a gating factor in network scaling!



http://www.cisco.com/univercd/cc/td/doc/product/software/ios120/120newft/120limit/120st/120st14/scalable.htm

…or just search CCO for “Scalability Enhancements for MPLS Traffic Engineering”




• Combining VPN+TE

• Scalability

• Management

• Security


Traffic Engineering MIBsTraffic Engineering MIBs

• Interfaces MIB

• MPLS-TE-MIB

• CISCO-TE-MIB

• MPLS-DS-TE-MIB


MPLS-TE-MIBMPLS-TE-MIB

• Goal: Exposes MPLS TE tunnels

Configured tunnel heads and path(s)

Active path(s)

Back-up/stand-by path(s)

Traps


MPLS-DS-TE-MIBMPLS-DS-TE-MIB

• Goal: Exposes DiffServ-Aware Traffic Engineering parameters.

• Extends the MPLS-TE-MIB and MPLS-LSR-MIBs.

• Work still in progress: presented version 00 in Minneapolis IETF meeting (March 2001).


Cisco-TE-MIBCisco-TE-MIB

• Exposes non-standardized TE features such as additional CSPF extensions, auto-bandwidth tunnels, link/node protection, path options, etc…, etc….

• Other vendors have similar proprietary MIBs.


TunnelVisionTunnelVision

• Need a tool to help manage TE LSPs?

• TunnelVision (server and client component, will run on Solaris and Win2k)

• Not a network modeling tool!

Use WANDL, Orchestream, MakeSys, Opnet, others


TunnelVision ArchitectureTunnelVision Architecture

Browser

Data

Control

TVApplet

Solaris WorkStation

TV Server

Web Server

Telnet

http

ApplicationCommands

SNMP


TunnelVision Client ScreenshotTunnelVision Client Screenshot


TunnelVisionTunnelVision

• Cisco is also working with an external partner to add node protection path calculation

• The partner has world-class algorithm development experience

• TunnelVision will feed topology to this tool, tool will calculate backup paths


Other ToolsOther Tools

• There are other MPLS-TE tools

WANDL, Make Systems, Orchestream, OpNet, etc.

• Off-net modeling and path calculation very important to help scale TE deployment




• Scalability

• Management

• Security


SecuritySecurity

• MPLS-TE is not enabled on externally facing intefaces

• Biggest security risk is spoofed RSVP

-hacker would have to know a lot about your topo to do anything

-RSVP authentication exists (rfc2747), not implemented


SecuritySecurity

• MPLS-TE can hide your network topology from the outside world

• Is this “security”? That’s debatable. But it’s certainly a neat knob!

RtrA(config)#no mpls ip propagate-ttl ?

forwarded Propagate IP TTL for forwarded traffic

local Propagate IP TTL for locally originated traffic


ConclusionConclusion

• TE is cool

• You should use lots of it

• It will make you popular

• It also cures leprosy, rickets, and tennis elbow!


AgendaAgenda






What Code Is MPLS-TE In?What Code Is MPLS-TE In?

• IS-IS Support: 12.0(5)S, 12.0(6)T

• OSPF Support: 12.0(8)S, 12.1(3)T

• Also in future derivatives of these trains


AgendaAgenda






Platform Issues in ImplementationPlatform Issues in Implementation

• Basic TE needs software only

RSVP, IS-IS, OSPF, TE

• DS-TE

Needs some form of LLQ

Queueing not tied to advertisement (yet!)

• FRR

Need some quick way to communicate cutover to LCs

• Label Push/Pop

Could push 2 labels (TE+LDP), 3 if VPN also


Reading MaterialReading Material

• ENG-59293 – MPLS Forwarding Spec

• ENG-42799 – TE FRR Design Spec


AgendaAgenda






Core TopologyCore Topology

SRP12N6

OC192N5

OC48N7

OC3POSN2

OC3POSN3

OC48N4

OC48N8

OC12N10 OC12

N11OC12N12

OC12N13

ATM OC12

ATM OC12

POS5/0 POS0/0

POS1/0 POS1/0

POS2/0

G S R 1

G S R 4 G S R 5

G S R 8

G S R 2

POS0/0

POS0/0

POS0/1

G S R 3

G S R 6

G S R 7

POS0/0

POS3/0

POS2/0 POS1/0

POS2/1

POS1/1

POS1/0

POS1/1

POS1/0

to vpnto vpn


TE TopologyTE Topology

AS3402

G S R 1V XR 15V XR 14

V XR 13 V XR 16

N23

N20

N21

N 22

N 25

N24 B G P

R IP

A S 65001

G S R 8

V XR 12

V XR 11

V XR 10

V XR 9

N26

N27

N29

N30

N 31

N 28

O S P F

B G P A S 65501

Tun12

Tun11

Tun15

N O TE : Tun12 and Tun15flow across the bo ttom(long) path and arepro tected via the toppath .

Documents

1 © 2001, Cisco Systems, Inc. Course Number Presentation_ID MPLS TE TOI [email protected]