Multipath RoutingMultipath Routing

Jennifer RexfordJennifer Rexford

Advanced Computer NetworksAdvanced Computer Networkshttp://www.cs.princeton.edu/courses/archive/fall08/http://www.cs.princeton.edu/courses/archive/fall08/

cos561/cos561/Tuesdays/Thursdays 1:30pm-2:50pmTuesdays/Thursdays 1:30pm-2:50pm

Outline

• Mostly single-path routing today– BGP and IGPs

• Multipath intradomain– Equal-cost multipath– MultiProtocol Label Switching

• Multipath interdomain– Intelligent route control by stub ASes– Overlay routing through intermediate node– Multipath extensions to BGP

• Preventing out-of-order packets

Conventional IP Routing Protocols

BGP is a Single-Path Protocol

• Each router picks a single best route– From the routes learned from its neighbors

• And announces that route to its neighbors

1

2

34

5

67

Client Web server

Intradomain Routing is (Mostly) Single Path

• Shortest-path routing as sum of link weights– Equal splitting over multiple shortest paths

• No traffic sent along the non-shortest paths

32

2

1

13

1

4

5

3

Why Single Path?

• Simple routing protocol– Low computational overhead– Limited control-plane messages

• Simple packet forwarding– One, or at most a few, forwarding-table

entries– Easy to do hop-by-hop forwarding– Packets generally delivered in-order

• More control over the flow of traffic– Little control relinquished to upstream

neighbors– Use the announced path, or not…

Motivations for Multipath

• Better efficiency– Splitting load over multiple paths

• Better performance– Selecting the low-delay (or high-throughput)

path

• Better reliability– Faster failover from one path to another

• Better security– Prevent on-path adversary from seeing all

packets

• More control– Providing greater flexibility to upstream ASes

Intradomain Multipath

Relatively Easy in Equal-Cost Multipath

• Routers compute shortest paths– Identify next-hops along shortest paths– Put multiple entries in the forwarding table– Split traffic evenly over the paths

• Still, no global information for smarter splitting

32

2

1

13

1

3

5

3

Multi-Topology Routing

• Extension to existing intradomain routing– Multiple weights on each link– Compute shortest paths with each set of

weights

• Separate paths for different traffic classes– Minimize delay for VoIP and gaming– Maximize throughput for Web downloads

• Forward packets selectively on the paths– E.g., based on type-of-service bits in IP

header

Flexible Multipath With Virtual Circuits

• Establish one or more paths within an AS– Explicit signaling of the path in advance– Control over which packets use each path

• Virtual Circuit Identifier (VC ID)– Source set-up: establish path for the VC– Switch: mapping VC ID to an outgoing link– Packet: fixed length label in the header

1

2

1: 72: 7

link 7 1: 142: 8

link 14link 8

Multi-Protocol Label Switching

• Problem: using VC ID along the whole path– Each virtual circuit consumes a unique ID– Starts to use up all of the ID space in the

network

• Label swapping– Map the VC ID to a new value at each hop

• Table has old ID, next link, and new ID

– Allows reuse of the IDs at different links1

2

1: 7: 202: 7: 53 link 7

20: 14: 7853: 8: 42

link 14link 8

Multipath Routing With MPLS

• Establish multiple paths– Signaling message to explicitly set-up the paths

• Flexible splitting over the paths– Configurable splitting ratios

• Flexible control over which traffic uses paths– Configurable “forwarding equivalence classes”

• Fast recovery from failures– Pre-configuration of backup paths– To protect primary paths, or even individual links

Interdomain Multipath: Multihoming

Outbound Traffic: Pick a BGP Route

• Easier to control than inbound traffic– IP routing is destination based– Sender determines where the packets go

• Control only by selecting the next hop– Border router can pick the next-hop AS– Cannot control selection of the entire path

Provider 1 Provider 2

“(2, 7, 8, 4)”“(1, 3, 4)”

Outbound Traffic: Primary and Backup

• Single policy for all prefixes– High local-pref for session to primary

provider– Low load-pref for session to backup provider

• Outcome of BGP decision process– Choose the primary provider whenever

possible– Use the backup provider when necessary

• But…– What if you want to balance traffic load?– What if you want to select better paths?

Outbound Traffic: Load Balancing

• Selectively use each provider– Assign local-pref across destination prefixes– Change the local-pref assignments over

time

• Useful inputs to load balancing– End-to-end path performance data

• E.g., active measurements along each path

– Outbound traffic statistics per destination prefix• E.g., packet monitors or router-level support

– Link capacity to each provider– Billing model of each provider

Splitting Over Multiple Paths

• Use multiple outbound paths at the same time– Completely under the control of the edge

router– No announcements sent to any neighbors– No need to encapsulate or mark the packets

• Still can only pick among the next-hop ASes Provider 1 Provider 2

“(2, 7, 8, 4)”“(1, 3, 4)”

Inbound Traffic: Influencing What Others Do

• Harder to control than outbound traffic– IP routing is destination based– Sender determines where the packets go

• Control only by influencing others’ decisions– Explicitly tell the providers what to do– Indirectly try to influence their decisions

Provider 1 Provider 2

Research Challenges

• How to monitor performance efficiently?– Ping? TCP transfers? HTTP downloads?– Per prefix? Per popular prefix?

• How to optimize the load balancing?– Considering load, performance, and cost

• Impact on the upstream providers?– Uncertainty about the offered load

• How to ensure global stability?– What if everyone starts doing it?

• How to balance goals of sender and receiver?– Joint control over path selection?

More Flexible Interdomain Multipath

Source Routing

• Source routing– Propagate topology information– Let end hosts or edge routers pick paths– Carry the path information in the packets

• Scalability challenges– Large topology and frequent churn– Can operate at (say) the AS level

• Tussles over control– End-hosts/edge-routers wanting more

control– ISPs not wanting to relinquish control

Multipath Through Overlays

Premise: by building application overlay network, can increase performance and reliability of routing

Two-hop (application-level)Berkeley-to-Princeton route

application-layer router

Princeton Yale

Berkeley

http://nms.csail.mit.edu/ron/

How Does RON Work?

• Exchange the results of the probes– Each host shares results with every other

host– Essentially running a link-state protocol!– So, every host knows the performance

properties

• Forward through intermediate host when needed

AC

BB

Extensions to BGP to Propagate Extra Routes

• BGP sessions over multiple hops – E.g., AS 7 tells AS 6 about “(7, 3, 2, 1)”

• BGP sessions announcing multiple paths– E.g., AS 4 also announces “(4, 7, 3, 2, 1)” to

5

1

2

34

5

67

Client Web server

Discussion

• Why can overlay routing perform well?– Even better than the “underlay” it runs on?

• Should the overlay run directly on the routers?– Avoid load (and delay) through intermediate hosts– Have access to additional network-level info

• What kind of business relationships needed?– To make the additional path diversity available

• How to get accurate path information?– Propagate in the routing messages? (Trust?)– Measure end-to-end? Is that good enough?

Out-of-Order Packets

• Multipath can lead to out-of-order delivery– Packets on different paths may get out of order

• No problem for packets from different flows– E.g., different TCP connections

• Can direct related packets onto the same path– Hash-based splitting of traffic

• Can direct bursts of packets onto same path– “Flow-let” switching, change paths only after

gap

• Maybe transport shouldn’t be so sensitive– Being less sensitive to out-of-order delivery

Conclusions

• Multipath routing is useful– Load balancing, reliability, security,

performance

• Multipath routing is challenging– Scalability, control over data plane, tussle

over control over the flow of traffic

• Variety of techniques– Flexible splitting, multi-homing, MPLS,

deflection through an intermediate node, …

• Many open research questions– Scalability, stability,