BANANAS: A Connectionless Approach to Intra- and Inter-Domain Traffic Engineering

Shivkumar KalyanaramanRensselaer Polytechnic Institute 1

BANANAS: A Connectionless Approach to Intra- and Inter-Domain Traffic Engineering

Hema T. Kaur, Shiv Kalyanaraman,

Rensselaer Polytechnic Institute

[email protected], [email protected]

http://www.ecse.rpi.edu/Homepages/shivkuma


Acknowledgements Biplab Sikdar (faculty colleague) Andreas Weiss (MS) Shifalika Kanwar (MS) Mehul Doshi (MS) Ayesha Gandhi (MS) Niharika Mateti (MS) Also thanks to:

Satish Raghunath (PhD) Jayasri Akella (PhD) Hemang Nagar (MS)

Work funded in part by DARPA-ITO, NMS Program. Contract number: F30602-00-2-0537


The Question Can we emulate a subset of MPLS properties without signaling?

Key: Can we do source routing ? without signaling without variable per-packet overhead being backward compatible with OSPF & BGP allowing incremental network upgrades

Shortest Path MPLS…

BANANAS-TE

Signaled TE

TE Spectrum


Why cannot we do it today? Connectionless TE today uses a parametric approach:

Eg: changing link weights in OSPF, IS-IS or parameters of BGP-4 (LOCAL_PREF, MED etc)

Performance limited by the single shortest/policy path

A

B

C

D

1

1 2

1

E

2

Can not do this with OSPFA

B

C

D

1

1 2

1

E

2

Links AB and BD are overloaded

A

B

C

D

1

1 2

4

E

2

Links AC and CD are overloaded

Alt: Connection-oriented/signaled approach (eg: MPLS)

Complex to extend MPLS-TE across multiple areas.

Not a solution for inter-AS issues.

MPLS also needs the support of all the nodes along the path


MPLS Signaling and Forwarding Model

Miami

Seattle

SanFrancisco(Ingress)

New York(Egress)

MPLS label is swapped at each hop along the LSP Labels = LOCAL IDENTIFIERS …

Signaling maps global identifiers (addresses, path spec) to local identifiers

1321

5

120

IP 1321

IP 120

IP 0

IPLabe

l


Global Path Identifiers

Instead of using local path identifiers (Labels in MPLS), we propose the use of global path identifiers

10

Miami

Seattle

9

27


New York(Egress)

18

1

5

4

3

5

IP

IP PathId

4

IP 36

IP 27

IP 0


Global Path Identifier: Key Ideas

ik j

m-11

2w1

w2

wm

IP PathId(i,j)

IP PathId(1,j)

Key ideas: 1. Swap global pathids instead of local labels!2. Unlike source-routing that is simple (IP) or signaled (MPLS),upgraded intermediate nodes need to locally compute the valid PathIDs.


Global Path Identifier (continued)

Path = {i, w1, 1, w2, 2, …, wk, k, wk+1, … , wm, j} Sequence of globally known node IDs & Link weights Global Path ID is a hash of this sequence => locally computable without the need for signaling!

Potential hash functions: [j, { h(1) + h(2) + …+h(k)+ … +h(m-1) } mod 2b ]: node ID sum MD5 one-way hash, XOR, 32-bit CRC etc… We propose the use of MD5 hashing of the subsequence of nodeIDs followed by a CRC-32 to get a 32-bit hash value

Very low collision (i.e. non-uniqueness) probability

i

k

j

m-11

2w1

w2

wm

Path su

ffix


Abstract Forwarding Paradigm Forwarding table (Eg; at Node k):

[Destination Prefix, ] [Next-Hop, ] [j, ] [k+1, ]

i

k

j

m-1

1

2w1

w2

wm

Path su

ffix

Incoming Packet Hdr: Destination address (j) & PathID = H{k, k+1, … , m-1} Outgoing Packet Hdr: [j, PathID = H{k+1, … , m-1} ]

Longest prefix match + exact label match + label swap!PathID mismatch => map to shortest (default) path, and set PathID = 0No signaling because of globally meaningful pathIDs!

PathID SuffixPathID

H{k, k+1, … , m-1} H{k+1, … , m-1}


BANANAS TE: Explicit, Multi-Path Forwarding

Explicit Source-Directed Routing: Not limited by the shortest path nature of IGP Different PathIds => different next-hops (multi-paths) No signaling required to set-up the paths

Traffic splitting is decoupled from route computation

10

Miami

Seattle

9

27


New York(Egress)

18

1

5

4

3

5

IP

IP PathId

4IP 5

IP 0

IP 36IP 27

IP 0


BANANAS TE: Partial Deployment Only “red” routers are upgraded

Non-upgraded routers forward everything on the shortest path (default path): forming a “virtual hop”

10

Miami

Seattle

9SanFrancisco(Ingress)

New York(Egress)

28

1

5

4

30

1

IP

4IP 5

IP 0

IP 27

IP 0

27

1

3

2

IP 27

IP 27

X


Route Computation: All-Paths Under Partial Upgrades (AP-PU)

Assume 1-bit in LSA’s to advertise that an upgraded router is “multi-path capable” (MPC)

Two phase algorithm: (assume m upgraded nodes) 1. (N-m) Dijkstra’s for non-upgraded nodes or one all-pairs

shortest path (Floyd Warshall)

2. DFS to discover valid paths to destinations. Explore all neighbors of upgraded nodes Explore only shortest-path next-hop of non-upgraded nodes Visited bit set to avoid loops

Computes all possible valid paths under PU constraints in a fully distributed manner (global consistency)


Zebra/Click Implementation on Linux (Tested on Utah Emulab)

Part of table at node1: (PathID= Link Weights, for simplicity)

3 9 6

74

5 8

1 2

10

53

13 75

4

5145

83

21

3

6793

5 67

38

51

Destination PathID NextHop SuffixPathID

4 260 2 177 (=260 – 83)

4 98 3 0 (= 98 – 98)

4 51 4 0 (= 51 – 51)

4 160 5 0 (=160 – 160)


A

C

B

ED

A-MPC

Nodes

Avg. # of Paths to each Dest

A 6.3B 5.7D 5.6P-MPC Nodes: #Paths

Avg. 7.7Max 13.1Min 2.8

A-MPC

Nodes


B 6.4D 6.9E 7.9P-MPC Nodes: #Paths

Avg. 6.9Max 15.5Min 2.7

A-MPC

Nodes


B 6.7C 9.4D 6.2P-MPC Nodes: #Paths

Avg. 7.2Max 13.9Min 2.8

SSFnet Simulation Results

Flat OSPF Area, 19 Nodes; Only 3 Active-MPC nodes


Heterogeneous Route Computation

Goal: Upgraded nodes (eg: A, D, E) can use any route computation algorithm, so long as it computes the shortest (default) path!

Eg: k shortest-paths from a given source s to each vertex in the graph, in total time O(E + V log V + kV): lower complexity than AP-PU

Issue: Forwarding for k-shortest paths may not exist Need to validate the forwarding availability for paths!

A

ED

CB

F

2

21

3

1

1

2

53

5

2


Two-Phase Path Validation Algorithm Concept: Forwarding for path exists only if the forwarding for

each of its suffixes exists. Phase 1 (cont’d):

compute {k-shortest} paths for all other upgraded nodes, and 1-shortest paths for non-upgraded nodes.

Sort computed paths by hopcount

Phase 2: Validate paths starting from hopcount = 1. All 1-hop paths valid. p-hop paths valid if the (p-1)-hop path suffix is valid Throw out invalid paths as they are found

Polynomial complexity to discover all valid paths in the network & validation can be done in the background Validation algorithm correct by mathematical induction


Linux/Zebra/Emulab Results

D

B

C

Active Nodes


B(k=3) 2.94D(k=3) 2.94C(k=3) 2.79Avg. # of Paths/k *100

B 98%D 98%C 93%

Active Nodes


B(k=7) 6.5

D(k=5) 4.78C(k=5) 4.44Avg. # of Paths/k *100

B 93%D 96%C 89%

Active Nodes


B(k=5) 4.83D(k=5) 4.78C(k=5) 4.44Avg. # of Paths/k *100

B 97%D 96%C 89%

Flat OSPF Area, 3 Active-MPC nodes; Upto k-shortest, validated paths


Inter-domain TE Outbound TE:

Multi-exit (or Explicit-exit) routing Useful to manage peering vs transit costs

Hash = (Exit ASBR, destination address) Forwarding paradigm: Connectionless tunneling thru the

AS

Inbound TE: NOT ADDRESSED DIRECTLY

Multi-AS-Path or Explicit AS-Path routing: Framework similar to IGP: e-PathID concept


BGP Explicit-Exit Routing: Route Selection Explicit-Exit routing is easier than Explicit-Path Routing

Only the “source” and “exit” nodes need upgrades ! Explicit exit routing easily extended to “multi-exit” routing

Upgrade selected EBGP and IBGP routers

All BGP routers synchronize on the default policy route to every destination prefix (as usual)

Only upgraded IBGP routers and EBGP routers synchronize on a set of exits for chosen prefixes

Upgraded IBGP routers can independently choose any exit without further synchronization with other BGP nodes


BGP Explicit-Exit Routing: Forwarding IBGP locally installs explicit & default exits for chosen prefix

Dest-Prefix Exit-ASBR Next-Hop Dest-Prefix Default-Next-Hop Next-hop refers to the IGP next hop to reach Exit-ASBR Default-Next-Hop: regular IBGP function

When a packet matches the explicit route (policy definable):

Push its destination address into an Address Stack field

Replace destination address with Exit-ASBR address.

Emulates 1-level label-stacking (I.e. tunneling)

Exit-ASBR simply swaps back the destination address, before regular IP lookup => popping the stack


Explicit-Exit Routing Example

AS1

AS2

AS3

AS4 Dest. d

ABR1

ASBR2

ASBR3

ASBR4ASBR1

ABR2

Default (AS Path , Exit) to d = (1-3-4, ASBR3) Now, ABR1 can have explicit exits ASBR4 (implied ASPath = 1-2-4), ASBR2

(implied ASPath =1-3-4) as well!!


Inter-AS Explicit AS-Path Choice

AS0

AS1

AS2

AS3

AS4 Dest. d

ASBR1

ASBR2

ASBR3

Allow AS0 to explicitly choose an AS-PATH: e.g. 0-1-2-4 or 0-1-3-4,

Explicit AS-Path choice encoded as an e-PathID = Hash{1,2,4}e-PathID is updated only when the packet leaves the AS at Exit border routers.

At ASBR1, this explicit AS-path choice is mapped to an exit ASBR.Within an upgraded AS, the packet is tunneled using the routing header as explained earlier. Only selected EBGP nodes need be upgraded & synchronized


Re-advertisements of Multi-AS-Paths

AS2

AS1

AS0

AS3

AS4 Dest. d

AS5

ASBR1ASBR2

iBG-1 iBG-3

3 AS-paths to “d”(0 4) (0 3 4) (0 5 4)

1 AS-path or3 AS-paths to “d”??

Issue: in path-vector algorithms, without re-advertisements (of a subset of paths), remote AS’s cannot see the availability of multiple paths But, re-advertisements adds control traffic overhead An AS may choose to re-advertise only, and not support multi-path forwarding (I.e. interpreting e-PathID or Address Stack fields)


Summary TE: “Towards Better routing performance”:

Key: Decoupling route availability and setup issues from traffic mapping issues, without signaling

BANANAS-TE can leverage the rich interconnectivity and multi-homed nature of the Internet, with manageable increase in complexity

TE spectrumShortest Path MPLS…

BANANAS-TE

Signaled TE


Extra Slides


BGP Explicit Exit: SSFnet Simulation Example


AS 2

AS 5

AS 4

AS 3

0.0.0.48/29

0.0.0.0/27

0.0.0.56/29

0.0.0.32/28

1

1

1

1

2

2

4

3

2

2

3


Outgoing Packet from AS2 router 1

AS 2

94/32

18/32

10/32 2/32

14//32

1/32

4

3

16/32

12/3217/32

Dest address is pushed to stack @ AS2 (1); like MPLS => tunneling emulation!

Destination Address Stack

0.0.0.17/32 0.0.0.48/29

1

Destination NextHop Exit ASBR Source

0.0.0.48/29 0.0.0.18/32 0.0.0.17/32 IBGP

0.0.0.17/32 0.0.0.18/32 --- IBGP

Forwarding Table@ AS2 (Router 1)

2


AS 2

94/32

18/32

10/32 2/32

14//32

1/32

2

4

3

16/32

12/3217/32

AS 3

290/32

42/32

33/32

38/32

Destination address is popped back from Address Stack at AS2 (2)

Outgoing packet from AS2 router 2

Destination Address Stack

48/29 ---

Destination NextHop Exit ASBR Src

0.0.0.17/32 0.0.0.17/32 SELF IBGP

0.0.0.48/29 0.0.0.90/32 SELF EBGP

Forwarding Table@ AS2 (Router 2)

1

1


AS1-AS2-AS4-AS3-AS5

AS 1

AS 2

AS 5

AS 4

AS 3

0.0.0.64/29

0.0.0.48/29

0.0.0.0/27

0.0.0.56/29

0.0.0.32/28

1

1

1

1

1

2

2

4

3

2

2

3


AS 2

94/32

18/32

10/32

22/32

2/32

14//32

1/32

5/32

1

12

AS1-AS2-AS4-AS3-AS5

4

3

16/32

11/32

Destination Next Hop Learnt from ePathID AS Path Outgoing ePathID

48/29 94/32 EBGP 1996809222 2-5 4038336721

48/29 94/32 EBGP 1473492148 2-3-5 1044010488

48/29 94/32 EBGP 1272272860 2-4-3-5 3884942939

Forwarding Table at Router 1 of AS1

Destination ePathID

48/29 3884942939

Outgoing Packet at 1 of AS1

UpgradedUpgraded

AS 1


AS 2

94/32

18/32

10/32

22/32

2/32

14//32

1/32

5/32

1

12

AS1-AS2-AS4-AS3-AS5

4

3

16/32

11/32

12/32


48/29 10/32 IBGP 4038336721 2-5 4038336721

48/29 22/32 IBGP 1044010488 2-3-5 1044010488

48/29 18/32 IBGP 3884942939 2-4-3-5 3884942939


Destination ePathID

48/29 3884942939


UpgradedUpgraded

AS 1


94/32

18/32

10/32

22/32

2/32

14//32

1/32

5/32

12

3

16/32

11/32

12/32

Upgraded

81/321

2

86/32

0.0.0.56/29


48/29 11/32 IBGP 4038336721 2-5 4038336721

48/29 17/32 IBGP 1044010488 2-3-5 1044010488

48/29 86/32 EBGP 3884942939 2-4-3-5 1572206147*

17//32

Non Upgraded


AS1-AS2-AS4-AS3-AS5Destination ePathID

48/29 1572206147*


AS 2

AS 4

83/32

*-- ePathID = Hash(3-5 )instead of 4-3-5

*Alternatively -- ePathID = 0


81/321

2

AS 3

90/32

37/3238/32

33/32

42/32

1

3

87/32

AS1-AS2-AS4-AS3-AS5

Destination Next Hop Learnt From AS Path

48/29 87/32 IBGP 4-3-5


Destination Next Hop Learnt From AS Path

48/29 42/32 EBGP 3-5


Destination ePathID

48/29 1572206147

*Outgoing Packet from router 1 & 2 of AS4

AS 4

Upgraded

Non Upgraded

83/32

*No change in ePathID


AS 3

90/32

37/3238/32

33/32

42/32

1

3Upgraded

AS 5

85/32 58/32

77/321

20.0.0.48/29

0.0.0.32/28

Non Upgraded


48/29 38/32 IBGP 1572206147 3-5 1572206147

48/29 90/32 IBGP 1044010488 3-2-5 1044010488

48/29 83/32 IBGP 3884942939 3-4-2-5 3884942939


2AS1-AS2-AS4-AS3-AS5

Destination ePathID

48/29 1572206147

Incoming and Outgoing Packet from Router 2 of AS3


AS 3

90/32

37/3238/32

33/32

42/32

1

3Upgraded

AS 5

85/32 58/32

77/321

20.0.0.48/29

0.0.0.32/28

Non Upgraded

2AS1-AS2-AS4-AS3-AS5


48/29 77/32 EBGP 1572206147 5 0

48/29 37/32 IBGP 1044010488 3-2-5 1044010488

48/29 33/32 IBGP 3884942939 3-4-2-5 3884942939

Destination ePathID

48/29 0


Outgoing Packet from Router 3 of AS3


Simulation/Implementation/Testing Platforms

Utah’s Emulab Testbed: Experiments with

Linux/Zebra/Click implementation

MIT’s Click Modular RouterOn Linux:

Forwarding Plane

SSFnet Simulation for OSPF/BGP Dynamics

Modular Router


Even a few A-MPC routers makes appreciable number of paths available in the network!

P-MPCs (eg: edge-routers) could act as “sources”

Managing Complexity: Active/Passive MPC Routers

Issue: DFS computation complexity and number of paths grow exponentially as a function of MPC nodes

Solution 1: Divide upgraded routers into two sets: Passive MPC routers

(P-MPC) and Active MPC (A-MPC) Only A-MPC routers set the MPC bit in LSAs Effective maximum number of MPC routers “seen”

= Number of A-MPC routers + 1


Router LSA Modifications

MPC-Bit: Unused Bit #7 of options

ki value used at router i: Unused 8 Bits after Router Type

(reproduced from John Moy’s OSPF book)

Documents

BANANAS: A Connectionless Approach to Intra- and Inter-Domain Traffic Engineering