Can the Border Gateway Can the Border Gateway Protocol (BGP) be fixed? Protocol (BGP) be fixed?

UCLOct 15, 2003

Timothy G. Griffin Intel Research,

Cambridge UK

tim.griffin@intel.com

How do you connect to the Internet?

Physical connectivity isjust the beginning of thestory….

Architecture of Dynamic Routing

AS 1

AS 2

EGP (= BGP)

EGP = Exterior Gateway Protocol

IGP = Interior Gateway Protocol

Metric based: OSPF, IS-IS, RIP, EIGRP (cisco)

Policy based: BGP

The Routing Domain of BGP is the entire Internet

IGP

IGP

BGP Table Growth

Thanks to Geoff Huston. http://bgp.potaroo.net on May 30, 2003

How Many ASNs are there?

Thanks to Geoff Huston. http://bgp.potaroo.net on May 30, 2003

Partial View of www.cam.ac.uk (131.111.8.46) Neighborhood

AS 786 ja.net(UKERNA)

AS 1239 Sprint

AS 4373 Online Computer Library Center

Originates > 180 prefixes, Including 131.111.0.0/16

AS 3356Level 3

AS 6461AboveNet

AS 1213 HEAnet(Irish academic and research)

AS 7 UK Defense Research Agency

AS 5459 LINX

AS 702 UUNET

AS 20965 GEANT

• Topology information is flooded within the routing domain

• Best end-to-end paths are computed locally at each router.

• Best end-to-end paths determine next-hops.

• Based on minimizing some notion of distance

• Works only if policy is shared and uniform

• Examples: OSPF, IS-IS

• Each router knows little about network topology

• Only best next-hops are chosen by each router for each destination network.

• Best end-to-end paths result from composition of all next-hop choices

• Does not require any notion of distance

• Does not require uniform policies at all routers

• Examples: RIP, BGP

Link State Vectoring

Technology of Distributed Routing

8

BGP Route Processing

Best Route Selection

Apply Import Policies

Best Route Table

Apply Export Policies

Install forwardingEntries for bestRoutes.

ReceiveBGPUpdates

BestRoutes

TransmitBGP Updates

Apply Policy =filter routes & tweak attributes

Based onAttributeValues

IP Forwarding Table

Apply Policy =filter routes & tweak attributes

Open ended programming.Constrained only by vendor configuration language

9

Shedding Inbound Traffic with ASPATH Prepending

Prepending will (usually) force inbound traffic from AS 1to take primary linkAS 1

192.0.2.0/24ASPATH = 2 2 2

customerAS 2

provider

192.0.2.0/24

backupprimary

192.0.2.0/24ASPATH = 2

Yes, this is a Glorious Hack …

10

… But Padding Does Not Always Work

AS 1

192.0.2.0/24ASPATH = 2 2 2 2 2 2 2 2 2 2 2 2 2 2

customerAS 2

provider

192.0.2.0/24

192.0.2.0/24ASPATH = 2

AS 3provider

AS 3 will sendtraffic on “backup”link because it prefers customer routes and localpreference is considered before ASPATH length!

Padding in this way is oftenused as a form of loadbalancing

backupprimary

11

COMMUNITY Attribute to the Rescue!

AS 1

customerAS 2

provider

192.0.2.0/24

192.0.2.0/24ASPATH = 2

AS 3provider

backupprimary

192.0.2.0/24ASPATH = 2 COMMUNITY = 3:70

Customer import policy at AS 3:If 3:90 in COMMUNITY then set local preference to 90If 3:80 in COMMUNITY then set local preference to 80If 3:70 in COMMUNITY then set local preference to 70

AS 3: normal customer local pref is 100,peer local pref is 90

Don’t celebrate just yet…

customer

peering

provider/customer

Provider B (Tier 1)Provider A (Tier 1)

Provider C (Tier 2)

Now, customer wants a backup link to C….

provider/customer

Customer installs a “backup link” …

customer

Provider B (Tier 1)Provider A (Tier 1)

Provider C (Tier 2)

customer sends “lower my preference” Community value

primarybackup

Disaster Strikes!

customer

Provider B (Tier 1)Provider A (Tier 1)

Provider C (Tier 2)primary

backup

customer is happy that backup was installed …

The primary link is repaired, and something odd occurs…

customer

Provider B (Tier 1)Provider A (Tier 1)

Provider C (Tier 2)primary

backup

YIKES --- routing DOES NOT return to normal!!!

WAIT! It Gets Better…

A

P

B

BB

C

B

D

P = primary B = backup

OOOOOPS!

A

P

B

BB

C

B

DSuppose A, B, C all break ties in the same direction(clockwise or counter-clockwise)

No solution =Protocol Divergence

What the heck is going on?

• There is no guarantee that a BGP configuration has a unique routing solution. – When multiple solutions exist, the (unpredictable) order

of updates will determine which one is wins.

• There is no guarantee that a BGP configuration has any solution!– And checking configurations NP-Complete [GW1999]

• Complex policies (weights, communities setting preferences, and so on) increase chances of routing anomalies.– … yet this is the current trend!

Larry Speaks

http://www.larrysface.com/

Is this any way to run an Internet?

What Problem is BGP Solving?

Underlying problem

Shortest Paths

Distributed means of computing a solution.

????

RIP, OSPF, IS-IS

BGP

[GSW1998, GSW2002]

Stable Paths

Separate dynamic and static semantics

SPVP = Simple Path Vector Protocol, a distributed algorithm for solving SPP

BGP

SPVP

Booo Hooo, Many, many complications...

BGP Policies

Stable Paths Problem (SPP)

“static”semantics

dynamicsemantics

Worst case, This is an exponentialTime and space translation

1

An instance of the Stable Paths Problem (SPP)

2 5 5 2 1 0

0

2 1 02 0

1 3 01 0

3 0

4 2 04 3 0

3

4

2

1

•A graph of nodes and edges, •Node 0, called the origin, •For each non-zero node, a set or permitted paths to the origin. This set always contains the “null path”. •A ranking of permitted paths at each node. Null path is always least preferred. (Not shown in diagram)

When modeling BGP : nodes represent BGP speaking routers, and 0 represents a node originating some address block

most preferred…least preferred

5 5 2 1 0

1

A Solution to a Stable Paths Problem

2

0

2 1 02 0

1 3 01 0

3 0

4 2 04 3 0

3

4

2

1

•node u’s assigned path is either the null path or is a path uwP, where wP is assigned to node w and {u,w} is an edge in the graph,

•each node is assigned the highest ranked path among those consistent with the paths assigned to its neighbors.

A Solution need not represent a shortest path tree, or a spanning tree.

A solution is an assignment of permitted paths to each node such that

An SPP may have multiple solutions

First solution

1

0

2

1 2 01 0

1

0

2

1

0

2

2 1 02 0

1 2 01 0

2 1 02 0

1 2 01 0

2 1 02 0

Second solutionDISAGREE

BAD GADGET : No Solution

2

0

31

2 1 02 0

1 3 01 0

3 2 03 0

4

3

This is an SPP version of the example first presented in Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan,and Deborah Estrin. Computer Networks, Jan. 2000

SURPRISE!

2

0

31

2 1 02 0

1 3 01 0

3 4 2 03 0

4

4 04 2 04 3 0

Becomes a BAD GADGET if link (4, 0) goes down.

BGP is not robust : it is not guaranteed to recover from network failures.

Can BGP be fixed?

Joint work with Aaron Jaggard (UPenn Math) and Vijay Ramachandran (Yale CS) SIGCOMM 2003

• BGP policy languages have evolved organically

• A policy language really should be designed!

• But how?

Design Dimensions

• Robustness (required!)• Transparency (required!)• Expressive Power• Autonomy (“freedom of

independent action”) • Global Consitency• Policy Opaqueness

Tradeoffs abound

Robustness

Partially Partially Ordered (PP0): For all paths P and Q, (P < Q and Q < P) implies (P = Q or last(P) = last(Q))

Checking robustness is an NP-hard

P < Q : transitive closure of (subpath relation on permitted paths union the path ranking relation at each node)

This is a sufficient condition for robustness

Transparency, Autonomy

• Transparency: protocol will compose its transformation with transformation of policy writer.

• Autonomy: measure of “wiggle room”– Weak autonomy: neighbors can’t dictate

relative ranking of routes– Stronger: policy writer can classify

neighbors and rank routes based on class (“autonomy of neighbor ranking”).

Need Global Constraints

Theorem: Any robust system supporting both transparency and autonomy must have a non-trivial global constraint

Global constraints must be a part of design from the start

A Partial Ordered for the Design Space

( J , L ) < ( J , L ) 11 2 2

if and only if for all S : SPP

1. J(S) implies J(S)

2. L(S) implies L(S) 2

2

21

1

Local ConstraintGlobal Constraint

Robust Designs

( J, L ) is robust if and only if

2

(J and L ) implies PPO

Examples:

( True, SP )

( PPO, True )

Robust Subspace

( PPO, True )

( True, SP )

Exp

ress

ive P

ow

er

Con

stra

int

Sim

plic

ity

Not tractable

Tractable

Hierarchical BGP (HBGP)

HBGP

HBGP +PEER + BU

HBGP +PEER HBGP + BU

[GR2000, GGR2001]

Next?

• Need techniques for constructing policy languages.

• Design of protocols to enforce global constraints.

• Is there a general formalism to capture autonomy?

References

• [VGE1996, VGE2000] Persistent Route Oscillations in Inter-Domain Routing. Kannan Varadhan, Ramesh Govindan, and Deborah Estrin. Computer Networks, Jan. 2000. (Also USC Tech Report, Feb. 1996)

• [GW1999] An Analysis of BGP Convergence Properties. Timothy G. Griffin, Gordon Wilfong. SIGCOMM 1999

• [GSW1999] Policy Disputes in Path Vector Protocols. Timothy G. Griffin, F. Bruce Shepherd, Gordon Wilfong. ICNP 1999

• [GW2001] A Safe Path Vector Protocol. Timothy G. Griffin, Gordon Wilfong. INFOCOM 2001

• [GR2000] Stable Internet Routing without Global Coordination. Lixin Gao, Jennifer Rexford. SIGMETRICS 2000

• [GGR2001] Inherently safe backup routing with BGP. Lixin Gao, Timothy G. Griffin, Jennifer Rexford. INFOCOM 2001

– [GW2002a] On the Correctness of IBGP Configurations. Griffin and Wilfong.SIGCOMM 2002.

– [GW2002b] An Analysis of the MED oscillation Problem. Griffin and Wilfong. ICNP 2002.