Internet Routing (COS Internet Routing (COS 598A)598A)

Today: Interdomain TopologyToday: Interdomain Topology

Jennifer RexfordJennifer Rexford

http://www.cs.princeton.edu/~jrex/teaching/http://www.cs.princeton.edu/~jrex/teaching/spring2005spring2005

Tuesdays/Thursdays 11:00am-12:20pmTuesdays/Thursdays 11:00am-12:20pm

Outline

• Interdomain topology– AS graph– Inferring the topology from routing data– Structure of the AS graph

• AS relationships– Common pair-wise relationships– Inferring the relationships from routing data– Characteristics of the relationship graph

• Peering policies– Example of AOL’s peering agreement

What is the AS Graph?

• Node: Autonomous System• Edge: Two ASes that speak BGP to each

other

1

2

3

4

5

67

What is an Edge, Really?

• Edge in the AS graph– At least one BGP session between two ASes– Some destinations reached from one AS via the other

AS 1

AS 2

d

Exchange Point

AS 1

AS 2

d

AS 3

How Do You Know a Node or Edge Exists?

• Consult the Whois database?– Tells which ASes have been allocated

– But, might be out-of-date on who owns it

– … and often doesn’t say who the neighbors are

• See a path that uses the node/edge– Collect measurements of AS paths

– Extract all of the nodes and edges

– E.g., AS path “7018 1 88” implies• Nodes: 7018, 1, and 88

• Edges: (7018, 1) and (1, 88)

Interdomain Routing: Border Gateway Protocol

• ASes exchange info about who they can reach– IP prefix: block of destination IP addresses– AS path: sequence of ASes along the path

• Example: a BGP route in AS 7018 shows– Prefix 12.34.158.0/24 has path “7018, 1, 88”

70181 88

12.34.158.5

“12.34.158.0/24: path (7018,1,88)” “12.34.158.0/24: path (88)”

data traffic data traffic

Where to Get BGP Routes: Public Servers

80

701

7018

4

1221

37867

BGP sessions

9.184.112.0/20

3.0.0.0/8

Example: BGP Table (“show ip bgp” at RouteViews)

Network Next Hop Metric LocPrf Weight Path* 3.0.0.0/8 205.215.45.50 0 4006 701 80 i* 167.142.3.6 0 5056 701 80 i* 157.22.9.7 0 715 1 701 80 i* 195.219.96.239 0 8297 6453 701 80 i* 195.211.29.254 0 5409 6667 6427 3356 701 80 i*> 12.127.0.249 0 7018 701 80 i* 213.200.87.254 0 3257 701 80 i* 9.184.112.0/20 205.215.45.50 0 4006 6461 3786 i* 195.66.225.254 0 5459 6461 3786 i*> 203.62.248.4 0 1221 3786 i* 167.142.3.6 0 5056 6461 6461 3786 i* 195.219.96.239 0 8297 6461 3786 i* 195.211.29.254 0 5409 6461 3786 i

AS 80 is General Electric, AS 701 is UUNET, AS 7018 is AT&TAS 3786 is DACOM (Korea), AS 1221 is Telstra

Characteristics of the AS Graph

• AS graph structure– High variability in node degree (“power

law”)– A few very highly-connected ASes– Many ASes have only a few connections

1 10 100 1000

CC

DF

1

0.1

0.01

0.001

AS degree

All ASes have degree >= 1

Very few have degree >= 100

Characteristics of AS Paths

• AS path may be longer than shortest AS path• Router path may be longer than shortest path

s d

3 AS hops, 7 router hops

2 AS hops, 8 router hops

Problem of Missing Edges

• Limited collection of paths– Some edges might never be traversed– Especially low in the AS hierarchy– … and backup links

• Example: paths from two tier-1 ISPs miss an edge

AT&T Sprint

Harvard HarvardB-schoold1 d2

???

Problem of Missing Nodes

• Route aggregation– AS advertises a larger address block– Smaller address block not seen everywhere– Can cause an AS not to appear in BGP table

– C’s table: has paths “C”, “C D”, and “C E”– B’s table: has only path “C” for 12.0.0.0/8

A CBD

E

D: 12.1.0.0/16

E: 12.2.0.0/16

C: 12.0.0.0/8

Research Questions

• Incomplete data– How to get more data?– How much does missing data affect

answers?– What kinds of questions can be answered

safely?

AS Relationships

Interdomain Routing Policies

• Two main decisions– Path selection: which of the paths to use?– Path export: which neighbors to tell?

• Both driven by business relationships, e.g.,– Customer pays provider for Internet access– Peers find it mutually advantageous to cooperate

32 1

12.34.158.5

“12.34.158.0/24: path (2,1)” “12.34.158.0/24: path (1)”

data traffic data traffic

Customer-Provider Relationship

• Customer needs to be reachable from everyone– Provider exports routes learned from customer to

everyone

• Customer does not want to provide transit service– Customer does not export from one provider to another

d

d

provider

customer

customer

provider

Traffic to the customer Traffic from the customer

advertisements

traffic

Peer-Peer Relationship

• Peers exchange traffic between customers – AS exports only customer routes to a peer

– AS exports a peer’s routes only to its customers

peerpeer

Traffic to/from the peer and its customers

d

advertisements

traffic

Paths You Should Never See (“Invalid”)

Customer-provider

Peer-peer

two peer edges

transit through a customer

Other Kinds of Relationships

• Siblings– Same company– Mutual transit service– Like one bigger AS– Mergers, acquisitions, …

• Backup– Used only when failure– Second provider– Backup peering

• Geography-specific– Customer in U.S.– Peer in Europe

A

B

C

D

E

F

G

H

primary

backup

AS Relationships Matter

• Scientific understanding– Understanding Internet structure and evolution– Understanding why certain paths are used for

traffic

• Placement of Web servers– Want to be close to most customer networks

• Business decisions– Selecting new peer or provider, or renegotiating

relations

• Security policies– Knowing which BGP routes look suspicious

• Analyzing BGP convergence– Relationships have a big impact here (more later!)

Inferring AS Relationships

• Top down: how routes are selected– AS relationships define routing policy– Routing policy determines the routes you see

• Bottom up: how policies can be inferred– Routing data are available from public sources– The chosen routes tell you about the policy

• Example: seeing path “A B C” tells you…– B permits A to transit through B to reach C– (A,B) and (B,C) cannot both be peering links– A and C are not both upstream providers of B

Type-of-Relationship Problem

• Given the inputs– AS graph G(V,E) with vertices V and edges E– Set of paths P on the graph G

• Find a solution that– Labels each edge with an AS relationship – Minimizes the number of “invalid” paths in P

• Rich area of research work– http://www-unix.ecs.umass.edu/~lgao/ton.ps– http://www.cs.princeton.edu/~jrex/papers/

infocom02.pdf– http://www.cs.berkeley.edu/~sagarwal/research/BGP-hierarchy/

– … lots of scope for algorithms-oriented research project

AS Relationship Graph (2002)

• Lowest level: Stubs– Leaf nodes: no peers or downstream customers – 8898 of the 10915 ASes (82.5% of ASes)– Ex: UC Berkeley (25), Princeton (88), AT&T

Labs (6431), and INRIA (1300)

• Next lowest level: Regional ISPs– Leaf nodes after successive pruning of leaf

nodes– 971 ASes of the 10915 ASes (8.9% of ASes)– Ex: PacBell (5676), US West (6223), and UUNET

Canada (815)

• Remaining 1046 ASes: Core

AS Relationship Graph (2002), Continued

• Dense core: Tier-1 providers– (Nearly) fully-connected nodes with no providers– Around 15-20 ASes in a near-clique– Ex: Sprint, UUNET, AT&T, Verio, Level3, C&W,…

• Transit core: – ASes that peer with the dense core and each other– 129 ASes, including top providers in Europe and Asia– Ex: UUNET Europe, KDDI, and Singapore Telecom

• Outer core– All of the remaining ASes in the core– 897 ASes, including large regional and national ISPs– Ex: Turkish Telecom and Minnesota Regional Network

Node Degree is Not Enough

• Node degree ignores relationships– A stub AS may have many upstream providers– A core AS may have a small number of peers– Some ASes have customers that don’t have AS

numbers

Ideas for Class Projects

• AS relationship inference– New algorithms for inferring AS relationships– Longitudinal study of AS relationship graph– Influence of policies on measuring AS graph

• AS peering policies– Analysis of incentives to peer or not– Study of how one AS can game another– Analysis of whether regulation is necessary to

keep large ASes from locking out smaller ones

• Alternate settlement models– Associating prices with routes?– Source-based routing with third-party control?

Peering Policies

• Contracts that outline:– Operational requirements on peer network– Backbone and peering capacity requirements– Number and geographic diversity of peering

points– Volume and ratio of traffic between two peers– Routing-policy requirements

• AOL’s Settlement-Free Interconnection Policy– http://www.atdn.net/settlement_free_int.shtml

AOL’s Settlement-Free Interconnection Policy

• Operational requirements on a peer network– Handle a single-node outage w/o traffic impact– Single AS number– Network Operations Center staffed at all times

• Backbone capacity– At least 10 gigabits/sec between 8 or more cities– Minimum peering link speed of 622

megabits/sec

• Peering locations (in U.S.)– At least four locations– Must include D.C. area, middle of country, Bay

area, and NYC or Atlanta

Efficient Early-Exit Routing

• Diverse peering locations– Both costs, and

middle

• Comparable capacity at all peering points– Can handle even load

• Consistent routes– Same destinations

advertised at all points

– Same AS path length for a destination at all pointsCustomer A

Customer B

multiplepeeringpoints

Provider A

Provider B

Early-exit routing

AOL Routing Requirements

• Consistent advertisements– All customer routes– At all peering points– With the same AS path length

• Address blocks– Routes aggregated as much as possible– No address blocks smaller than /24– Address blocks are registered (e.g., with

ARIN)

• No default routing– Only send traffic to destinations AOL

advertises

For Next Tuesday’s Class

• Adapting routing inside an AS to the traffic– “The Revised ARPANET Routing Metric” (1989)– “Traffic Engineering With Traditional IP Routing

Protocols” (survey paper, 2002)• Written one-page review of first paper

– Brief summary of the paper– Reasons to accept the paper– Reasons to reject the paper– Suggestions for future research directions

• Just to skim…– RFC 3272: “Overview and Principles of Internet

Traffic Engineering”