Internet Routing (COS Internet Routing (COS 598A)598A)

Today: Router ConfigurationToday: Router Configuration

Jennifer RexfordJennifer Rexford

http://www.cs.princeton.edu/~jrex/teaching/http://www.cs.princeton.edu/~jrex/teaching/spring2005spring2005

Tuesdays/Thursdays 11:00am-12:20pmTuesdays/Thursdays 11:00am-12:20pm

Outline

• Individual router– Router CPU and interfaces– Links, adjacencies, and sessions– Filtering and injecting routes

• Network routing design– Case study of an ISP backbone– BGP community to convey state

• Current state of the art– Problems with today’s languages– Static analysis to detect mistakes– Template-based automation

Configuring the Router CPU

• Loopback interface– IP address for accessing the CPU

• Access control for router CPU– Command-line interface and SNMP set/get– Id and password, and levels of access

permission

• Default parameters– E.g., TCP parameters

• Services– E.g., logging, telnet, TFTP, etc.

Access Interfaces: The Basics

• High-level information– Media (e.g., Serial, Packet-Over-Sonet, and

ATM)– Location (e.g., slot 10, port adaptor 1, port 0)– Description (i.e., a comment field)– Capacity (i.e., bandwidth)

• Diverse communication media at layer 2– Serial link, ATM, packet over SONET, etc.– Various low-level, media-specific parameters

• Addressing and routing– IP address and network mask – Static route to map destination prefix to

interface

Access Interfaces: Layer-3 Resource Allocation

• Access control– Filtering of IP packets

• Based on packet-header bits (e.g., IP addresses, type-of-service bits, port numbers, and protocol)

• Buffer management– Maximum queue size– Parameters for Random Early Detection

(RED)• Link scheduling

– Mapping of packets to queues• Based on packet-header bits

– Allocation of bandwidth resources• E.g., weights for class-based queuing

Example of Statically-Routed Customer

hostname big-fat-router

interface Loopback0

ip address 12.123.5.240 255.255.255.255

!

interface Serial10/1/0/12:0

description Static Customer

ip address 12.7.35.1 255.255.255.252

ip access-group 666 in

!

ip route 12.34.158.0 255.255.254.0 Serial10/1/0/12:0

access-list 666 permit 12.34.158.0 0.0.1.255

access-list 666 permit 12.7.35.0 0.0.0.3

Static route for 12.34.158.0/23

Allow incomingpackets with sourcein 12.34.158.0/23or 12.7.35.0/30Implicit “deny” at the end…

Link Between Two (or More) Routers

• A link is just a (very small) network– Single subnet (e.g., 12.7.35.0/30)

• IP addresses for each interface– One on each router

• Special IP addresses– Broadcast address– Network address

12.123.5.240 12.7.108.312.7.35.0/30

12.7.35.1 12.7.35.2

Intradomain Routing Protocol

• Interface participation– Enable interface participate in OSPF– Assign an OSPF weight (and area)

• Link participation– All interfaces enabled in OSPF, in same area– Triggers them to establish an adjacency

• Network of OSPF-speakers– Routers flood the link-state advertisements– Each router constructs view of the OSPF

topology

BGP Session With a Neighbor

• BGP session– AS number of the neighbor– TCP connection between two IP addresses

• Reaching the remote end of the connection– Local router must be able to reach the neighbor router– Need to be able to route to establish the session!

• Diversity of configurations– One link vs. multiple links– Direct connection vs. intermediate components

LR NR LR LRNR NR

firewall, NAT, etc.multiple access linkssingle access link

Session With Directly-Connected Interface

• BGP session with the other end of the link– Single link directly to the neighbor’s router– Local router knows how to reach the address

12.7.35.2

LR

NR

12.7.35.1BGP

session

interface Serial10/1/0/12:0

ip address 12.7.35.1 255.255.255.252

!

router bgp 7018

neighbor 12.7.35.2 remote-as 18585

neighbor 12.7.35.2 <bgp command>…

!

AS 7018

AS 18585

Static Routes to Remote BGP Speaker

• BGP session associated with static routes– Multiple access links to the neighbor’s router– Or, intermediate components on the path

BGP session

NR

interface POS7/0

ip address 12.7.35.1 255.255.255.252

!

interface POS8/0

ip address 12.7.45.1 255.255.255.252

!

router bgp 7018

neighbor 12.7.108.3 remote-as 18585

neighbor 12.7.108.3 <bgp command>…

!

ip route 12.7.108.3 255.255.255.255 POS7/0

ip route 12.7.108.3 255.255.255.255 POS8/0

12.7.108.3

LR

POS7/0 POS8/0

Filtering BGP Route Advertisements

• E.g., filter routes not belonging to neighbor

• E.g., filter routes for martian IP

addresses

router bgp 7018

neighbor 12.7.108.3 remote-as 18585

neighbor 12.7.108.3 distribute-list CUSTOMER in

!

ip prefix-list CUSTOMER seq 5 12.34.158.0/23

ip prefix-list CUSTOMER seq 15 135.207.0.0/16

router bgp 7018

neighbor 137.39.3.128 remote-as 701

neighbor 137.39.3.128 prefix-list MARTIAN in

!

ip prefix-list MARTIAN seq 5 0.0.0.0/0

ip prefix-list MARTIAN seq 15 10.0.0.0/8 le 32

…

Introducing Routes Into BGP

interface Serial10/1/0/12:0 description Static-routed customer ip address 12.7.35.1 255.255.255.252 ip access-group 666 in!

router bgp 7018

network 12.34.0.0 mask 255.255.0.0

redistribute static

!ip route 12.34.158.0 255.255.254.0 Serial10/1/0/12:0

• Introducing routes into BGP– Explicit announcement (“network”)– Redistribution from other protocols

Network Routing Design

Network Routing Design: ISP Backbone

• Inside the network– OSPF

• Compute paths between routers

– Internal BGP• Distribute BGP routes inside

• Periphery of the network– Packet filters

• Block incoming packets at entry points

– Static routes• Learn how to reach non-BGP customers

– External BGP• Exchange reachability information with BGP

neighbors

Inside the Network: Interior Gateway Protocol

• OSPF: all internal links participate– Area structure driven by Point-of-Presence

• Backbone area (area 0): inter-PoP links• Non-backbone areas: intra-PoP links

– Weights driven by topological properties• Physical distance• Link bandwidth

20

10

10 15

PoP

Inside the Network: Internal BGP

• iBGP: all routers participate– Route-reflector hierarchy driven by PoP

• Route reflectors: two RRs per Point-of-Presence• RR clients: connect to both RRs in their PoP

– Full mesh of top-level route reflectors

PoP

Periphery of the Network: Packet Filters

• Permit valid customer to send (prevent spoofing)– Permit source addresses belonging to customer

• Permit routing protocols and management to work– Permit source address of remote BGP speaker– Permit customer to “ping” your end of the link

• Prevent bogus IP addresses– Deny source addresses of “martians”– Deny source address of your own services

• Prevent access to your own equipment– Deny packets destined to routers from unexpected

source

• Prevent unwanted applications/services– Deny SNMP port number or multicast addresses– Deny BGP port number from unexpected source

Periphery of Network: External BGP

• External BGP sessions– Session to each BGP-speaking neighbor– Import and export policy for each session

• Policy mechanisms– Filtering: discard route announcements– Preference: assign high/low preference– Tagging: mark routes with extra state

• Policy goals– Business relationships– Traffic engineering– Route aggregation

Business Relationships: Assigning Preference

route-map INPEER permit 100set local-preference 80

!route-map INCUST permit 100

set local-preference 90!

• Prefer-customer policy– Session with peer

• Import policy: assign local-pref of 80

– Session with customer• Import policy: assign local-pref of 90

Business Relationships: Tagging for Memory

• Tag to remember the type of route– Session with peer: community 0:1000– Session with customer: community 0:2000

route-map INPEER permit 100set local-preference 80set community 0:1000

!route-map INCUST permit 100

set local-preference 90set community 0:2000

!

Business Relationships: Filtering Based on Tag

• Export policy based on tag– Export all routes to customers– Export only customer-learned routes to peers

route-map INPEER permit 100set local-preference 80set community 0:1000

!route-map OUTPEER permit 100

match community 0:2000 !route-map INCUST permit 100

set local-preference 90set community 0:2000

!route-map OUTCUST permit 100

match community 0:1000 0:2000!

Filter routesfrom peers

Periphery of Network: Static Routes

• Static route for destination prefix– Reaching destinations of a non-BGP customer– Reaching the remote end of eBGP session

• Tagging to limit export of static routes in BGP– Local to the router: no injection in BGP

• Prefix contained in block allocated to provider’s router• … and customer connects to Internet in only one place

– Just inside the AS: inject with “no export”• Prefix contained in provider’s address space• … and customer connects to only one provider

– Both in and out of AS: inject with no restriction• Prefix not contained in provider’s address space• … or customer connects to multiple providers

Complexity of BGP Policies

• Remembering from one router to another– Business relationship for export policy– Scope of prefix for route-aggregation– Geographic location

• Side information unknown to routers– Business relationship with neighbor AS?– Customer’s prefix falls in provider’s supernet?– Customer has multiple access links or

providers?

• Intrinsic complexity and diversity of policies– Business relationships, traffic engineering,

route aggregation, security, etc.

Current State of the Art

Manual Configuration

• Dangerous– Typo in a routing policy: black hole– Interfaces in different OSPF areas: no traffic on link– Missing a packet filter: denial-of-service

vulnerability

• Expensive– Delays in deploying equipment– Hiring and training skilled engineers– Lock-in to the router vendor

• Disruptive– Half of network outages (Yankee Group)– Failures of Internet services (USITS’03)– BGP routing anomalies (SIGCOMM’02, NSDI’05)

The Problem With Configuration Languages

• Heterogeneity– No standard language for the industry– Differences across products and versions by a vendor– Change over time due to new protocols and features

• Low-level language– Thousands of different commands– Complex inter-relationships between commands– Very little abstraction or nesting– Sometimes no public grammar or parser

• Poor commit semantics– Command-line interface, one command at a time– Often no support for atomic actions

Problem With Configuration Languages

• Router-level configuration– Configuration of individual routers, not the network– E.g., configure each end of a link separately

• Complexity of distributed protocols– Some things are hard to do in distributed fashion– E.g., ensuring complete visibility of routes in iBGP– E.g., the need for BGP communities to pass state

• Lack of abstractions– Research emphasis on speed and features– … not on simplicity, managability, and clean

abstractions– Config languages are no better (though often worse!)

than our abstractions for mechanisms, protocols, and practices

Reducing Configuration Errors

• Configuration checking tools– Dump the running configuration of all routers– Parse, join, and query the data– Generate reports of problems

• Automated configuration– Templates and rules for generating

configuration– Database for storing the values of variables– Software to fill templates and apply commands

• Better configuration languages– Vendor neutral languages for routing policy

(RPSL)– Research on network-wide configuration

Configuration Checking

Configuration Checking: Types of Checks

• Errors: clear mistakes– Variables used but not defined– Mismatch between two ends of link or session

• Warnings: risky behavior– Variables defined but not used– Dependence on default configuration values– Violations of best common practices

• Inconsistencies: pattern mismatches– Same variable defined differently on two routers– Variables with different names defined same way

• Local policy violations: – Mismatch with the operator’s explicit intent

Configuration Mistakes: BGP Prefix Filtering

• Martian prefixes– Goal: block announcements for bogus prefixes– Variable: prefix-list define martian prefixes– Instantiation: eBGP sessions apply the prefix-

list

• Configuration mistakes– Error: MARTIAN instantiated but not defined– Warning: MARTIAN defined but not instantiated– Inconsistency: MARTIAN defined differently on

different routers– Local policy violation: MARTIAN not defined as

0.0.0.0/0, 10.0.0.0/8-32, 127.0.0.0/8-32, etc.

Configuration Mistakes: iBGP Configuration

• Internal BGP sessions– Goal: disseminate reachability info inside the AS– Mechanism: iBGP sessions between routers

• Configuration mistakes– Error: Router A has iBGP session to Router B,

but B does not have an iBGP session to A– Warning: iBGP session between non-Loopback

addresses– Inconsistency: all routers have two route

reflectors (RRs), except one– Local policy violations: all client routers should

have two RRs; route RRs should form a full mesh

Automated Configuration

Automated Configuration System

DB interface Serial10/1/0/12:0 description CBB Customer ip address 12.7.35.1 255.255.255.252 ip access-group 666 in!router bgp 7018 neighbor 12.7.35.2 remote-as 18585 neighbor 12.7.35.2 route-map CUST-FACE in neighbor 12.7.35.2 route-map FULL-ROUTES out neighbor 12.7.35.2 distribute-list 13 in neighbor 12.7.35.2 soft-reconfiguration-inbound!

What is your AS number?What export policy do you want?Do you want a dynamic default?What are your address blocks?Do you need to receive communities?

query

template

RULES

configlet

Technical Questions (TQ)

interface <name> description <cust name> ip address <addr> <mask> ip access-group <acl> in!router bgp 7018 neighbor <ip> remote-as <asn> neighbor <ip> route-map CUST-FACE in neighbor <ip> route-map <outmap> out neighbor <ip> distribute-list <racl> in neighbor <ip> soft-reconfiguration-inbound [neighbor <ip> send-community]!

router

• Automation through template and database• … but doesn’t raise the level of abstraction• So building these kind of systems is challenging

Automation Stages

• Initializing a new router– Base configuration to load on the router– E.g., access control for the router CPU– E.g., enabling of services (e.g., telnet, logging)– E.g., defining MARTIAN lists and routing policies

• Changing the configuration– Use cases for different activities– … with TQ, database variables, and template– E.g., “add link,” “add BGP-speaking customer,”

or “move static-route customer to BGP customer)

– Generates a configlet applied to the router

Conclusion

• Router configuration is hard– Low-level configuration languages– Wide variety of protocols and mechanisms

• Improving on manual configuration– Configuration-checking tools– Automated configuration systems– Research needed on languages and

abstractions• Possibility of auto-configuration?

– Specify high-level network design– Ship a router and plug it in to the network– Router auto-configured from a server

Next Time: Removing Routing from Routers

• Three short papers– “Routing as a Service”– “The Case for Separating Routing From Routers”– “Network-Wide Decision Making: Toward a Wafer-

Thin Control Plane”

• Review just of the first paper– Summary– Why accept– Why reject– Directions for future work

• Plan for a discussion-driven class on Thursday• Reminder: no class next week!!!