1 © 2003 Cisco Systems, Inc. All rights reserved. MPLS VPN Inter-AS, 12/03 INTER-AUTONOMOUS SYSTEM MPLS VPN: ADVANCED CONCEPTS December 2003

1© 2003 Cisco Systems, Inc. All rights reserved.

MPLS VPN Inter-AS, 12/03

INTER-AUTONOMOUS SYSTEM MPLS VPN: ADVANCED CONCEPTSDecember 2003



• Routing between sub-autonomous systems

• Inter-AS scaling

• Inter-AS filtering and route distribution

• Load balancing

• RT rewrite

• Services in Inter-AS

• Inter-AS and CSC comparison

• Inter-AS Summary

Agenda



ROUTING BETWEEN SUB-AUTONOMOUS SYSTEMS

33© 2003 Cisco Systems, Inc. All rights reserved.Presentation_ID



• Separate IGPs

Each sub-confederations runs a single IGP

• Route-reflectors are used as peering points between sub-confederations for better scaling

• Next-hop self done by border routers on eBGP and iBGP sessions towards intra-confederation peers

Confederation Multiple IGP Domains



PE-1

CEGBP-1

CE-2

CEGBP-2

PE-3

CE-1

PE-2

CE-5

CE-4CE-3

Core of P LSRs

Core of P LSRs

Confederation

Sub-AS1 withIGP-1

Sub-AS2 with IGP-2

MP-eBGP intra confederationfor VPNv4 routes with label distribution

PEs exchange VPNv4 addresses with labels

Next-hop and labels are changed (next-hop self is used)

PE1 and PE-2 addresses are known in both IGPs

MP-iBGP

Confederation Multiple IGP Domains



PE-1

CEGBP-1

CE-2

CEBGP-2

PE-3

CE-1

PE-2

CE-5

CE-4

CE-3

Core of P LSRs

Core of P LSRs

Confederation

Sub-AS1 withIGP-1

Sub-AS2 with IGP-2

Network=NNext-hop=CE2

Network=NNext-hop=PE3

Network=RD1:NNext-hop=PE1Label=L1

Network=RD1:NNext-hop=RR1Label=L2

Network=RD1:NNext-hop=RR2Label=L3

Confederation Multiple IGP Domains (Cont.)



• Route reflectors exchange routes

Using Route reflectors is a natural approach since they already have all VPN routes

• Next-hop-self choices

Option-1: eBGP only

Option-2: eBGP and iBGP on border routers

• When next-hop self is used on both iBGP and eBGP sessions (in CEBGP-1 and CEBGP-2) the topology is similar to a Multi-provider-VPN topology

Confederation Multiple IGP Domains:Important Points



Confederation Multiple IGP Domains:Important Points (Cont.)

• CEBGP-1 and CEBGP-2 each need to be known in both IGPs

• CEBGP-1 and CEBGP-2 use interface addresses for their BGP session

• Label has to be bound on peer address; single label is used between sub-confederations

• Neighbor route needs to be known either a static router, or by using PPP neighbor-route discovery

• Implementation will create a neighbor route for the BGP peer address



SCALING INTER-PROVIDER SOLUTIONS

99© 2003 Cisco Systems, Inc. All rights reserved.Presentation_ID



PE-ASBR Memory Consumption

VPNv4 MP-iBGP Sessions

PE-ASBR Memory

No. VPN Routes

Memory Consumption



PE-ASBR Memory Scaling

• Potentially large amounts of VPN routing information that may not need to be carried on PE-ASBRs

Large percentage will be local VPN prefixes

• PE-ASBRs must hold relevant VPN routing information such as VPN prefix details

• Two methods available to aid scaling

ARF with local VRF import

ARF disabled with inbound filtering



ARF with Local VRF Import

• Automatic Route Filtering (ARF) for non-imported routes

If RT does not match locally configured import statement then drop the route

• Each PE-ASBR holds VRFs for Inter-AS VPNs and imports routes based on RT values

• PE-ASBR acts like normal PE routers with MP-eBGP sessions



BGP Memory

VRFsCEF Memory

MPLS Memory

Routing Table Memory

MP-iBGP VPNv4

Au

t om

ati c Ro

ute

Filt er in

g

BGP, CEF, MPLS & RT Memory per-VRF

ARF with Local VRF Import (Cont.)



ARF Disabled With Inbound Filtering

• Automatic Route Filtering (ARF) enabled by default

if no VRFs are configured then ALL VPN routes are dropped by the PE-ASBR

• Automatic Route Filtering may be disabled with no default BGP route-target filter command within the BGP configuration

• Disabling of ARF will cause ALL routes to be accepted by the PE-ASBR

Additional filtering mechanisms should be used to drop unwanted routes



BGP MemoryMP-iBGP VPNv4

NO

Au

t om

a ti c Ro

ute

Filt er in

gNO per-VRF CEF or RT Memory, only BGP & LFIB

router bgp 1

!

no bgp default route-target filter

!

address-family vpnv4

neighbor 154.27.0.134 activate

neighbor 154.27.0.134 send-community extended

neighbor 154.27.0.134 route-map vpn-routes-filter in

LFIB Memory

VRF & CEF memory not required

Routing Table memory not required

ARF Disabled With Inbound Filtering (Cont.)



Next-Hop-Self Effect On LFIB

BGP Memory 1000 prefixes

MP-iBGP VPNv4

Next-hop-self increase amount of LFIB entries on receiving PE-ASBR

LFIB Memory 1000 prefixes

With NHS Without NHS


LFIB Memory 1000 prefixes


LFIB memory 1 prefix for BGP next-

hop

1000 prefixes in total



FILTERING AND ROUTER DISTRIBUTION MECHANISMS



Various Filtering Points In Inter-AS

PE

PE

RR

2. Outbound filtering per-peer

4. Inbound filtering per-peer OR rr-group

1. Inbound filtering on PE-ASBR

3. Automatic route filtering inbound

AS #100

AS #200RR

RR

AS #300

PE

5. Automatic route filtering inbound



Inbound Filtering On PE-ASBR

BGP MemoryRT 214:27

NO

Au

t om

a ti c Ro

ute

Filt er in

gNO ARF – Filter inbound on per-peer basis

router bgp 1

!

no bgp default route-target filter

!




neighbor 154.27.0.134 route-map vpn-routes-filter in

!

ip extcommunity-list 1 permit rt 214:27 rt 214:94

!

route-map vpn-routes-filter permit 10

match extcommunity 1

LFIB MemoryRT 214:94

Blue VPN routes discarded

RT 214:129



Outbound Filtering On PE-ASBR

BGP Table


neighbor 157.27.0.132 route-map MPeBGP-2 out


!

route-map MPeBGP-2 permit 10

match extcommunity 214:27

!


match extcommunity 214:94

RED VPN

GREEN VPN

AS #300

AS #200



Downstream RT Allocation

• Inbound and outbound filtering are restrictive with a large number of VPN clients

Each RT must be known, and the filters must be established

• Changes to VPN client membership will cause configuration changes on PE-ASBRs

Each filter must be updated to reflect the addition/deletion of VPN clients

• Simplified filtering scheme is needed with a large number of clients

Provided with “downstream provider RT allocation” scheme



RED VPN

AS #300

RT 129:102

AS #100

GREEN VPN

RED VPN RT 129:12090

GREEN VPN RT 129:12001




neighbor 154.27.0.134 route-map asbr-routes-filter in



!

ip extcommunity-list 1 permit rt 129:101 rt 129:102

ip extcommunity-list 16 permit rt 129:101

ip extcommunity-list 17 permit rt 129:102

Export RT 129:12090 RT

129:102

Export RT 129:12001 RT

129:101

AS #200

RT 129:101

route-map asbr-routes-filter permit 10


!



!



Downstream RT Allocation (Cont.)



LOAD BALANCING: DISTRIBUTION OF TRAFFIC LOAD BETWEEN PROVIDERS



• Balancing of Inter-AS traffic is an important issue for distribution of traffic and redundancy of network design

• All Inter-AS traffic must pass through PE-ASBRs

As BGP next-hops are reachable via these routers

• Multiple links provide traffic distribution

These do not provide redundancy due to single point of failure of the PE-ASBR

Load Balancing Between Backbones



VPN Client Traffic Flow

PE-1 PE-2

VPN-B

CE-2 CE-3

VPN-B

PE-ASBR-1 PE-ASBR-2

152.12.4.0/24

BGP, OSPF, RIPv2 152.12.4.0/24,NH=CE-2

BGP, OSPF, RIPv2 152.12.4.0/24,NH=CE-2

VPN-v4 update:RD:1:27:152.12.4.0/24,

NH=PE-1RT=1:222, Label=(L1)

VPN-v4 update:RD:1:27:152.12.4.0/24,


VPN-v4 updates: NH=PE-ASBR-1VPN-v4 updates: NH=PE-ASBR-1


ALL Inter-AS traffic flows across PE-

ASBR-2 to PE-ASBR-1 link

VPN Client to VPN Client traffic flow via Inter-AS Link



Load Balancing Between PE-ASBRs

PE-ASBR-1

Network Y

BGP NH=PE-ASBR-2 LO0

Network Y

BGP NH=PE-ASBR-2 LO0

Network Y

PE-ASBR-2

Loopback Interface Loopback Interface

BGP peering (Multi-HOP MP-eBGP) between

loopbacks

Routing Table

PE-ASBR-2 LO0 via 193.1.1.9

via 193.1.1.13

via 193.1.1.17

Routing Table

PE-ASBR-2 LO0 via 193.1.1.9

via 193.1.1.13

via 193.1.1.17

193.1.1.9

193.1.1.13

193.1.1.17

Load Balancing across multiple PE-ASBR links

Static’s or IGP AND LDP



Redundant PE-ASBR Connections

PE-ASBR-1 PE-ASBR-2

PE-ASBR-3 PE-ASBR-4

PE-1





RR will choose BGP best path and advertise only this path to receiving

clients


VPN-v4 update:RD:1:27:152.12.4.0/24,


VPN-v4 update:RD:1:27:152.12.4.0/24,


VPN-B VPN-B

Inter-site traffic flow

Redundant PE-ASBR used purely for backup



Redundant PE-ASBR Load Balancing

PE-ASBR-1 PE-ASBR-2

PE-ASBR-3

PE-ASBR-4

PE-1





iBGP multipath support provides ability to load

balance between two exit points

VPN-v4 update:RD:1:27:152.12.4.0/24,


VPN-v4 update:RD:1:27:152.12.4.0/24,


VPN-BVPN-B

Load balancing PE-ASBR links without Route Reflectors

Network 152.12.4.0/24

BGP NH=PE-ASBR-2

PE-ASBR-4

Network 152.12.4.0/24

BGP NH=PE-ASBR-2

PE-ASBR-4



RT REWRITE



RT Rewrite

• RTs identify the VRF routing tables into which the prefix carried by the update is to be imported

Carried as extended community attributes in bgp-vpnv4 updates

• RT RewritesSupported for VRF export-maps

Allow the replacement of route-targets on incoming and outgoing BGP updates

Enables Service Providers to customize Route Targets within their network

RT replacement can be performed at ASBRs exchanging VPNv4 prefixes

RTs can also be replaced by PEs or RRs



RT Rewrite Memory and Performance Impact

• Memory impact should be insignificant, as it modifies the update itself without requiring storage

Other transient memory requirements are minimal

• Performance impact will depend on the product of the number of updates and the size (length, depth) of the route-map

• To perform RT replacement, each extended-community list is examined while matching and again while deleting the RT



RT Rewrite Sample ConfigurationReplace RT X with Y

• Use BGP inbound or outbound route-map at the receiving PE(ASBR, RR):

ip extcommunity-list <X> permit rt c:d

!

route-map extmap permit <#1>

match extcommunity X

set extcomm-list <X> delete

set extcomm-list <Y> additive

<!continue #2 to the next route-map if have more

RT to change. Can use c:* for additional RTs>

!

address family vpnv4

neighbor <ASBR IP#> route-map extmap <in/out>



RT Rewrite Verification Commands

• Verify route target replacement

show ip bgp vpnv4 [all]

• Verifying the Route Target Replacement Policy

debug ip bgp updates <ASBR IP Address>



SHARED SERVICES IN INTER-AS



Supported Shared Services in Inter-AS

• Network Address Translation

Address Translation at the egress point of the peering Service Provider is possible

• Redundancy (HSRP, VRRP, GLBP)

Two ASBRs will reside in a single SP network

• IP Address Management and assignment

DHCP, ODAP will be supported for Inter-AS

• Security

AAA Servers

• Troubleshoot/Management

Ping, Traceroute, SAA, Netflow



INTER-AS VERSUS CARRIER SUPPORTING CARRIER



CSC versus Inter-AS

Carrier Supporting Carrier• Opportunity: Offer backbone services to peer or smaller carriers

Inter-Provider Access• Opportunity: Provide carrier services on behalf of other carriers

Backbone Carrier

CustomerCarrier A

POP1

Carrier A

Carrier BCustomerCarrier A

POP2



CSC versus Inter-AS (Cont.)

CSC Inter-AS

Client-server topologies Peer-to-peer topologies

ISP or MPLS VPN provider is a customer of another MPLS VPN backbone provider

Two ISPs peer up providing services to some of the common customer base

MPLS VPN backbone services needed between the same carrier POPs

Single SP POPs not available in all geographical areas required by their

customers

Subscribing service provider may or may not have MPLS enabled

Participating Providers must supportMPLS VPNs

Customers sites do not distribute reachability information to the backbone carrier

Customers sites distribute reachability information directly to the participating

service providers

MPLS VPN in a BGP confederation



INTER-AS SUMMARY



• Service Providers have deployed Inter-AS for:

Scalability purposes

Partitioning the network based on services or management boundaries

• Some contract work is in progress amongst Service Providers to establish partnership and offer end-end VPN services to the common customer base

• Service Provider networks are completely separate

Do not need to exchange internal prefix or label information

• Each Service Provider establishes a direct MP-eBGP session with the others to exchange VPN-IPv4 addresses with labels

• /32 route to reach the ASBR is created by default so ASBRs can communicate without a need for IGP

Must be redistributed in the receiving Service Provider’s IGP

Inter-AS Summary



• IGP or LDP across ASBR links is not required

Labels are already assigned to the routes when exchanged via MP-eBGP

Interface used to establish MP-eBGP session does not need to be associated with a VRF

• Direct eBGP routes and labels can be exchanged.

• Next-Hop self can be turned on on ASBRs, enabling the ASBR to use its own address for next-hop

• Using the next-hop self requires an additional entry in the TFIB for each VPNv4 route (about 180) bytes

• If the Service Provider wishes to hide the Inter-AS link then use the next-hop-self method otherwise use the redistribute connected subnets method

Inter-AS Summary (Cont.)



• Multi-hop MP-eBGP sessions can be passed between Service Providers without conversions to VPNv4 routes

• Configuration of VRFs is not required on the ASBRs because bgp default route-target filter (automatic route filtering feature) has been disabled

• To conserve memory on both sides of the boundary and implement a simple form of security, always configure inbound route-maps to filter only routes that need to be passed to the other AS

Inter-AS Summary (Cont.)



References

• Inter-AS for MPLS VPNs CCO Documentation:

www.cisco.com/univercd/cc/td/doc/product/software/ios121/121newft/121t/121t5/interas.htm

• MPLS and VPN architectures Jim Guichard/Ivan Pepelnjak ISBN 1-58705-002-1:

www.ciscopress.com/book.cfm?book=168

• Support for Inter-provider MPLS VPN ENG-48803 Dan Tappan, (internal only)



Documents

1 © 2003 Cisco Systems, Inc. All rights reserved. MPLS VPN Inter-AS, 12/03 INTER-AUTONOMOUS SYSTEM MPLS VPN: ADVANCED CONCEPTS December 2003