RFC 2547bis: BGP/MPLS VPN Hierarchical and Recursive ... · BGP/MPLS VPNs, refer to the RFC 2547bis: BGP/MPLS VPN Fundamentals white paper. Perspective: BGP/MPLS VPNs Defined In traditional

White Paper

RFC 2547bis: BGP/MPLS VPNHierarchical and Recursive Applications

Juniper Networks, Inc.1194 North Mathilda AvenueSunnyvale, CA 94089 USA408 745 2000 or 888 JUNIPERwww.juniper.net

Part Number : 200014-0003 07/01

Chuck SemeriaMarketing Engineer

Contents

Executive Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Perspective: BGP/MPLS VPNs Defined . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5Carrier of Carriers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

Internet Service Provider as a Customer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Network Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Control and Data Flows . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Control Flow: LSP Establishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Control Flow: Distribution of Routing Information . . . . . . . . . . . . . . . . . . . . . . . . . . 9Data Flow: Forwarding User Traffic from ASBR_1 to 20.2/16 . . . . . . . . . . . . . . . . 15Observations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

A BGP/MPLS VPN Service Provider as a Customer . . . . . . . . . . . . . . . . . . . . . . . . . . . 18Network Topology . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Control Flow: LSP Establishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Control Flow: Distribution of Routing Information . . . . . . . . . . . . . . . . . . . . . . . . . 21Data Flow: Forwarding User Traffic from PE_3 to 10.2/16 . . . . . . . . . . . . . . . . . . 28

Multi-AS BGP/MPLS VPN Operations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31EBGP VRF-to-VRF Connections Between AS Border Routers . . . . . . . . . . . . . . . . . . . 32MP-EBGP Distribution of Labeled VPN-IPv4 Routes Between AS Border Routers . 33Multi-hop MP-EBGP Distribution of Labeled VPN-IPv4 Routes

Between PE Routers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Example #1: Multi-AS Operations Using a BGP/MPLS VPN Capable

Transit Provider . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34Example #2: Multi-AS Operations Using an MPLS Capable Transit Provider . . 37Example #3: Multi-AS Operations Using Direct Interconnect . . . . . . . . . . . . . . . . 39

Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41Acronyms . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

Copyright © 2001, Juniper Networks, Inc.

List of Figures

Figure 1: BGP/MPLS VPN Routing Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6Figure 2: Hierarchy of BGP/MPLS VPNs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7Figure 3: Network Topology with ISP as a Customer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Figure 4: Control and Data Flows in the Case Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8Figure 5: LSP Established from PE_1 to PE_2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9Figure 6: External Route Flooded Across Site 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10Figure 7: CE_2 Advertises Site 2 Internal Routes to PE_2 . . . . . . . . . . . . . . . . . . . . . . . . . . 10Figure 8: PE_2 Distributes Site 2’s Internal Routes to PE_1 . . . . . . . . . . . . . . . . . . . . . . . . . 11Figure 9: PE_1 Distributes Site 2’s Internal Routes to CE_1 . . . . . . . . . . . . . . . . . . . . . . . . . 12Figure 10: CE_1 Distributes Site 2’s Internal Routes to all Routers in Site 1 . . . . . . . . . . . 13Figure 11: CE_1 Distributes Site 2’s Internal Routes to all Routers in Site 1 . . . . . . . . . . . 14Figure 12: Forwarding Table Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Figure 13: Forwarding Across Customer Site 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15Figure 14: Forwarding Across BGP/MPLS VPN Service Provider Backbone . . . . . . . . . . 16Figure 15: Forwarding Across ISP Customer Site 2 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17Figure 16: Topology Differences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19Figure 17: Network Topology — BGP/MPLS VPN Provider as a Customer . . . . . . . . . . 20Figure 18: LSP Establishment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20Figure 19: Route to PE_4 is Advertised Across Customer Site 2 to CE_2 . . . . . . . . . . . . . 22Figure 20: CE_2 Advertises Customer Site 2’s Internal Routes to PE_2 . . . . . . . . . . . . . . . 23Figure 21: PE_2 Distributes Customer Site 2’s Internal Routes to PE_1 . . . . . . . . . . . . . . . 24Figure 22: PE_1 Distributes Customer Site 2’s Internal Routes to CE_1 . . . . . . . . . . . . . . . 25Figure 23: CE_1 Distributes Customer Site 2’s Internal Routes to all Routers in Site 1 . . 26Figure 24: PE Routers in Customer Site 1 Establish an MP-IBGP Session with PE_4 . . . 27Figure 25: Forwarding Table Information . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28Figure 26: Forwarding Across Customer BGP/MPLS Service Provider Site 1 . . . . . . . . . 29Figure 27: Forwarding Across Backbone BGP/MPLS VPN Service Provider Network . 30Figure 28: Forwarding Across Customer BGP/MPLS Service Provider Site 2 . . . . . . . . . 31Figure 29: VRF-to-VRF Connections at AS Border Routers . . . . . . . . . . . . . . . . . . . . . . . . . 32Figure 30: MP-EBGP Distribution of Labeled VPN-IPv4 Routes at ASBRs . . . . . . . . . . . . 33Figure 31: Multi-AS Operations: BGP/MPLS VPN Capable Transit Provider . . . . . . . . . 34Figure 32: Compare and Contrast BGP/MPLS VPN Capable Transit Provider . . . . . . . . 36Figure 33: Multi-AS Operations: MPLS Capable Transit Provider . . . . . . . . . . . . . . . . . . . 37Figure 34: Compare and Contrast MPLS Capable Transit Provider . . . . . . . . . . . . . . . . . . 38Figure 35: Multi-AS Operations: Direct Connection Between BGP/MPLS VPN Providers 39Figure 36: Compare and Contrast Directly Connected BGP/MPLS VPN Providers . . . . 40Figure 37: BGP/MPLS VPNs — Carrier-of-Carriers vs. Multi-AS Operations . . . . . . . . . 41


List of Tables

Table 1: Comparison of ISP Customer and BGP/MPLS VPN Customer . . . . . . . . . . . . . . 18


Executive Summary

The objective of this paper is to provide a detailed understanding of the operation of hierarchical and recursive BGP/MPLS VPNs as described in RFC 2547bis. As BGP/MPLS VPNs are deployed in the Internet, it is possible that the customer of a BGP/MPLS VPN service provider is another service provider rather than an enterprise customer. For this scenario, the customer service provider depends on the BGP/MPLS VPN service provider to deliver a transport service between the customer service provider’s POPs or regional networks. If all of the customer service provider’s sites have the same AS number, then the BGP/MPLS VPN transit service provider delivers a carrier-of-carriers service. If the customer service provider’s sites have different AS numbers, then the BGP/MPLS VPN transit service provider supports multi-AS operations.

This paper assumes that you have a basic understanding of BGP/MPLS VPNs as described in RFC 2547bis and an understanding of the operation of OSPF, IS-IS, BGP (both IBGP and EBGP), LDP, RSVP, and MPLS. If you are not familiar with the fundamental operation of BGP/MPLS VPNs, refer to the RFC 2547bis: BGP/MPLS VPN Fundamentals white paper.

Perspective: BGP/MPLS VPNs Defined

In traditional IP routing architectures there is a clear distinction between internal routes and external routes. From the perspective of an ISP, internal routes include all of the provider’s internal links (including BGP next hops) and loopback interfaces. These internal routes are exchanged with other routers in the ISP’s network via an interior gateway protocol (IGP) such as OSPF or IS-IS. All routes learned at Internet peering points or from customer sites are classified as external routes and are distributed via an exterior gateway protocol (EGP) such as BGP. In traditional IP routing architectures, the number of internal routes is much smaller than the number of external routes.

The traditional distinction between internal routes and external routes also applies to BGP/MPLS VPN routing architectures (Figure 1).

Copyright © 2001, Juniper Networks, Inc. . 5

RFC 2547bis: BGP/MPLS VPN Hierarchical and Recursive Applications

Figure 1: BGP/MPLS VPN Routing Architecture

The cloud of provider (P) routers maintains only service provider internal routes (to provider edge [PE] routers and other P routers, but it does not maintain VPN routes). PE routers are the only devices in the provider network that are required to maintain external routes.

The glue that binds external routing with internal routing is the BGP next hop.

■ The BGP next hop is advertised with each external route in BGP advertisements.

■ The route to the BGP next hop is an internal route that is advertised by the IGP.

■ MPLS provides packet forwarding from the ingress PE router to BGP next hop egress PE router.

When working with hierarchical BGP/MPLS VPN applications, keep three fundamental concepts in mind.

■ Each BGP/MPLS VPN customer in the hierarchy must identify which customer routes are internal and which customer routes are external.

■ Internal customer routes must be maintained by the BGP/MPLS VPN service provider in its PE routers.

■ External customer routes are carried only by the customer’s routers and not by the BGP/MPLS VPN service provider’s routers.

■ If customer sites belong to the same AS, then the customer’s external routes are exchanged via IBGP. This scenario is referred to as the Carrier-of-Carriers Model.

■ If customer sites belong to different autonomous systems, then the customer’s external routes are exchanged via EBGP. This scenario is referred to as the Multi-AS Operations Model.

In general, each provider in a BGP/MPLS VPN hierarchy is required to maintain its own internal routes in its P routers and the internal routes of its customers in its PE routers. By recursively applying this rule, it is possible to create a hierarchy of BGP/MPLS VPNs (Figure 2).

M160

M40M20

M10 M10

Site 3Blue VPN

Site 2Red VPN

Site 1Red VPN

CE CE

PE PE

Provider Network

Site 4Blue VPN

CE CE

P

P

P

6 Copyright © 2001, Juniper Networks, Inc.


Figure 2: Hierarchy of BGP/MPLS VPNs

In Figure 2, Customer Blue’s internal routes and Customer Green’s internal routes are also Provider Red’s external routes. Additionally, Customer Red’s internal routes are also Provider Black’s external routes.

Carrier of Carriers

At times, the customer of the BGP/MPLS VPN service provider might be the network of another service provider. There are two potential scenarios in the carrier of carrier’s operational model.

■ The VPN customer is an ISP that uses the BGP/MPLS VPN service provider’s backbone to obtain backbone connectivity for its geographically dispersed POPs or regional networks. In this situation, the customer might or might not elect to run MPLS within its POPs or regional networks.

■ The VPN customer is itself a BGP/MPLS VPN provider that offers RFC 2547bis VPN service to its customers. The customer BGP/MPLS service provider relies on the backbone BGP/MPLS service provider for intersite connectivity. In this scenario, the customer BGP/MPLS VPN service provider is required to run MPLS within its POPs or regional networks.

Internet Service Provider as a Customer

Our first example describes how a BGP/MPLS VPN service provider can use its backbone to provide inter-site connectivity for a customer that is itself an ISP. Assume that each customer site is either a POP or a regional network and that the VPN customer wants to interconnect its sites using the BGP/MPLS VPN service provider’s backbone.

Network Topology

Figure 3 shows the network topology for this case study. The BGP/MPLS VPN service provider has a customer that is an ISP with two POPs: Site 1 and Site 2. Note that in this example both POPs are shaded red to indicate that they represent different sites of the same ISP. The prefix 20.2/16 is a public Internet route that ASBR_2 in Site 2 learns from customer router R_Y.



Figure 3: Network Topology with ISP as a Customer

This case study describes how the prefix 20.2/16 is distributed across the BGP/MPLS backbone to ASBR_1 so that ASBR_1 can advertise the route to customer router R_X. After describing how the routing information is distributed from ASBR_2 to ASBR_1, the example shows how a packet addressed to host 20.2.1.2 is forwarded across the BGP/MPLS backbone from ASBR_1 to ASBR_2.

Control and Data Flows

Recall that the BGP/MPLS VPN operational model consists of two flows.

■ A control flow that distributes routing information and supports the establishment of label switched paths (LSPs) between PE routers.

■ A data flow that forwards customer data traffic from one VPN site to another VPN site.

Figure 4 shows the control and data flows that are needed to support the forwarding of traffic addressed to prefix 20.2/16 from ASBR_1 to ASBR_2. This diagram depicts a tremendous amount of information; you need not understand it now, but rather keep it in mind for future reference as this case study proceeds.

Figure 4: Control and Data Flows in the Case Study

M40M40 M160 M10M10M10 VRF VRF M10

CE_2CE_1 PPE_1R_1ASBR_1 R_2 ASBR_2PE_220.2/16

R_X R_Y

BGP/MPLS VPN BackboneISP POP(Customer Site 1)

ISP POP(Customer Site 2)

M40M40 M160 M10M10M10 VRF VRF M10

CE_2CE_1 PPE_1R_1ASBR_1 R_2 ASBR_2PE_2

DataFlow

Control Flow

IP IP IP IP IPMPLSMPLS

CustomerEBGP

CustomerIGP

CustomerEBGP

RoutingUpdate

ProviderLDP/RSVP

Routing Update & LDP

Customer IGP

Customer IBGPCustomer IBGPCustomer IBGP

20.2/16

Provider MP-IBGP



While this example does not demonstrate the use of route reflectors, operational deployment likely requires the installation of a route reflector at each POP to eliminate a complex mesh of IBGP connections. The routers within each POP become clients of the local route reflector, and the route reflectors establish an IBGP mesh among themselves to distribute the external IPv4 routes. Note that this example uses IBGP, not EBGP, because all of the customer POP sites are members of the same organization or AS.

Control Flow: LSP Establishment

The BGP/MPLS VPN service provider is aware of the LSPs that need to be established across its backbone to provide the required PE-to-PE connectivity. In this case study, the BGP/MPLS VPN service provider uses LDP or RSVP to establish an LSP from PE_1 to PE_2 (Figure 5). The Label = 200 shown at the ingress of the LSP is the initial label that PE_1 uses when it forwards a packet across the LSP to PE_2.

Figure 5: LSP Established from PE_1 to PE_2

Control Flow: Distribution of Routing Information

This section describes how the prefix 20.2/16 is distributed from ASBR_2 to ASBR_1 across the BGP/MPLS service provider’s backbone.

Step 1: External Route is Flooded Across Site 2

The route to 20.2/16 is received by ASBR_2 from R_Y. This route was originally advertised from one of the customer ISP’s subscriber sites or from an Internet peering point. Hence, the route to 20.2/16 is an ISP customer external route. ASBR_2 installs the following entry in its IP routing table:

Destination Next-Hop20.2/16 R_Y

After installing this route, ASBR_2 advertises the prefix 20.2/16 into Site 2’s IBGP as an external route identifying itself as the next hop. The external route is distributed to all IBGP peer routers in Site 2 (Figure 6).

20.2/16M40M40 M160 M10M10M10 VRF VRF M10

CE_2CE_1 PPE_1R_1ASBR_1 R_2 ASBR_2PE_2if_2if_1

LSP200





Figure 6: External Route Flooded Across Site 2

CE_2 receives both the IGP’s router links advertisement for ASBR_2 and the IBGP route to 20.2/16. CE_2 creates the following entries in its IP routing table:

Prefix Next-Hop Source20.2/16 ASBR_2 ExternalASBR_2 R_2 Internal

Step 2: CE_2 Advertises Site 2 Internal Routes to PE_2

CE_2 advertises all of Site 2’s internal routes (but not Site 2’s external routes) to PE_2 following normal BGP/MPLS VPN procedures and identifies itself (CE_2) as the next hop toward these routes (Figure 7).

Figure 7: CE_2 Advertises Site 2 Internal Routes to PE_2

The internal routes advertised by CE_2 to PE_2 include the following.

■ All of customer ISP’s internal networks located at Site 2.

■ The routes to all of the ASBRs in Site 2, including ASBR_2.

Note that CE_2 does not advertise the external router to 20.2/16 because it is an external route; the objective of this approach is to keep the customer’s external routes out of the BGP/MPLS VPN service provider’s VRFs.

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160 M10M10M10 VRF VRF




D=20.2/16NH=ASBR_2

M10

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160 M10M10M10 VRF VRF


if_2if_1



D=20.2/16NH=ASBR_2

M10

MPLS Table in label act outif_1 700 pop if_2



PE_2 learns Site 2’s internal routes across the interface providing connectivity to CE_2, which causes PE_2 to install Site 2’s internal routes in VRF_Red. Following standard BGP/MPLS VPN operation, when PE_2 learns a route from CE_2, it assigns that route a label that PE_2 can later use to identify the CE router that originally advertised the route. Assuming that PE_2 assigns the Label = 700 to the route to ASBR_2, PE_2 installs an MPLS mapping such that when it receives a packet from the backbone with a Label = 700, PE_2 can simply pop the label and forward the IPv4 packet directly to CE_2.

As a result of processing this routing information, PE_2 creates the following entry in VRF_Red.

Bottom TopDestination Next Hop Interface Label LabelASBR_2 Direct if_2 700 -

PE_2 also creates the following entry in its MPLS forwarding table.

Input OutputInterface Label Action Interface if_1 700 Pop if_2

Step 3: PE_2 Distributes Site 2’s Internal Routes to PE_1

PE_2 advertises the following routing update to each of its MP-IBGP peers across the BGP/MPLS backbone (Figure 8).

Destination = RD_Red:ASBR_2Next Hop = PE_2Label = 700Route Target = Red

Figure 8: PE_2 Distributes Site 2’s Internal Routes to PE_1

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160M10M10 VRF


if_2if_1if_2if_1



D=20.2/16NH=ASBR_2

M10


MPLS Table in label action outif_1 800 Swap(700) if_2 Push(200)

M10VRF



PE_1 receives PE_2’s MP-IBGP advertisement across the MP-IBGP connection, assigns an export Label = 800 to the route, and installs the following entry in its MPLS forwarding table:

Input OutputInterface Label Action Interface if_1 800 Swap (700) if_2 Push (200)

The Label = 700 is assigned as the bottom label because this is the label originally assigned to and advertised with the route by PE_2. The Label = 200 is assigned as the top label because this is the initial label for the LSP that provides connectivity from PE_1 to PE_2.

If the BGP/MPLS VPN backbone provider deploys route reflectors, the direct MP-IBGP session is not established between PE_1 and PE_2. In this case, PE_1 and PE_2 exchange their routes via the route reflector.

Step 4: PE_1 Distributes Site 2’s Internal Routes to CE_1

PE_1 advertises the following routing information to CE_1. PE_1 uses LDP to advertise the label to CE_1 (Figure 9).

Destination = ASBR_2Next Hop = PE_1Label = 800

Figure 9: PE_1 Distributes Site 2’s Internal Routes to CE_1

CE_1 receives PE_1’s advertisement and installs the following route in its IP routing table.

Destination Next-Hop LabelASBR_2 PE_1 Push(800)

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160M10M10 VRF


if_2if_1if_2if_1



D=20.2/16NH=ASBR_2

M10



M10VRF

D=ASBR_2NH=PE_1Label=800



Step 5: CE_1 Distributes Site 2’s Internal Routes to all Routers in Site 1

After installing the route into its IP forwarding table, CE_1 distributes the prefix 20.2/16 to all routers in Site 1 via IBGP with itself (CE_1) as the next hop (Figure 10).

Figure 10: CE_1 Distributes Site 2’s Internal Routes to all Routers in Site 1

As a result of this activity, every router in Site 1 (including ASBR_1) receives all of Site 2’s internal routes and creates an entry for ASBR_2 in its routing table.

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160M10M10 VRF


if_2if_1if_2if_1



D=20.2/16NH=ASBR_2

M10



M10VRF


D=ASBR_2NH=R_1

D=ASBR_2NH=CE_1



Step 6: All Routers in Site 1 Establish an IBGP Session With ASBR_2

After installing the route to ASBR_2 into its IP forwarding table, every router in Site 1 establishes an IBGP session with ASBR_2. (Figure 11). Of course, route reflectors can be deployed within each POP to significantly reduce the number of IBGP sessions that must be established.

Figure 11: CE_1 Distributes Site 2’s Internal Routes to all Routers in Site 1

These IBGP sessions exchange the customer ISP’s external routing information, including the route to 20.2/16. As a result of this exchange of routing information, every router in Site 1 (including ASBR_1) creates another entry in its IP routing table:

Destination Next-Hop20.2/16 ASBR_2

At this point, every router in Site 1 contains all of the customer ISP’s internal routes and external routes, VRF_Red in each of the BGP/MPLS service provider PE routers contains only the customer ISP’s internal routes, and the P routers in the core of BGP/MPLS service provider backbone do not contain any of the customer ISP’s routes (internal or external).

D=20.2/16NH=R_Y

D=20.2/16NH=R_2

20.2/16

M40M40 M160M10M10 VRF


if_2if_1if_2if_1

D=20.2/16NH=ASBR_2

M10



M10VRF

D=20.2/16NH=ASBR_2

D=ASBR_2NH=R_1

D=20.2/16NH=ASBR_2

D=ASBR_2NH=CE_1

D=20.2/16NH=ASBR_2


Customer I-BGPCustomer I-BGPCustomer IBGP




Data Flow: Forwarding User Traffic from ASBR_1 to 20.2/16

Figure 12 shows the routing information for prefix 20.2/16 that has been installed in each of the routers along the path from ASBR_1 to ASBR_2.

Figure 12: Forwarding Table Information

Step 1: Forwarding Across ISP Customer Site 1

Assume that ASBR_1 receives a native IPv4 packet addressed to Host 20.2.1.2 from R_X (Figure 13).

Figure 13: Forwarding Across Customer Site 1

ASBR_1 receives the packet addressed to Host 20.2.1.2 and performs a longest match route lookup. Since ASBR_1’s BGP next hop to prefix 20.2/16 is ASBR_2, and since ASBR_1’s next hop to ASBR_2 is R_1, ASBR_1 forwards the native IPv4 packet to R_1.

M40 M10M10

PE_1R_1 CE_1CE_1ASBR_1ASBR_1R_X

D=20.2/16NH=ASBR_2

D=ASBR_2NH=R_1

D=20.2/16NH=ASBR_2

D=ASBR_2NH=PE_1


20.2.1.2Payload

20.2.1.2Payload

20.2.1.2Payload

80020.2.1.2Payload

D=20.2/16NH=ASBR_2

D=ASBR_2NH=R_1

D=20.2/16NH=ASBR_2

D=ASBR_2NH=CE_1

D=20.2/16NH=ASBR_2


CE_1ASBR_1

BGP/MPLS VPNBackbone Network



R_1 receives the packet addressed to Host 20.2.1.2 and performs a longest match route lookup. Since R_1’s BGP next hop to prefix 20.2/16 is ASBR_2, and since R_1’s next hop to ASBR_2 is CE_1, R_1 forwards the native IPv4 packet to CE_1.

CE_1 receives the native IPv4 packet addressed to Host 20.2.1.2 and performs a longest match route lookup. Since CE_1’s BGP next hop to prefix 20.2/16 is ASBR_2, and since CE_1’s next hop to ASBR_2 is PE_1, CE_1 prepares to forward the packet to PE_1. However, before forwarding the packet to PE_1, CE_1 adds an MPLS shim header and pushes the Label = 800 onto the empty label stack.

Step 2: Forwarding Across the BGP/MPLS VPN Service Provider Backbone

PE_1 receives an MPLS packet with a top Label = 800 from CE_1 (Figure 14).

Figure 14: Forwarding Across BGP/MPLS VPN Service Provider Backbone

PE_1 receives the MPLS packet on if_1 and performs an exact match lookup in its MPLS forwarding table. Since the packet contains a Label = 800, PE_1 swaps the top label with a Label = 700 and then pushes the Label = 200 onto the label stack. The Label = 700 is assigned as the bottom label because it is the label originally advertised with the route by PE_2. The Label = 200 is assigned as the top label because it is the initial label for the LSP that provides connectivity from PE_1 to PE_2.

Router P receives the MPLS packet with a Label = 200 on if_1 and performs an exact match lookup in its MPLS forwarding table. Since Router P is the penultimate hop in the LSP from PE_1 to PE_2, Router P pops the top label and forwards the MPLS packet with a single Label = 700 out if_2 to PE_2.

PE_2 receives the MPLS packet with a Label = 700 on if_1 and performs an exact-match lookup in its MPLS forwarding table. PE_2 pops the label and forwards the packet out if_2 to CE_2 as a native IPv4 packet.

M10M10

20.2.1.2Payload

70020.2.1.2Payload

80020.2.1.2Payload



LSPLSPVRF VRF

CE_1

if_2if_1 if_2

CE_2

80020.2.1.2Payload

200

700

if_2if_1 if_1

P

M40

PE_1 PE_2PE_1 PE_2

PE_1 MPLS Table in label action outif_1 800 Swap(700) if_2 Push(200)

PE_2 MPLS Table in label act outif_1 700 pop if_2

P MPLS Table in label action outif_1 200 Pop if_2

PE_1 MPLS Table in label action outif_1 800 Swap(700) if_2 Push(200)

PE_2 MPLS Table in label act outif_1 700 pop if_2

P MPLS Table in label action outif_1 200 Pop if_2



Step 3: Forwarding Across ISP Customer Site 2

CE_2 receives a native IPv4 packet from PE_2 (Figure 15).

Figure 15: Forwarding Across ISP Customer Site 2

CE_2 receives the packet addressed to Host 20.2.1.2 and performs a longest match route lookup. Since CE_2’s next hop to prefix 20.2/16 is R_2, CE_2 forwards the native IPv4 packet to R_2.

R_2 receives the native IPv4 packet addressed to Host 20.2.1.2 and performs a longest match route lookup. Since R_2’s next hop to prefix 20.2/16 is ASBR_2, R_2 forwards the native IPv4 packet to ASBR_2.

When ASBR_2 receives the native IPv4 packet addressed to Host 20.2.1.2, it performs a longest match route lookup. Since ASBR_2’s next hop to prefix 20.2/16 is R_Y, ASBR_2 forwards the native IPv4 packet to R_Y.

Observations

This example requires only a couple of minor extensions to the conventional BGP/MPLS VPN operational model.

■ MPLS must be extended from the PE router to the CE router. However, MPLS does not have to extend into the ISP customer’s sites.

■ The PE router must ensure that the data traffic of one customer is not spoofed by other customers. This security can be accomplished by having the PE router examine the labels in the MPLS traffic that each CE router transmits to the PE router, and then verify that each packet contains a label that the PE router previously advertised to the particular CE router.

M40 M10M10

CE_1CE_1ASBR_1ASBR_1


20.2.1.2Payload

20.2.1.2Payload

20.2.1.2Payload

20.2.1.2Payload


PE_2 R_2CE_2 ASBR_2 R_Y

D=20.2/16NH=R_2

D=20.2/16NH=R_Y

D=20.2/16NH=ASBR_2

D=20.2/16NH=R_2

D=20.2/16NH=R_Y

D=20.2/16NH=ASBR_2



A BGP/MPLS VPN Service Provider as a Customer

Another possibility is that the BGP/MPLS VPN service provider has customers that are themselves BGP/MPLS VPN service providers. This example is also referred to as hierarchical or recursive BGP/MPLS VPNs. In this scenario, the customer BGP/MPLS VPN service provider’s VPN-IPv4 routes are considered external routes, and the backbone BGP/MPLS VPN service provider does not import them into its VPN Routing and Forwarding (VRF) table. The backbone BGP/MPLS VPN service provider only imports the customer BGP/MPLS VPN service provider’s internal routes into its VRF.

The operational model for this scenario is very similar to the previous case study where the BGP/MPLS VPN service provider’s customer is an ISP. Figure 1 illustrates the main differences between these two case studies:

Figure 16 shows the differences in the network topologies when the customer of the BGP/MPLS VPN service provider is an ISP and when the customer is itself a BGP/MPLS VPN service provider.

Table 1: Comparison of ISP Customer and BGP/MPLS VPN Customer

Comparison Point ISP Customer BGP/MPLS VPN Provider Customer

Customer edge device ASBR PE router

IBGP sessions Carry IPv4 routes Carry external VPN-IPv4 routes with associated labels

Forwarding within the customer network

MPLS is optional MPLS is required



Figure 16: Topology Differences

Network Topology

The network topology for the BGP/MPLS VPN service provider as a customer application is shown in Figure 17. The BGP/MPLS VPN backbone service provider has a customer that is itself a BGP/MPLS VPN service provider (the VPN Red customer). In this example both of the customer BGP/MPLS VPN service provider sites are shaded red to indicate that they represent different sites with the same AS number. The BGP/MPLS VPN Red service provider has two enterprise VPN customers: VPN Blue and VPN Green. The prefix 10.2/16 is a VPN Red external route that PE_4 learns from the CE router deployed at VPN Blue Site 2.

PE

Site 1

Site 1

Site 2

Site 2

M40M40 M160 M10M10M10

VRF

VRF VRF

VRF

M10

VRF

VRFCE_2CE_1 PPE_1R_1

PE_3R_2

PE_4

BGP/MPLS VPN Backbone Provider

BGP/MPLS VPN Provider (Customer Site 1)

PE_2


10.2/16

M40M40 M160 M10M10M10 VRF VRF M10

20.2/16CE_1

R_X

PPE_1R_1ASBR_1 R_2 ASBR_2PE_2



ISP as a Customer

BGP/MPLS VPN Provider as a Customer

CE_2



Figure 17: Network Topology — BGP/MPLS VPN Provider as a Customer

This case study describes how the prefix 10.2/16 is distributed across the BGP/MPLS backbone to PE_3 so that PE_3 can advertise the route to VPN Blue Site 1. After describing how the routing information is distributed from PE_4 to PE_3, we show how a packet addressed to host 10.2.1.2 is forwarded across the BGP/MPLS backbone from PE_3 to PE_4.

Control Flow: LSP Establishment

Both the BGP/MPLS VPN backbone service provider and the BGP/MPLS VPN customer service provider are aware of the LSPs that need to be established to provide the required PE-to-PE connectivity. In this case study, both the backbone and customer BGP/MPLS VPN service providers use LDP or RSVP to establish three LSPs (Figure 18). The label shown at the ingress to each LSP is the initial label that the ingress LSR uses when it forwards a packet across that LSP.

Figure 18: LSP Establishment

PE

Site 1

Site 1

Site 2

Site 2

M40M40 M160 M10M10M10

VRF

VRF VRF

VRF

M10

VRF

VRFCE_2CE_1 PPE_1R_1

PE_3R_2

PE_4



PE_2


10.2/16

PE

Site 1

Site 1

Site 2

Site 2

M40M40 M160 M10M10M10

VRF

VRF VRF

VRF

M10

VRF

VRF

LSP200LSP100 LSP300

CE_2CE_1 PPE_1R_1PE_3

R_2PE_4



PE_2


10.2/16



Control Flow: Distribution of Routing Information

This section describes how the prefix 10.2/16 is distributed from PE_4 in Red BGP/MPLS VPN customer Site 2 across the BGP/MPLS VPN backbone provider network to PE_3 in BGP/MPLS VPN customer Site 1.

At first glance it might appear that labels need to be distributed so that PE_3 can push three labels onto the label stack.

■ A bottom label that is associated with the route to 10.2/16

■ A middle label that is associated with the route to PE_4

■ A top label that is associated with the route to CE_1

While this solution is certainly valid, this example only requires that labels be distributed so that PE_3 pushes two labels onto the label stack.

■ A bottom label that is associated with the route to 10.2/16

■ A top label that is associated with the route to PE_4

Only two labels are required because Site 1 and Site 2 of the customer BGP/MPLS VPN service provider are part of the same AS, so they are part of the same IGP cloud. Hence, PE_3, R_1, and CE_1 all have a label associated with the route to PE_4 because the route to PE_4 is an interior route of the customer BGP/MPLS service provider. The labels associated with the route to PE_4 are different at each router hop within Site 1, but each router has a label that it can use to forward MPLS traffic to PE_4. Thus, the top and middle labels of the anticipated three-label stack can be integrated into a single label that is associated with the route to PE_4.

Step 1: The Route to PE_4 is Distributed Across BGP/MPLS VPN Provider Site 2

The CE router located at VPN_Blue Site 2 advertises the route to 10.2/16 to PE_4. Since the route is received on an interface that is associated with VRF_Blue, PE_4 installs the following route to 10.2/16 in VRF_Blue.

Destination Next Hop Interface 10.2/16 Direct if_2

Following standard BGP/MPLS VPN procedures, PE_4 assigns Label=500 to the route that goes to 10.2/16. As a result of this process, PE_4 creates the following entry in its MPLS forwarding table.


Independent of installing the route to 10.2/16 into VRF_Blue and assigning it a Label = 500, PE_4 floods its router link advertisements into Customer Site 2’s IGP. The router link advertisement flooded by the IGP does not carry the label that has been associated with the route to PE_4. CE_2 receives the Label = 300 that is associated with the route to PE_4 via RSVP or LDP, and this label is received by CE_2 from R_2.

As a result of these steps, the following forwarding information is installed in the routers in customer BGP/MPLS VPN Provider Site 2 (Figure 19).



Figure 19: Route to PE_4 is Advertised Across Customer Site 2 to CE_2

R_2 receives the IGP’s router links advertisement for PE_4. R_2 places the link-state advertisement (LSA) into its link state database and runs an SPF calculation across the link-state database resulting in the following entry in its IP routing table.

Prefix Next-Hop SourcePE_4 PE_4 Internal

R_2 also creates the following entry in its MPLS forwarding table to support the LSP from CE_2 to PE_4.


CE_2 receives the IGP’s router links advertisement for PE_4. CE_2 places the LSA into its link-state database and runs an SPF calculation across the link-state database, resulting in the following entry in its IP routing table.

Prefix Next-Hop SourcePE_4 R_2 Internal

CE_2 also creates the following entry in its MPLS forwarding table to support the LSP from CE_2 to PE_4.

Input OutputInterface Label Action Interface if_1 600 Swap (300) if_2

Label=600 is assigned as the export label while Label = 300 is swapped for the top label because it is the initial label for the LSP that provides connectivity from CE_2 to PE_4.

Site 1

Site 1

M40M40 M160 M10M10M10

VRF

VRF VRF

VRFCE_2CE_1 PPE_1R_1PE_3 R_2



PE_2


if_1 if_2

if_1

in label act outif_1 300 pop if_2


in label act outif_1 600 swap if_2 (300)

PE

Site 2

Site 2

M10

VRF

PE_410.2/16

if_3if_2

VRF



Step 2: CE_2 Advertises Customer Site 2’s Internal Routes to PE_2

CE_2 advertises all of Site 2’s internal routes (including the route to PE_4) to PE_2 following standard BGP/MPLS VPN procedures and identifies itself (CE_2) as the next hop toward these routes (Figure 20).

Figure 20: CE_2 Advertises Customer Site 2’s Internal Routes to PE_2

The internal routes advertised by CE_2 to PE_2 include the following.

■ All of the networks configured within Customer BGP/MPLS VPN Provider Site 2.

■ The routes to all of the PEs in Customer BGP/MPLS VPN Provider Site 2, including PE_4.

PE_2 learns Site 2’s internal routes across the interface providing connectivity to CE_2, which causes PE_2 to install Site 2’s internal routes in VRF_Red. Following normal BGP/MPLS VPN operation, PE_2 assigns the Label = 700 to the route to PE_4 and installs an MPLS mapping such that when PE_2 receives a packet from the backbone with a Label = 700, it can simply pop the label and forward the packet directly to CE_2.

As a result of this process, PE_2 creates the following entry in its MPLS forwarding table.

Input OutputInterface Label Action Interface if_1 700 Swap(600) if_2

Site 1

Site 1

M40M40 M160 M10M10M10

VRF

VRF VRF

VRF

CE_2CE_1 PPE_1R_1PE_3 R_2



PE_2


if_1

if_2

if_1if_1




PE

Site 2

Site 2

M10

VRF

VRF

PE_4

10.2/16

if_3if_2

if_2



if_1

if_2

[D=PE_4, NH=CE_2, L=600]



Step 3: PE_2 Distributes Customer Site 2’s Internal Routes to PE_1

PE_2 advertises the following routing update to each of its MP-IBGP peers across the BGP/MPLS backbone (Figure 21).

Destination = RD_Red:PE_4Next Hop = PE_2Label = 700Route Target = Red

Figure 21: PE_2 Distributes Customer Site 2’s Internal Routes to PE_1

PE_1 receives PE_2’s MP-IBGP advertisement across the MP-IBGP connection and installs the following entry in its MPLS forwarding table.

Input OutputInterface Label Action Interface if_1 800 Swap (700) if_2 Push (200)

The Label = 700 is assigned as the bottom label because this is the label originally assigned to and advertised with the route by PE_2. The Label = 200 is assigned as the top label because this is the initial label for the LSP from PE_1 to PE_2.

Site 1

Site 1

M40M40 M160M10M10

VRF

VRF

VRF

CE_2CE_1 PE_1R_1PE_3 R_2


PE_2


if_1

if_2

if_1if_1




PE

Site 2

Site 2

M10

VRF

VRF

PE_4

10.2/16

if_3if_2

if_2


if_1

if_2

if_1

if_2

if_1

if_2

[D=RD_Red:PE_4, NH=PE_2, L=700 Target=Red]

in label action outif_1 800 Swap(700) if_2 Push(200)


M10VRF



Step 4: PE_1 Distributes Customer Site 2’s Internal Routes to CE_1

PE_1 advertises the following route to CE_1 (Figure 22).

Destination = PE_4Next Hop = PE_1Label = 800

Figure 22: PE_1 Distributes Customer Site 2’s Internal Routes to CE_1

CE_1 receives PE_1’s advertisement and installs the following route in its IP routing table.

Destination Next-HopPE_4 PE_1

As a result of this process, CE_1 creates the following entry in its MPLS forwarding table.

Input OutputInterface Label Action Interface if_1 400 Swap(800) if_2

Site 1

Site 1

M40M40 M160M10

VRF

VRF

CE_2CE_1 PE_1R_1 PPE_3

R_2


PE_2


if_1

if_2

if_1if_1




PE

Site 2

Site 2

M10

VRF

VRF

PE_4

10.2/16

if_3if_2

if_2




if_1

if_2

if_1

if_2

if_1

if_2



M10VRF


if_1

if_2

in label act outif_1 400 Swap if_2 (800)

[D=PE_4, NH=PE_1 L=800]

M10VRF



Step 5: CE_1 Distributes Customer Site 2’s Internal Routes to all Routers in Customer Site 1

After installing the route into its IP forwarding table, CE_1 propagates all of Customer Site 2’s internal routes by flooding them into Site 1’s IGP with itself (CE_1) as the next hop (Figure 23).

Figure 23: CE_1 Distributes Customer Site 2’s Internal Routes to all Routers in Site 1

As a result of this process, every router in Site 1 (including PE_3) creates the following entry in its IP routing table.

Destination Next-HopPE_4 <router specific, but via CE_1>

Site 1

Site 1

M40M40 M160M10

VRF

VRF

CE_2CE_1 PE_1 PPE_3

R_2PE_2


if_1

if_2

if_1if_1




PE

Site 2

Site 2

M10

VRF

VRF

PE_4

10.2/16

if_3if_2

if_2




if_1

if_2

if_1

if_2

if_1

if_2



M10VRF


if_1

if_2

D=PE_4NH=R_1

[D=PE_4, NH=CE_1]

M10VRF


D=PE_4NH=CE_1

if_1

if_2if_3

if_2

R_1

if_1



Step 6: PE Routers in Customer Site 1 Establish an MP-IBGP Session With PE_4

After installing the route to PE_4 into its IP forwarding table, each PE router in customer BGP/MPLS VPN Provider Site 1 establishes an MP-IBGP session with PE_4. (Figure 24). Alternatively, a single route reflector that supports all of the VPN_Blue sites can be deployed somewhere within the network to significantly reduce the number of MP-IBGP sessions that must be established. If this option is chosen, all customer BGP/MPLS VPN Provider PE routers that are directly connected to a VPN_Blue site become clients of VPN_Blue route reflector.

Figure 24: PE Routers in Customer Site 1 Establish an MP-IBGP Session with PE_4

These MP-IBGP sessions are used to exchange the customer BGP/MPLS BPN service provider’s external routing information (the VPN-IPv4 routes at customer sites and their associated labels), including the route to RD_Blue:10.2/16 with a Label=500. As a result, PE_3 and PE_4 exchange VPN_Blue routes and populate their VRF_Blue.

As a result of this procedure, PE_3 creates an entry in its VRF_Blue.

Destination Next-Hop Label10.2/16 PE_4 500

Site 1

Site 1

M40M40 M160M10

VRF

VRF

CE_2CE_1 PE_1 PPE_3

R_2PE_2


if_1

if_2

if_1if_1




PE

Site 2

Site 2

M10

VRF

VRF

PE_4

10.2/16

if_3if_2

if_2




if_1

if_2

if_1

if_2

if_1

if_2



M10VRF


if_1

if_2

D=PE_4NH=R_1L=100

M10VRF


D=10.2/16NH=PE_4L=500

if_1

if_2if_3

if_2

R_1

if_1



[D=RD_Blue:10.2/16, NH=PE_4, L=500, Target=Blue]



Data Flow: Forwarding User Traffic from PE_3 to 10.2/16

Figure 25 shows the MPLS forwarding information for prefix 10.2/16 that has been installed in each of the routers along the path from PE_3 to PE_4.

Figure 25: Forwarding Table Information

When customer traffic from VPN_Blue Site 1 arrives at PE_3, each packet receives two labels: the first (bottom) Label = 500 is the label associated with the route to prefix 10.2/16 and the second (top) Label = 100 is the label associated with the route to PE_4 (via CE_1). The top label of the two-label packet is swapped at each subsequent hop until the packet arrives at PE_1. PE_1 pushes a third Label = 200 onto the label stack; this label is for the route that PE_1 associates with the route to PE_2. The third label is popped by penultimate router P, and the packet arrives at PE_2 with a stack that contains two labels. The top label of the two-label packet is swapped at each subsequent hop until the packet arrives at penultimate router R_2. R_2 pops the top label and forwards the packet to PE_4 with a single Label = 500. PE_4 pops this label and forwards the packet to the CE router located at VPN_Blue Site 2.

CE_2CE_1 PE_1R_1 PPE_3

R_2PE_2

if_1

if_2

if_1if_1PE

PE_4

10.2/16

if_3if_2

if_2

if_1

if_2

if_1

if_2

if_1

if_2if_2if_2

if_1if_1

if_2



Site 1 Site 2

R_2: in label action out

if_1 300 Pop if_2

PE_4: in label action out

if_1 500 Pop if_2

CE_2: in label action out

if_1 600 Swap if_2

(300)


if_1 800 Swap(700) if_2

Push(200)

CE_1: in label action out

if_1 400 Swap if_2

(800)

if_3if_1

P: in label action out

if_1 200 Pop if_2


if_1 700 Swap if_2

(600)


R_1: in label action out

if_1 100 Swap if_2

(400)

PE_3:Dest NH action out

10.2/16 R_1 Push(500) if_2

Push(100)

M40 M40 M10M10 M10M160VRF VRFM10

VRF

VRF

VRF

VRF



Step 1: Forwarding Across Customer BGP/MPLS VPN Service Provider Site 1

Assume that PE_3 receives a native IPv4 packet on if_1 that is addressed to Host 10.2.1.2 from the CE router at VPN_Blue Site 1 (Figure 26).

Figure 26: Forwarding Across Customer BGP/MPLS Service Provider Site 1

PE_3 performs a longest match route lookup in its IP forwarding table. As a result of the lookup, PE_3 pushes the Label = 500 onto the label stack and then pushes the Label = 100 onto the label stack. The Label = 500 is assigned as the bottom label because it is the label that was originally advertised with the route by PE_4 via the MP-IBGP session. The Label = 100 is assigned as the top label because it is the label that PE_3 associates with the route to PE_4.

R_1 receives the MPLS packet with a top Label = 100 over if_1 and performs an exact-match lookup in its MPLS forwarding table. R_1 swaps the top Label = 100 with a Label = 400 and forwards the MPLS packet out if_2 to CE_1.

CE_1 receives the MPLS packet with a Label = 400 on if_1 and performs an exact match lookup in its MPLS forwarding table. As a result of the lookup, CE_1 swaps the Label= 400 at the top of the label stack with a Label = 800 and forwards the MPLS packet out if_2 to PE_1.


Site 1 LSP LSPBGP/MPLS VPN Backbone Provider

M40M10 M10VRF VRF

100

10.2.1.2Payload

500

400

10.2.1.2Payload

500

800

10.2.1.2Payload

50010.2.1.2Payload

10.2.1.2Payload

PE_1R_1PE_3 CE_1

if_1

if_2

if_1

if_2

if_1

if_2

in label action outif_1 100 Swap if_2 (400)


Dest NH action out10.2/16 R_1 Push(500) if_2 Push(100)



Step 2: Forwarding Across the Backbone BGP/MPLS VPN Service Provider Network

PE_1 receives an MPLS packet with a top Label = 800 on if_1 from CE_1 (Figure 27).

Figure 27: Forwarding Across Backbone BGP/MPLS VPN Service Provider Network

PE_1 performs an exact match lookup in its MPLS forwarding table. Since the packet contains a Label = 800, PE_1 swaps the top label with a Label = 700 and then pushes the Label = 200 onto the label stack. The Label = 700 is assigned because this label is the one that PE_2 originally advertised with the VPN-IPv4 route to RD_Red:PE_4. The Label = 200 is assigned as the top label because it is the initial label for the LSP from PE_1 to PE_2.

Router P receives the MPLS packet with a top Label = 200 on if_1 and performs an exact match lookup in its MPLS forwarding table. Since Router P is the penultimate hop in the LSP from PE_1 to PE_2, Router P pops the top label and forwards the MPLS packet with a top Label = 700 out if_2 to PE_2.

PE_2 receives the MPLS packet with a top Label = 700 on if_1 and performs an exact match lookup in its MPLS forwarding table. PE_2 swaps the top Label = 700 with a Label = 600 and forwards the MPLS packet out if_2 to CE_2.



Step 3: Forwarding Across Customer BGP/MPLS VPN Service Provider Site 2

CE_2 receives the MPLS packet with a top Label = 600 on if_1 from PE_2 (Figure 28).

Figure 28: Forwarding Across Customer BGP/MPLS Service Provider Site 2

CE_2 performs an exact match lookup in its MPLS forwarding table. Since the packet contains a top Label = 600, CE_2 swaps the top Label = 600 with a Label = 300. The Label = 300 is assigned as the top label because it is the initial label for the LSP from CE_2 to PE_4.

R_2 receives the MPLS packet with a top Label = 300 on if_1 and performs an exact match lookup in its MPLS forwarding table. Since R_2 is the penultimate hop in the LSP from CE_2 to PE_4, R_1 pops the top Label = 300 and forwards the MPLS packet with a single Label = 500 out if_2 to PE_4.

PE_4 receives the MPLS packet with a Label = 500 on if_1 and performs an exact match lookup in its MPLS forwarding table. Recall that PE_4 originally assigned the Label = 500 to the prefix 10.2/16 when in learned the route from the CE router at VPN_Blue site 2 and installed the route into VRF_Blue. PE_4 pops the label and then forwards a native IPv4 packet out if_2 to the CE router at VPN_Blue site 2.

Multi-AS BGP/MPLS VPN Operations

When working with multi-AS BGP/MPLS VPN applications, your initial impression might lead you to consider the case of multiprovider operations. In this application, a given customer is connected to two different BGP/MPLS VPN service providers, and the customer desires RFC 2547bis service between its sites. While multi-AS application is valid, probably a more immediate application concerns the case where a single service provider is composed of multiple autonomous systems, and it wants to offer seamless RFC 2547bis service to all of its customers. All of the case studies described in this section apply to situations when the customer BGP/MPLS VPN providers are two independent service providers (with different AS numbers) or are a single service provider with multiple AS numbers.

There are a number of approaches that can be used to support multi-AS operations.

Site 2

PE_2 CE_2


PE_4


M10VRF

10.2.1.2Payload

500

300

10.2.1.2Payload

500

600

10.2.1.2Payload

50010.2.1.2Payload

10.2.1.2Payload

if_2

in label action outif_1 300 Pop if_2

in label action outif_1 500 Pop if_2


LSP LSP

R_2

M40M10 VRFif_1

if_2

if_1if_1

if_2



■ EBGP VRF-to-VRF connections between AS border routers

■ MP-EBGP distribution of labeled VPN-IPv4 routes between AS border routers

■ Multi-hop MP-EBGP distribution of labeled VPN-IPv4 routes between PE routers

EBGP VRF-to-VRF Connections Between AS Border Routers

In the VRF-to-VRF approach, a PE router in one AS is directly connected to a PE router in the other AS. Each of the PE routers also functions as an AS border router (ASBR). The PE routers are connected by multiple sub-interfaces, where each of the VPNs that are required to exchange routes across the AS boundary is assigned to at least one of these sub-interfaces. This approach is also referred to as Inter-Provider Backbones Option A in RFC 2547bis.

Figure 29 shows how the VPN_Blue route 10.2/16 is distributed from VPN_Blue Site 2 to VPN_Blue Site 1. The two BGP/MPLS VPN backbones are shaded different colors to indicate that they have different AS numbers.

Figure 29: VRF-to-VRF Connections at AS Border Routers

Each of the PE (ASBR) routers treats the other PE (ASBR) router as if it were a CE router; it associates each sub-interface with a VRF and uses conventional EBGP to distribute unlabeled IPv4 routes to its peer. While this approach provides a workable solution and does not require the complexity of running MPLS at the boundary between autonomous systems, it does have scalability limitations because it requires per-VPN configuration on the PE (ASBR) routers and can potentially require ASBRs to maintain an extremely large number of VPN-IPv4 routes.

VRF

VRF

VRF

VRFVRFVRF

M10VRF M10VRF

BGP/MPLS VPN Backbone (AS 1)

BGP/MPLS VPN Backbone (AS 2)

Site 1

Site 1

PE PE(ASBR)

PE(ASBR)

PE

PE

PESite 2

Site 2

MP-IBGP

Label +RD_Blue:10.2/16

EBGP

10.2/16

10.1/16

IGP

10.2/16

IGP 10.2/16MP-IBGP




MP-EBGP Distribution of Labeled VPN-IPv4 Routes Between AS Border Routers

In the second approach, the PE routers within an AS use MP-IBGP to distribute labeled VPN-IPv4 routes to a PE (ASBR) or a route reflector of which the PE (ASBR) is a client. The PE (ASBR) then uses MP-EBGP to distribute the labeled VPN-IPv4 routes to its peer PE (ASBR) in the neighboring AS. Finally, the peer PE (ASBR) use MP-IBGP to distribute labeled VPN-IPv4 routes to PE routers or to a route reflector of which the PE routers are a client. This approach is also referred to as Inter-Provider Backbones Option B in RFC 2547bis.

Figure 30 shows how the VPN_Blue route 10.2/16 is distributed from VPN_Blue Site 2 to VPN_Blue Site 1. The two BGP/MPLS VPN backbones are shaded different colors to indicate that they have different AS numbers.

Figure 30: MP-EBGP Distribution of Labeled VPN-IPv4 Routes at ASBRs

In this approach, the following four fundamental rules apply.

■ VPN-IPv4 routes must only be accepted across private peering points that are established as part of a trusted relationship between autonomous systems.

■ VPN-IPv4 routes must not be distributed to or accepted from the public Internet.

■ VPN-IPv4 routes must not be distributed to or accepted from untrusted MP-EBGP peers.

■ An ASBR must never accept an MPLS packet from an MP-EBGP peer unless it has distributed the top label in the label stack to that peer.

This approach enhances the scalability of the EBGP VRF-to-VRF solution because it eliminates the need for per-VPN configuration on the PE (ASBR)s. However, it also introduces greater complexity because it requires the following.

■ An LSP be established from the ingress PE router to the egress PE router.

■ Trust relationships exist between and among the set of autonomous systems along the path from the ingress PE router to the egress PE router.

■ Understandings exist between and among the autonomous systems concerning which ASBRs receive routes with specific Route Target attributes.

M10VRF M10VRF M10VRF M10VRF

M10

10.2/16

BGP/MPLS VPN Backbone (AS 1) BGP/MPLS VPN Backbone (AS 2)

Site 1MP-EBGP

MP-IBGP MP-IBGP

Site 2

PEPE(ASBR)

PE(ASBR) PE

MP-IBGP RR

Label + RD_Blue:10.2/16


10.2/16 10.2/16

IGP IGP





Multi-hop MP-EBGP Distribution of Labeled VPN-IPv4 Routes Between PE Routers

While the previous two proposals provide workable solutions for inter-AS operations, the long-term applicability of these approaches is limited because their scalability is determined by the amount of VPN routing information carried by the individual BGP/MPLS VPN service providers. Ultimately, service providers will want to deploy the multi-hop MP-EBGP approach because it does not require that VPN-IPv4 routes be maintained and distributed by ASBRs. Multi-hop MP-EBGP is required because the EBGP peers are not directly connected. This approach significantly enhances the solution’s scalability because the load on the ASBRs is independent of the amount of VPN routing information carried by BGP/MPLS VPN service providers. This solution is also referred to as Inter-Provider Backbones Option C in RFC 2547bis.

Example #1: Multi-AS Operations Using a BGP/MPLS VPN Capable Transit Provider

Figure 31 shows the network topology for multi-AS operations using a BGP/MPLS VPN capable transit provider. The BGP/MPLS VPN transit provider has two customers that are themselves BGP/MPLS VPN service providers (the VPN Red customer and the VPN Gold customer). The customer BGP/MPLS VPN service provider networks are shaded different colors to indicate that they have different AS numbers.

Figure 31: Multi-AS Operations: BGP/MPLS VPN Capable Transit Provider

The prefix 10.2/16 is a VPN Gold external route that PE_4 learns from the CE router deployed at VPN Blue Site 2. The VPN Blue prefix 10.2/16 must be distributed across the BGP/MPLS VPN capable transit provider to PE_3 so that PE_3 can advertise the route to VPN Blue Site 1.

While at first glance this might appear to be an extremely complex problem, the solution is almost identical to carrier-of-carriers solution for the BGP/MPLS VPN service provider customer scenario discussed earlier (Figure 17 through Figure 28). Some of the similarities between this example and the previous carrier-of-carriers example include the following.

■ An LSP must exist from the ingress PE router (PE_3) to the egress PE router (PE_4).

■ From the BGP/MPLS VPN transit provider’s perspective, the Red and the Gold BGP/MPLS VPN service providers are considered a single VPN. Hence, the BGP/MPLS VPN transit provider’s PE routers are configured with Red_Gold_VRFs (instead of Red VRFs) that are assigned a RD with a value of RD_Red_Gold (instead of RD_Red).

The key differences between this approach and the previous carrier-of-carriers example are as follows.

R_1 PPE_3

R_2 PEPE_4

10.2/16

Site 1 Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

BGP/MPLS VPN Capable Transit Provider (AS 3)

BGP/MPLS VPN Provider (AS 1) BGP/MPLS VPN Provider (AS 2)

CE_2(ASBR)

CE_1(ASBR)

PE_1(ASBR)

PE_2(ASBR)

VRF VRF

Site 1 Site 2



■ The BGP/MPLS VPN transit provider’s BGP/MPLS VPN service provider customers have different AS numbers instead of the same AS number.

■ Since the BGP/MPLS VPN transit provider’s customer sites have different AS numbers, the BGP/MPLS VPN customer’s PE routers must use MP-EBGP instead of MP-IBGP to exchange external (VPN-IPv4) routing information.

■ Since the BGP/MPLS VPN transit provider’s customer sites have different AS numbers, the multi-AS operations example requires a three-label stack because the route to PE_4 in BGP/MPLS VPN customer Site 2 (AS 2) is an exterior route to BGP/MPLS VPN customer Site 1 (AS 1).

■ In the carrier-of-carriers example, the routing exchange between CE and PE routers is likely to occur using an IGP, while LDP supports label distribution. IGP and LDP are used because all sites within the VPN have the same AS number. In contrast, for the multi-AS operation example, the routing exchange between CE and PE routers is likely to occur using EBGP, which also carries the labels associated with each route. Thus, LDP is generally not required on the CE-to-PE link. EBGP is used because the sites within the VPN have different AS numbers.

Other than these relatively minor differences, the solutions are basically identical. Figure 32 compares and contrasts the multi-AS operations using a BGP/MPLS VPN capable transit provider solution with the previously discussed carrier-of-carriers for a BGP/MPLS VPN capable transit provider solution.



Figure 32: Compare and Contrast BGP/MPLS VPN Capable Transit Provider

R_1 PPE_3

R_2 PEPE_4

10.2/16

Site 1 Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

CE_2(ASBR)

CE_1(ASBR)

PE_1(ASBR)

PE_2(ASBR)

VRF

R_1 PPE_3

R_2 PEPE_4

10.2/16

Site 1

Site 1

Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

CE_2(ASBR)

CE_1(ASBR)

PE_1(ASBR)

PE_2(ASBR)

VRF VRF

VRF

MP-IBGP(for Provider'sInternal Routes)

Carrier-of-Carriers Solution with a BGP/MPLS VPN Capable Transit Provider

Transit Provider(AS 3)IGP + LDP

Provider (AS 1)IGP + LDP


MP-IBGP(for Provider'sExternal Routes)

IGP + LDP(for Provider's Internal Routes)

IGP + LDP(for Provider's Internal Routes)

Forwarding:PE_3 pushes two labels





Multi-AS Operations with a BGP/MPLS VPN Capable Transit Provider

Direct EBGP(for Provider's Internal Routes)

Direct EBGP(for Provider's' Internal Routes)

MP-EBGP(for Provider'sExternal Routes)

Forwarding:PE_3 pushes three labels



The scalability of this approach can be enhanced through the use of route reflectors. When deploying route reflectors, each PE router maintains a single MP-IBGP connection to the route reflector in its own AS. Instead of establishing a number of multi-hop MP-EBGP connections between PE routers in different autonomous systems, a single MP-EBGP connection is established between the route reflector in one AS and the route reflector in the other AS. Of course, the route reflectors must not modify the BGP next-hop attributes of the VPN-IPv4 routes when then distributing them over these connections.

Example #2: Multi-AS Operations Using an MPLS Capable Transit Provider

The network topology for multi-AS operations using an MPLS capable transit provider is shown in Figure 33. The only differences between this example and the previous example are as follows.

■ The transit provider supports MPLS, but does not offer BGP/MPLS VPN service.

■ The addresses used in each of the customer BGP/MPLS VPN service provider networks must be globally unique.

Figure 33: Multi-AS Operations: MPLS Capable Transit Provider

Other then these relatively minor differences, the solutions are basically identical. Figure 34 compares and contrasts the multi-AS operations using an MPLS capable transit provider solution with the previously discussed multi-AS operations using a BGP/MPLS VPN capable transit provider solution.

R_1PE_3

R_2 PEPE_4

10.2/16

Site 1

Site 1

Site 2

Site 2

M40 M40 M10M10 M160M10

VRF

VRF

VRF

VRF

MPLS VPN Capable Transit Provider (AS 3)


RASBR_2 ASBR_3 ASBR_4ASBR_1

Globally Unique Addresses Globally Unique Addresses



Figure 34: Compare and Contrast MPLS Capable Transit Provider

Globally unique addressesPublic or Private Addressing Globally unique addressesPublic or Private AddressingBGP/MPLS VPN Capable

R_1 PPE_3

R_2 PEPE_4

10.2/16

Site 1 Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

CE_2(ASBR)

CE_1(ASBR)

PE_1(ASBR)

PE_2(ASBR)

R_1 R3PE_3

R_2 PEPE_4

10.2/16

Site 1

Site 1

Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

ASBR_4ASBR_1 ASBR_2 ASBR_3

VRF VRF


Provider (AS 1 )IGP + LDP





Direct EBGP(for provider'sInternal Routes)


BGP/MPLS VPN Capable Transit Provider

MP-EBGP(for Provider'sExternal routes)


MPLS Capable Transit Provider

Direct EBGP(for Provider'sInternal Routes)



IBGP(for Provider'sInternal Routes)

Globally unique addressesGlobally Unique Addressing Globally unique addressesGlobally Unique AddressingMPLS Capable but NOTBGP/MPLS VPN Capable



As in the previous example, the use of route reflectors enhances the scalability of this solution.

Example #3: Multi-AS Operations Using Direct Interconnect

This example describes how multi-AS operations are supported when there is a direct connection between the BGP/MPLS VPN providers’ ASBRs. Figure 35 shows the topology for this example.

Figure 35: Multi-AS Operations: Direct Connection Between BGP/MPLS VPN Providers

This solution is virtually identical to the multi-AS operations using an MPLS capable transit provider except that the entire MPLS capable transit provider network is replaced by a single point-to-point wire.

■ A direct EBGP connection is established between ASBRs to exchange the BGP/MPLS VPN service providers’ internal IPv4 routes. LDP does not run over this connection because the direct EBGP connection also carries labels.

■ Multi-hop MP-EBGP sessions are established between PE routers to exchange the providers’ external VPN-IPv4 routes and the labels associated with these routes.

Figure 36 compares and contrasts the multi-AS operations using a direct connection between BGP/MPLS VPN service providers solution with the previously discussed multi-AS operations using an MPLS capable transit provider solution.

R_1PE_3

R_2 PEPE_4

10.2/16

Site 1

Site 1

Site 2

Site 2

M40 M40 M10M10

VRF

VRF

VRF

VRF


ASBR_4ASBR_1

Globally Unique AddressesGlobally Unique Addresses



Figure 36: Compare and Contrast Directly Connected BGP/MPLS VPN Providers

MPLS Capable Transit Provider

IBGP(for Provider'sRoutes)


R_1 R3PE_3

R_2 PEPE_4

10.2/16

Site 1 Site 2

Site 1 Site 2

M40 M40 M10M10 M10M160M10

VRF

VRF

VRF

VRF

ASBR_4ASBR_1 ASBR_2 ASBR_3

R_1PE_3

R_2 PEPE_4

10.2/16

Site 1

Site 1

Site 2

Site 2

M40 M40 M10M10

VRF

VRF

VRF

VRF

ASBR_1 ASBR_4

Transit Provider(AS 3) IGP + LDP



ProviderIBGP






Direct Connection Between ASBRs


Globally Unique AddressesGlobally Unique Addresses



Conclusion

Figure 37 compares and contrasts the differences between BGP/MPLS VPN connectivity solutions for hierarchical carrier-of-carriers applications and recursive multi-AS operation applications. The shading assigned to each provider network represents the service provider’s AS number.

Figure 37: BGP/MPLS VPNs — Carrier-of-Carriers vs. Multi-AS Operations

The most important difference between the two scenarios is whether the sites within a specific VPN have the same AS number (carrier of carriers) or have different AS numbers (recursive multi-AS operations). All other differences are a consequence of this key difference.

VRF

VRF

VRF

VRFVRF VRFVRF VRF

VRF VRFVRF VRF

Hierarcical Carrier-of-Carriers

VPN Provider A(AS1)

VPN Provider B(AS2)

TransitProvider(AS3)

VPN Provider B(AS2)

VPN Provider A(AS1)

MaintainsCustomer'sInternal Routes

MaintainsProvider A's Internal Routes

MaintainsProvider A'sInternal Routes

MaintainsProvider B's Internal Routes



IGP IGP IGP IGP IGP IGP

Recursive Multi-AS Operations

VPN Provider A(AS1)

VPN Provider B(AS2)

TransitProvider(AS3)

VPN Provider B (AS4)

VPN Provider A(AS5)


MaintainsProvider A's Internal Routes

MaintainsProvider A'sInternal Routes




IGPEBGP EBGP EBGP EBGP

IGP

MP-IBGPTransit Provider's External(Provider B's internal)

MP-IBGPProvider B's External(Provider A's Internal)

MP-IBGPProvider A's External(Customer's Internal)

MP-EBGPProvider B's External(Provider A's Internal) MP-IBGP

Transit Provider's External(Provider B's Internal)

MP-EBGPProvider A's External(Customer's Internal)



■ In the carrier-of-carriers scenario, because the BGP/MPLS VPN transit provider’s customer sites have the same AS numbers, PE routers use MP-IBGP to exchange external (VPN-IPv4) routing information. In contrast, for the multi-AS operations scenario, PE routers use MP-EBGP to exchange external (VPN-IPv4) routing information because the BGP/MPLS VPN transit provider’s customer sites have different AS numbers.

■ In the carrier-of-carriers scenario, the routing exchange between CE and PE routers usually occurs using an IGP, while LDP is used for label distribution because all VPN sites have the same AS number. In contrast, for the multi-AS operation scenario, the routing exchange between CE and PE routers is likely to occur using (single-hop) EBGP because the VPN sites have different AS numbers.

■ If (multi-hop) MP-EBGP or MP-IBGP runs between route reflectors, the route reflector must not set the BGP next hop to self, but rather propagate the BGP next hop included in the advertisement.

Acronyms

AS autonomous system

ASBR autonomous system boundary/border router

BGP Border Gateway Protocol

CE customer edge

EBGP external BGP

IBGP internal BGP

IGP Interior Gateway Protocol

IP Internet Protocol

ISP Internet service provider

IS-IS Intermediate System to Intermediate System

LDP Label Distribution Protocol

LSP label switched path

MPLS Multiprotocol Label Switching

OSPF Open Shortest Path First

P provider

PDU protocol data unit

PE provider edge

POP point of presence

RD route distinguisher



RFC Request for Comments

RR route reflector

RSVP Resource Reservation Protocol

SP service provider

VPN virtual private network

VRF VPN routing and forwarding

Copyright © 2001, Juniper Networks, Inc. All rights reserved. Juniper Networks is a registered trademark of Juniper Networks, Inc. Internet Processor, Internet Processor II, JUNOS, JUNOScript, M5, M10, M20, M40, and M160 are trademarks of Juniper Networks, Inc. All other trademarks, service marks, registered trademarks, or registered service marks may be the property of their respective owners. All specifications are subject to change without notice. Printed in USA.


Documents

RFC 2547bis: BGP/MPLS VPN Hierarchical and Recursive ... · BGP/MPLS VPNs, refer to the RFC 2547bis: BGP/MPLS VPN Fundamentals white paper. Perspective: BGP/MPLS VPNs Defined In traditional