Introduction to MPLS

Introduction to MPLSBob Franklin UIS Networks, The University of Cambridge

AgendaBackground - why did we start doing MPLSBasic MPLS operationSteps to enable MPLS Layer 3 VPNExample configuration & outputExample traffic flowOther features of MPLS

Background

Core (x2)Distribution (x15)PoP [=CPE] (x175)Dept./College (x200)

VLAN+STPVLANExtensionHell!AnnexeDept.

VLAN extension problemsDifficult to manage and expand have to create VLANs and manually add to core/distribution switches and portsNeed STP to handle redundancy (e.g. proprietary Cisco Rapid-PVST+)Layer 2 problems are exposed up from the distribution layer into the core and affect other routers (and institutions) + STP fails openAll routers have to learn MAC addresses on those VLANsRouting only happens on the home routersSTP blocks links and doesn't make full use of available bandwidthOnly uses defined paths: more redundancy comes at a cost of management overhead and stability

Splitting VLANs with routing

VLAN+STPVLAN2

access listsDHCP relayWoL directed broadcast95% success

"Inside" VLANs

Firewall

INSIDEOUTSIDE

Virtual Private Network

PRIVATEINSIDENETWORK

OUTSIDE

MPLS L3 VPN with Virtual Routing and Forwarding (VRFs) allows these to be the same physical routers and use all the inherent redundancy + paths

What is MPLS?

What is MPLSMulti Protocol Label Switching another way of forwarding traffic around the network:Ethernet switching forwards traffic based on MAC addressesIP routing forwards traffic based on IP addressMPLS forwards traffic based on labelsLabels added at ingress to the MPLS network and removed at egress (end hosts never see them)Unlike IP addresses, you do not explicitly choose the labels: the routers do choose them for youMPLS is NOT about switching packets faster but more cleverly!

MPLS router typesThere are two main types of MPLS router:Provider Edge (PE) routers sit at the edge of the MPLS network and change packets between non-MPLS (e.g. IP) and MPLS packetsProvider (P) routers sit in the middle of the network and interconnect PE and P routers but do not connect directly with customer networksYou dont explicitly configure the type: its determined by the services configured on the router

MPLS forwardingPE routers do one of two things before forwarding the packet:On ingress, push one or more labels onto the front of non-MPLS packet to turn it into an MPLS packet to be label-switchedOn egress, pop the label(s) off the front of an MPLS packet to turn it back into the original trafficP routers do one of two things before forwarding the packet:Swap the topmost label for a different onePop the topmost label off if the destination is a PE router ("Penultimate Hop Popping" [PHP] more later)You don't explicitly configure this: the routers work all this out automatically

Meaning of labelsLabels are arbitrary 20-bit numbers written in decimal (if you want to look at them you normally only do this when debugging) Packets can (and usually do) have more than one labelThe topmost label identifies the egress PE routerIt NEVER survives more than one hop: it will be swapped even though the destination router is the same (each hop router requires a different label for the same destination)Not needed if the next hop IS the egress router (instead do PHP)The second level label identifies the destination network and is only understood by the egress PE routerThe intermediate (P) routers only need to understand the topmost label to forward traffic towards the PE router

Enabling MPLSL3 VPN

Enabling MPLS checklistNeed capable hardware (e.g. Cisco Catalyst 6500 w/ PFC3B or 6800)Need appropriate router feature set / licencee.g. Advanced IP Services or Advanced Enterprise Services for Cisco Catalyst 6500/6800Increase MTU on inter-router linksEnable LDP (Label Distribution Protocol) on inter-router linksConfigure iBGP with VPN address familiesCreate VRFs (virtual routing instance)Create VLANs and SVIs (VLAN interfaces) in VRF

Interface MTUMPLS works by prefixing frames/packets with MPLS labels i.e. a type of encapsulation, similar to 802.1Q (although that is usually handled automatically)1500 is the default for IP over Ethernet1512 adequate for layer 3 VPN with Traffic EngineeringHowever, 1534 useful for layer 2 virtual circuits tunnelling Q-in-QWe use 1534 bytesSimplest to just set it onceOnly needs setting on inter-router linksFieldSize (bytes)Ethernet header(implicit)MPLS label 0(Traffic Engineering)4MPLS label 1(IGP/LDP)4MPLS label 2(VPN)4IP packet1500TOTAL1512

LDPLabel Distribution ProtocolMPLS routers use this to advertise labels for specific destination IPv4 prefixes in the global address space to neighbourse.g. "if you send traffic to me for 192.0.2.244/32, use label 1555" or "pop the top label if sending to 192.0.2.240/32" [PHP]Does NOT control the actual routing decision made by the sending router that is still determined by the IGP (OSPF, IS-IS, etc.), including supporting multipathUsed to determine the topmost (next hop) label to be used

LDP configurationBy default will advertise labels for ALL routes in the routing tableUsually configured to only advertise the loopback addresses of the routersThese are the next hop addresses used in the iBGP routes to be distributed (later)Once enabled ALL traffic to those addresses will be labelled: not just MPLS VPN traffic

iBGPMPLS L3 VPN needs to distribute VPN routes via BGP using the vpnv4 (IPv4 VPN) and vpnv6 (IPv6 VPN) address families:Regular BGP information:the destination prefix (e.g. 192.168.100.0/24)the next hop address (e.g. 192.0.2.244) in BGP, the next hop might be several hops away across the network; the IGP determines how to get there: this selects the topmost label, advertised by LDP, to be usedSpecial to MPLS VPN:the identifiers of the private network to which they belong (later) requires extended communities to be sentthe [second level] label to use for itNote that IPv6 VPNs use IPv4 peering addresses (as they use IPv4 next hop addresses and LDP to determine the topmost label)!

VRFsVirtual Routing and Forwarding instance local to a single routerA separate routing table from the global one (and each other), private to the VPNCan have overlapping routes (e.g. 192.168.1.0/24 in VRF 'finance' is distinct from 192.168.1.0/24 in VRF 'security')Must have a Route Distinguisher (RD)Can have one or more Route Targets (RT) to connect it with other routers in the MPLS VPN

Route Distinguishers (RDs)Just the prefix is no longer enough to make a unique route: there may be multiple 192.168.1.0/24s in separate VPNsThe RD is used to form a completely unique identifier for the route in form ::/Administrator Subfield represents the administrative authority: we use the public IPv4 loopback address of the router (but could be BGP ASN) so differs per router, in our caseNote: like OSPF router ID - not actually an IP address but just written in dotted-quad format and needs to be uniqueAssigned Number Subfield identifies the specific private network: we assign this internally and keep a register

Route Targets (RTs)Used to specify which routes will be imported to or exported from a VRF to create the VPN across routersSetting the export RT tags a route with it in BGPThe import RT identifies which routes are imported into the VRF on a particular routerAll routes in a particular VPN typically use the same RTHowever, can use different ones to create 'hub and spoke' VPNs and minimise the number of routes on satellite PE routersSame format as RT we use the IPv4 netblock address for our loopbacks and the same assigned number

Example

Configuring VRF and BGPvrf definition MINCE-VRF rd 192.0.2.238:811 route-target export 192.0.2.0:811 route-target import 192.0.2.0:811 ! address-family ipv4 exit-address-familyrouter bgp 64602 address-family ipv4 vrf MINCE-VRF redistribute connected redistribute static maximum-paths ibgp 2 exit-address-familyinterface Vlan789 description mince-nms vrf forwarding MINCE-VRF ip address 10.0.1.253 255.255.255.0 no ip proxy-arp standby version 2 standby 81 ip 10.0.1.254 standby 81 priority 200 standby 81 preemptPut interface in VRFRoute Distinguisher (RD)Route Targets (RTs)Redistribute routes in VRF via BGP

DIST-NMS#show ip route vrf MINCE-VRFVRF routing tableDIST-NMS#show ip route vrf MINCE-VRF

Routing Table: MINCE-VRFCodes: L - local, C - connected, S - static, R - RIP, M - mobile, B - BGP D - EIGRP, EX - EIGRP external, O - OSPF, IA - OSPF inter area N1 - OSPF NSSA external type 1, N2 - OSPF NSSA external type 2 E1 - OSPF external type 1, E2 - OSPF external type 2 i - IS-IS, su - IS-IS summary, L1 - IS-IS level-1, L2 - IS-IS level-2 ia - IS-IS inter area, * - candidate default, U - per-user static route o - ODR, P - periodic downloaded static route, H - NHRP, l - LISP + - replicated route, % - next hop override

Gateway of last resort is not set

10.0.0.0/8 is variably subnetted, 5 subnets, 3 masksC 10.0.1.0/24 is directly connected, Vlan811L 10.0.1.253/32 is directly connected, Vlan811B 10.2.1.0/24 [200/0] via 192.0.2.244, 7w0d [200/0] via 192.0.2.234, 7w0dB 10.254.1.0/30 [200/0] via 192.0.2.234, 7w0dB 10.254.1.4/30 [200/0] via 192.0.2.244, 7w0d`Directly connected interface`[Multipath] MPLS VPN routes learnt via BGP

DIST-NMS#show bgp vpnv4 unicast vrf MINCE-VRF 10.2.1.0/24BGP informationDIST-NMS#show bgp vpnv4 unicast vrf MINCE-VRF 10.2.1.0/24BGP routing table entry for 192.0.2.238:811:10.2.1.0/24, version 1978Paths: (2 available, best #2, table MINCE-VRF)Multipath: iBGP Not advertised to any peer Refresh Epoch 6 65108, imported path from 192.0.2.244:811:10.2.1.0/24 (global) 192.0.2.244 (metric 21) from 192.0.2.240 (192.0.2.240) Origin incomplete, metric 0, localpref 100, valid, internal, multipath(oldest) Extended Community: RT:192.0.2.0:811 Originator: 192.0.2.244, Cluster list: 192.0.2.0 mpls labels in/out nolabel/749 rx pathid: 0, tx pathid: 0 Refresh Epoch 2 65108, imported path from 192.0.2.234:811:10.2.1.0/24 (global) 192.0.2.234 (metric 21) from 192.0.2.240 (192.0.2.240) Origin incomplete, metric 0, localpref 100, valid, internal, multipath, best Extended Community: RT:192.0.2.0:811 Originator: 192.0.2.234, Cluster list: 192.0.2.0 mpls labels in/out nolabel/949 rx pathid: 0, tx pathid: 0x0`Remote VRF's RD + prefix

Exported RTs on remote VRFImported to local VRF

VPN (second level) label

BGP next hop`Local VRF's RD + prefix(Other [multi]path entry)

DIST-NMS#show ip cef vrf MINCE-VRF 10.2.1.50 detailDIST-NMS#show mpls ldp bindings 192.0.2.244 32DIST-NMS#show ip route 192.0.2.244DIST-NMS#show ip cef vrf MINCE-VRF 10.2.1.50 detail10.2.1.0/24, epoch 1, flags rib defined all labels, per-destination sharing NetFlow: Origin AS 0, Peer AS 0, Mask Bits 24 recursive via 192.0.2.234 label 949 nexthop 192.0.2.33 TenGigabitEthernet1/1 label 1563 nexthop 192.0.2.113 TenGigabitEthernet1/2 label 27 recursive via 192.0.2.244 label 749 nexthop 192.0.2.33 TenGigabitEthernet1/1 label 1555 nexthop 192.0.2.113 TenGigabitEthernet1/2 label 951DIST-NMS#show mpls ldp bindings 192.0.2.244 32 lib entry: 192.0.2.244/32, rev 4149local binding: label: 38remote binding: lsr: 192.0.2.250:0, label: 951remote binding: lsr: 192.0.2.240:0, label: 1555Forwarding and LDPDIST-NMS#show ip route 192.0.2.244Routing entry for 192.0.2.244/32 Known via "ospf 1", distance 110, metric 21, type intra area Last update from 192.0.2.113 on TenGigabitEthernet1/2, 6d07h ago Routing Descriptor Blocks: 192.0.2.113, from 192.0.2.244, 6d07h ago, via TenGigabitEthernet1/2 Route metric is 21, traffic share count is 1 * 192.0.2.33, from 192.0.2.244, 1w0d ago, via TenGigabitEthernet1/1 Route metric is 21, traffic share count is 1

IGP next hop label via 192.0.2.240 from LDP

IGP next hopIGP next hop towards egress PE router

VPN (second) label from BGP

Topmost label

MPLS forwarding example

DstIPData10.2.1.50xxx

NH LblVPN LblDstIPData155574910.2.1.50xxxPE-E from P-1mince

VPN LblDstIPData74910.2.1.50xxxmince

DstIPData10.2.1.50xxx

PE-E192.0.2.244PE-I192.0.2.238

NH LblVPN LblDstIPData62674910.2.1.50xxxPE-E from P-2mince

P-2P-1192.0.2.240

2. Ingress PE pushes new labels3. P-1 swapstopmostlabel4. Penultimate P-2 popstopmost label (PHP)5. Egress PE pops label to selectVRF, restoring original IP packetSource sends IP packet

Other stuffEthernet over MPLS (EoMPLS) allows point-to-point layer 2 virtual circuitsVirtual Private LAN Service (VPLS) allows multipoint layer 2 services (like a VLAN)MPLS Traffic Engineering (TE) allows circuits with constrained paths (loose routes, bandwidth reservation) to be establishedMulticast supported through Multicast LDP (MLDP)BGP peerings inside a VPN ("carrier's carrier")Supports QoS (through Experimental bits)

End