METROPOLITAN ETHERNET DESIGN FUNDAMENTALS · 2017-10-10 · Metro Ethernet Connectivity Ethernet Wire Service • Features Point-to-point connectivity Carrier network transparency

© 2004 Cisco Systems, Inc. All rights reserved. Printed in USA.Presentation_ID.scr

1© 2004 Cisco Systems, Inc. All rights reserved.OPT-10429816_05_2004_c1

METROPOLITAN ETHERNET DESIGN FUNDAMENTALS SESSION OPT-1042


Architecture and Design ConsiderationsSP and Enterprise—QoS Model

Metro Ethernet: Services Drive Transport

Metro Ethernet ServicesEnterprise Drivers

SP and Enterprise—CPE Considerations

Agenda

© 2004 Cisco Systems, Inc. All rights reserved.OPT-10429816_05_2004_c1



Enterprise FocusProtect, Optimize and Grow Business

GROW REVENUE

OPTIMIZE COSTS

INCREASING PRODUCTIVITY

ADDRESSING UNCERTAINTIES

• Being prepared for the unpredictable

• What happens if there is a disaster at the headquarters site?

• Compliance with new regulation

• Being prepared for the unpredictable

• What happens if there is a disaster at the headquarters site?

• Compliance with new regulation

• Lowering Total Cost of Ownership (TCO) directly impacts profitability

• Doing something at a lower cost through technology investment and new business model

• Lowering Total Cost of Ownership (TCO) directly impacts profitability

• Doing something at a lower cost through technology investment and new business model

• Saving employees time• Improve responsiveness• Doing more with less• Improving business processes

• Saving employees time• Improve responsiveness• Doing more with less• Improving business processes

• Deliver better customer value• Pursue new growth opportunities• Build competitive advantage

• Deliver better customer value• Pursue new growth opportunities• Build competitive advantage

ENTERPRISE DRIVERS


Business Driven InitiativesThe Network Is the Key Enabler

• Distributed data centers• Business continuity• Disaster recovery• Remote storage• Secure networks

• Server consolidation• Storage area networking• Data/voice convergence• Virtualization• New IT model: On-demand/

outsourced

• Multimedia office applications• Distributed applications• Web-based applications• Application integration

• Customer relationship management

• Data warehousing• Customer portals

IMPROVING CUSTOMER VALUE

LOWERING COSTS

INCREASING PRODUCTIVITY

ADDRESSING UNCERTAINTIES

ENTERPRISEDRIVERS



Applications Driving Ethernet

Source: The Yankee Group, 2003

31

33

46

49

53

75

84

49

47

39

33

36

18

13

0 20 40 60 80 100

VoIP

Videoconferncing

BusinessContinuity

Extranet

VPN

LAN-to-LAN

Internet Access

CurrentlyUse

Will Deployin <24Months


Enterprise Requirements and Expectations from Service Providers

Classes of Service

Service Uptime Management

Multicast

Security

Service Level Agreement (SLA) Characteristics

• Cost Effective• Investment Protection• Interworking

• Analogous to WAN classes of service (4 levels or more, shaping and rate limiting)

• LAN extension, priority and non-priority, CIR and PIR

• High availability

• Resiliency/ redundancy

• Customers would pay a premium for dual redundancy

• Secure private networks (VPNs)

• Protection against hackers

• Mechanisms to prevent DOS

• Firewall/IDS

• Authentication/ login

• Minimal management overhead for provisioning from end-user perspective

• Bandwidth adjustment

• Self-provisioning may become a tie breaker

• Networks todaynot suited for any-to-any voice/video/ collaboration type of traffic

Protocols Handling

• Large routing domain between SP and enterprises

• Transport of enterprise L2 PDUs across SP network

CISCOVALUE



Network Uptime

Network Uptime

• High availability

• Resiliency• Redundancy• MTTR

Classes of Service

Classes of Service

SecuritySecurity ManageabilityManageability

• Centralized• Single login• Analysis/

planning tools• OSS

The Challenges of Metro EthernetThe Challenges of Metro Ethernet

Comprehensive Platform Capabilities to Address Enterprise Requirements

• Access control• Authentication/

login• Encryption• Client security• Firewall/IDS

• Multiple CoS• Policing• Traffic

classification• Congestion

avoidance• Scheduling

CISCOVALUE






Agenda




Analogous to Private Line over SONET/SDH/xWDM Network

Analogous to Frame Relay using VLANs for Multiplexing

Similar to a Leased Line over a Packet Network

Virtual Transparent LAN

Summary of Ethernet-Based Services

Point-to-Point Multipoint

Layer 2 Layer 3Layer 1

EthernetPrivate

Line

EthernetRelay

Service

Ethernet-Based ServicesEthernet-Based Services

EthernetPrivate

Ring

EthernetMultipoint

Service

EthernetWire

Service

EthernetRelay

MultipointService

MPLSVPN

Private LAN Service


Metro Ethernet Connectivity Ethernet Wire Service

• FeaturesPoint-to-point connectivity

Carrier network transparency

Tiered service offering based on bandwidth, CoS, distance

L2 transparency

SLA capability based on classes of service

Bandwidth granularity

• Sample SP service offeringEthernet local loop

Ethernet access to providers

Dedicated Internet access

SERVICE DESCRIPTION

Enterprise B

Enterprise C(HQ)

Enterprise C Branch Office

ISP PoPSP PoP

Enterprise A

IP VPNInternet

Metro EthernetService Provider

Network



Metro Ethernet Connectivity Switched Ethernet (Relay) Service

• FeaturesPoint-to-multipoint—Hub and spokeUses SP assigned VLAN IDService multiplexingScalability for large sitesService tiering based on bandwidth, CoS, distanceNo L2 BPDU transparencySLA—CIR/PIR/Burst, lossFR/ATM Interworking

• Sample SP service offeringRemote branch connectivityInternet accessInternet/Intranet/Extranet

SERVICE DESCRIPTION

MetroBranch 1

Metro HQ

Multiple EVCs at UNI

MetroBranch 2

MetroBranch 3

Metro EthernetService Provider

Network

CPE-Router


• FeaturesERS UNI that maps to MPLS VPN on PEL3 Multipoint service that maps VLANs to VRFsService multiplexed UNI (e.g. 802.1Q trunk)Opaque to customer PDUs (e.g. BPDUs)

• Sample ApplicationsRemote branch connectivityInternet access Internet/Intranet/Extranet

Metro Ethernet Connectivity L2 Access to MPLS VPN

MetroMetroBranch 1Branch 1

Metro HQMetro HQ

Multiple EVCs Multiple EVCs at UNIat UNI

MetroMetroBranch 2Branch 2

SP POPSP POP

CPE-Router

SERVICE DESCRIPTION

ISP

Blue VRF OrangeVRF



Metro Ethernet Connectivity Ethernet Multipoint Service

• FeaturesMP Any-to-any LAN 10/100/1000Mbps Ethernet customer interfaceRate limiting possibleL2 transparencyService Tiering based on bandwidth, CoS, distanceSLA—CIR/PIR/Burst, loss

• Sample SP service offeringCorporate/campus LAN extension Cost effective large bandwidth LAN Extension over WANSimplicity/transparency

SERVICE DESCRIPTION

HQ MetroBranch—1

Metro Branch—2

Data Back-UpSite

Service ProviderNetwork

SP VLAN

CPE-Router/Bridge


Metro Ethernet Connectivity Ethernet Private Line/Ring Service

• FeaturesPoint-to-Point or MultipointUses Sonet/SDH or RPR transportDedicated bandwidthHigh availability-protectedScalabilitySimple SLA—Uptime

• Sample SP service offeringMission criticalTypically Intra-MetroInternet accessData centersBusiness continuityHQ/campus ringNetwork consolidation

SERVICE DESCRIPTION

Secondary Data Center

Primary Data Center

Service Provider NetworkSONET/SDH/RPR

Metro Branch—2

HQ

Metro Branch—1

CPE-Router/Bridge







Agenda



Metro Ethernet Network Architecture

• Characteristics of each architecture-element/layer technology agnostic

Consistent Ethernet services

• Different technological solutions can co-exist within one network

Ethernet is usually the access/UNI, not necessarily the entire network

• Elements of different technological solutions can be combined—building block approach

• Transport protocols and topologies aredeployment options



MPLS/IP/TDM

N-PE

N-PE

N-PEP P

PP

GE Ring

Metro A U-PEPE-AGG

Metro C

U-PE

DWDM/CDWM

U-PE

User Facing Provider Edge (U-PE)

Network Facing Provider Edge (N-PE)

Metro Ethernet Architecture and Terminology

U-PE

RPR

Metro D

Full ServiceCustomer Equipment


Large ScaleAggregation

IntelligentEdge

MultiserviceCore

Efficient Access

Integrated SystemIntelligent

EdgeEfficientAccess

SiSi

SiSi

Metro B

10/100/1000 Mbps

10/100/1000 Mbps

10/100/1000 Mbps

10/100/1000 Mbps

Hub andSpoke


Emulated VC

(Pseudowire)

Extension VC

Attachment VC

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG

Metro Ethernet Network Architecture

Core Device (P)Frame Forwarding, Congestion Management

Emulated VC Endpoint (N-PE)*MPLS/L2TPv3 Pseudowire Encapsulation, L2 Interworking, IP Service Integration, Congestion Management, L3VPN Interconnection (for PEs Supporting that function)

Aggregation Device (PE-AGG)Traffic Aggregation and Congestion Management(Note: S-P and Extension VC’s Are Formally Identified in Switched Ethernet Access Domains, although the Functions Exist in FR and ATM Networks as Well)

Attachment VC UNI Endpoint (U-PE)*Admission Control, Policy Enforcement, Classification, Policing and Marking, Congestion Management, SLA Monitoring and Reporting, VC Mapping to L1 Channel, VC-ID Translation

VPLS Bridging

Specific for Ethernet:

Bridging

Specific for Ethernet:

These Different Roles Can Be Collapsed within a Single Box*draft-ietf-l2vpn-l2-framework-04.txt



Access Layer

• Service and admission control policies of the network

Security—802.1x authentication, port based security

• Traffic multiplexing and congestion managementQoS—classification, policing, marking and queuing, 802.1p bit mapping

• Copper and optical interfaces• Service definition layer

EMS, ERMS and EWS—L2PT, Tag Stacking (Q-in-Q)L3VPN—VRF-lite, VLAN taggingMapping function: “VPN Mapping” to a VLAN to SONET/SDH circuit, VLAN to EoMPLS tunnel,VRF lite to MPLS VPN

User-Facing Provider Edge: U-PE

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG


Aggregation Layer

• Efficient aggregation of traffic to higher speed connections

• Traffic multiplexing and congestion management

• Local switching for Ethernet services• Sparse topologies may not require an

aggregation layer

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG

Aggregation Device: PE-AGG



Service Application Layer

• High density optical interfaces • High-speed switching • Sophisticated traffic and congestion

management• MPLS and IP service gateway

VPLS and VPWS service definition layerL2VPN service inter-working gatewayL3VPN service layer

• High-touch Layer 3 service application device Content services, firewall, intrusion detection, etc.

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG

Network-Facing Provider Edge: N-PE


Core Layer

• High-speed packet forwarding

• Sophisticated traffic management

• Highly available• High-speed optical interfaces

OC-48/STM-16, OC-192/STM-64

GE, 10GE

• Convergence of packet-processing and optical (circuit-based) technology and (dependent on installed base) ATM, etc.

Core Node: P

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG



Metro Ethernet Network ArchitectureConnectivity Options: Behind the Clouds

• Relationship between layers/functional elements and components defines protocols, topologies and their deployment

Scalability

Topology—ring vs. hub and spoke

Protocols

Cost—fiber consumption, interface costs

Availability

STP convergence vs. SONET/SDH/RPR

Dual-homing/redundancy

SLAs

Fair and secure access, consistent SLA—e2e QoS

Service ubiquity—access over any technology/protocol

CE

PP

NN--PEPE

UU--PEPE

PEPE--AGGAGG


Metro Access Network May Be Deployed with Different Technologies…

(Migrate Rings withNew Low Cost Direct Connections)

MetroCore

Ethernet Using Spanning Tree(Inexpensive Interfaces,“Enterprise” Protocols)

DPT/RPR(Spatial Reuse for Local Traffic)

DWDM/CWDM(Point-to-Point Behavior without New Fiber)

Local Traffic

Local Traffic

Local TrafficLocal

Traffic

MetroCore



Metro Access Networks: Transport Options

Switched Gigabit EthernetHub and Spoke

Switched Gigabit EthernetHub and Spoke

Switched Gigabit EthernetRing

Switched Gigabit EthernetRing

DPT/RPRDPT/RPRDWDM/CWDMDWDM/CWDM

• Lower cost solution• Perceived simplicity of Ethernet

switching• Can be built on a fiber ring

infrastructure with CWDM• Consistent delay/jitter characteristics• Foundation for Ethernet/IP L2/3 VPN

• Lower cost solution• Flexible bandwidth• Easy to deploy over dark fiber• Sub-second convergence• The node position within the ring

influences delay/jitter, convergence time• Foundation for Ethernet/IP L2/3 VPN

• Shared packet ring scales bandwidth up to 5 Gbps today

• SONET/SDH framing provides insertion point for many providers

• Large number of nodes per ring• 50 ms convergence• Foundation for Ethernet/IP L2/3VPN

• Scales fiber capacity8Gbps, 320Gbps, 800Gbps

• Convergence dictated by xWDM solution

• Cost effective • Easy to deploy• Foundation for all services—enables

storage, etc. as well


Wire Center

Case Study: ILEC/PTT in Region

A Design Alternative for Markets with Dark Fiber Availability• Enough dark fiber to each customer

SP CoS Marking and Traffic Concentration Occurs at CO/POP Location

Dedicated Fiber for Every CE Connection

CustomerPremise

CustomerPremise

Customer PremiseCE Tx—Fx

CO/POPDistribution

Core

Fx—Tx

To MLPSBackbone



Wire Center

CO/POPDistribution

Core

To MLPSBackbone

Customer PremiseCE Edge



SP CoS Marking and Traffic Concentration Occurs at Customer Location

A Good Design for High Density Areas with:• Large multi-tenant buildings, and • Dark fiber available only to the buildings

Hub and Spokefrom Dark Fiber

Case Study: ILEC/PTT Out of Region/IXC


Wire Center

Case Study: ILEC/PTT in Region

Customer PremiseCE 15454

POP“U-PE”

N-PE

To MLPSBackbone

G

G15454

4000

SP CoS Marking Occurs within 4K at POP

Dedicated 15454 and 4000 Ports for Every CE

Dedicated Channelized Bandwidth for Every CE Connection

Multiple L2/L3 Boxes Needed in POP (4K Also Deployable at Customer Premises)

A Good Design for Buildings:• Without dark fiber, or with low bandwidth requirements • Without multiple customers (no need for local U-PE)

7600

SONET

Access SONET


303030

SERVICE ENABLERS



Service Traffic Patterns

L2

Core (IP/MPLS)

L2

Intra-EAD ServicesInter-EAD Services

Intra-EAD Services• Defined as services that are

contained within a L2 Ethernet Access Domain (EAD)

Inter-EAD Services• Defined as services that traverse

multiple L2 Ethernet Access Domains (EAD) over an IP/MPLS core



802.1Q Tunnelling (aka Q-in-Q)

• SP doesn’t coordinate CE VLANs (CE VLANs transparency) • CE VLANs can overlap• Increased VLAN space (4k VLANs x 4k VLANs) 802.1Q Trunk

802.1Q Tunneling




IntelligentEdge

MultiserviceCore

Efficient Access


EdgeEfficientAccess

SiSi

U-PE PE-AGG N-PE U-PEN-PEP

CE CE

CEVLAN_ID

100

802.1P802.1P

Etype0x8100

FCS4

bytesDataData

00––1500 bytes1500 bytes

Len/Type

2 bytes

.1Q.1Q4 4

bytesbytes

SMAC6 bytes

DMAC6 bytes

2 bytes 3 bits 12 bits

CEVLAN_ID

100

802.1P802.1P

Etype0x8100

FCS4

bytesDataData

00––1500 bytes1500 bytes

Len/Type

2 bytes

.1Q.1Q4 4

bytesbytes

SMAC6 bytes

DMAC6 bytes

2 bytes 3 bits 12 bitsFCS

4 bytes

DataData00––1500 bytes1500 bytes

Len/Type

2 bytes

.1Q.1Q44

bytesbytes

SMAC6 bytes

DMAC6 bytes

CEVLAN_ID

100

802.1P802.1P

Etype0x8100


.1Q.1Q4 4

bytesbytes

SPSPVLAN_IDVLAN_ID

200

802.1P802.1P

Etype0x8100



1:1 VLAN Translation

• CE VLAN preservation for ERS Services

• SP does not enforce VLAN IDs for ERS Services

• VLANs from different CEs may overlap, SP will translate them into different and unique SP VLAN IDs

VLAN 12 VLAN 12VLAN 152

802.1Q Trunk

Data.1QTAG

VLAN 12SMACDMAC Data

.1QTAG

VLAN 12SMACDMACData

.1QTAG

VLAN 152SMACDMAC



IntelligentEdge

MultiserviceCore

Efficient Access


EdgeEfficientAccess

SiSi


CE CE



2:1 VLAN Translation (Double VLAN Translation)

• Adds flexibility to ME Services based on Q-in-Q

• Allows to multiple different services on the same SP Q-in-Q interface

VLAN 111 +VLAN 15 VLAN 11+VLAN 15 VLAN 11VLAN 11 VLAN 11

Data.1QTAG

VLAN 111SMACDMAC

.1QTAG

VLAN 15Data

.1QTAG

VLAN 11SMACDMAC

.1QTAG

VLAN 15Data

.1QTAG

VLAN 11SMACDMAC

802.1Q Trunk802.1Q Tunneling



IntelligentEdge

MultiserviceCore

Efficient Access


EdgeEfficientAccess

SiSi


CE CE


2:1 VLAN Translation: Application Example

• L3 VPNTerminates both VLANs tags based on outer/inner 802.1Q tags combo

• The CEs see an EMS service while the N-PE sees an ERS service

SPNetwork

VLANs 400, 4000 N-PE2525Q

CE1

VLAN 1002525

2525I

Q

Q

Q

I

802.1Q Tunneling UNI

2:1 VLAN Translation Point

MP2MP Q-in-Q-tunnelCE2

CE3

VLAN 4000

VLAN 4000

VLAN 200IP-VPN

Internet

200< 25, 4000>100< 25, 400>

Translated VLAN ID

Outer/Inner 802.1Q Combo



EoMPLS

• Layer 2 tunneling technology to forward Ethernet frames across an MPLS network

• Allows connectivity between remote sites without the extension of spanning tree domains in service provider network

• EoMPLS Connections appear to be a point-to-pointlink between customer locations

• Simple to provision, no IP routing is needed between CE and PE

• Uses a pseudowire concept for connectivity between PE’s over a MPLS network


Deploying EoMPLS

L2IP/MPLS

L2

EoMPLS

L2 Point-to-Point Services

LOGICAL

Frame-Relay (H&S)

How Is it Possible to Offer Point-to-Point Inter-EAD

Ethernet Services over an IP/MPLS Core?

ERS and EWS Can Be Deployed within the L2

Domain Using Local Switching

A

B

C



VPLS

• ArchitectureIt is an end-to-end architecture that allows IP/MPLS networks toprovide Layer 2 multipoint Ethernet services while using LDP as

signaling protocol

• Bridge emulationEmulates an Ethernet bridge

• Bridge functionsOperation is the same as for an Ethernet bridge, ie forwards using the destination MAC address, learns source addresses and floods broad-/multicast and unknown frames

• Several drafts in existencedraft-ietf-l2vpn-vpls-ldp-01.txt

draft-ietf-l2vpn-vpls-bgp-01-txt


Deploying VPLS

IP/MPLS

L2

L2

VPLS!!

L2 Multipoint Services

How Is it Possible to Offer Multi-Point Inter-EAD Ethernet

Services over an IP/MPLS Core?

L2

L2

LOGICAL

A

B

CD

A B

CD


SECURITY



Protect Against DOS Attacks or Limited Resource Contention

VLAN 3VLAN 3VLAN 2VLAN 1


Metro Ethernet Trust Model

Untrusted Trusted

Protect from Compromised U-PEAuthenticate Customer UNI

Customer Protection

Network Protection

VLAN 2

(QinQ) VLAN 5(QinQ) VLAN 5


VLAN 1

PE

CE VLAN 1CE VLAN 2

Premises

POP

POPSwitch(N-PE/PE-AGG)

PremisesSwitch(U-PE)

10/100/1000

10/100/1000

10/100/1000VCs

Ensure the Configuration Can’t Be Accessed and Modified

Mostly Trusted

Gigabit Ethernet Transport



Attacks and Defensive Features/Actions

Secure Variants of Management Access Protocols—Not Telnet etc., but SSH,… and out of Band Management)Hijack Management Access

DHCP Snooping (Differentiate Trusted andUntrusted Ports)DHCP Rogue Server Attack

BPDU Guard, Root Guard, MD5 VTP Authentication Spanning Tree Attacks

Careful Configuration (Disable Auto-trunking, Used Dedicated VLAN-ID for Trunk Ports, Set User Ports to Non-trunking, Avoid VLAN 1, Disable Unused Ports,…)

VLAN Hopping, DTP Attacks

Private VLANs, Wire-Speed ACLs, Dynamic ARP Inspection

ARP Attacks (ARP Spoofing, Misuse of Gracious ARP)

Port Security, Per VLAN MAC LimitingMAC Attacks (CAM Table Overflow)

Defensive Features/ActionsAttack

Deploy MAC Level Port Security, Wire-Speed ACLs, 802.1xPro-Active Defence


Ethernet Security: SP Recommendations

NV 66VLAN 5VLAN 20 VLAN 30VLAN 40

Customer—SPBoundary

802.1QTrunk UNI

802.1QTrunk

CPE

Access

CoreSP

IP/MPLS/802.1Q

NetworkNV 5 NV 66Untagged

VLAN 10VLAN 20 VLAN 30VLAN 40

CE BPDUCE BPDUSP BPDUSP BPDU

X

X

X

VTP Mode TransparentEnable ROOT GuardPer VLAN MAC Limiting

Disable Password RecoveryVTP Mode Transparent

LOOP GuardPrune All Unused VLANs from Allowed ListRemove VLAN 1 and Reserved VLANs from TrunksReserve a VLAN ID for theNative VLAN on the SP Trunks

BPDU Filter (for Egress SP BPDU)MAC ACLs (for Ingress CE BPDU)

Enable Port SecurityEnable 802.1XDisable CDPRemove VLAN 1 and Reserved VLANs from UNIsSet DTP to “Non-Negotiate”Prune All Unused VLANs from Allowed ListUNI VLANs Must Not Be Used as Native VLANon SP Trunks


QUALITY OF SERVICE



Overview of QoS Functions

1 2

34

QoSQoS

Classif

icatio

n

and M

arkin

g

Policing

Queuing

Congestion

Avoidance



CoreEdgeAggregation CustomerEquipment

CustomerEquipment

Access AccessEdge

StepsSteps

Scheduler DropPolicer Drop

22

Classification, Marking Classification, Marking and Policingand Policing

3 3 3 3

Classification andClassification andQueuingQueuing

Scheduling, Bandwidth Management and

Congestion Avoidance

11 3

11

22 22 22 22

QoS Functions:What QoS Functions Happen at Each Area within the Network ?


What SLAs Can I Expect?

• One SLA per port: Best effort, CIR, or Voice on a port basis

EWS Service ClassU-PE

Best Effort 802.1p Cos=0

CE VLAN 103CE VLAN 102CE VLAN 101CE VLAN 100

PEVLAN 802.1Q

Tunnel802.1QTunnel

Business 802.1p Cos=2


PEVLAN 802.1Q

Tunnel802.1QTunnel

Best Effort 802.1p Cos=0


PEVLAN 802.1Q

Tunnel802.1QTunnel

EMS Service Class



802.1p Cos=0

802.1p Cos=0802.1p Cos=3802.1p Cos=3802.1p Cos=5802.1p Cos=5

802.1p Cos=0802.1p Cos=3802.1p Cos=3802.1p Cos=5802.1p Cos=5

VLAN 202 802.1p Cos=5VLAN 202 802.1p Cos=5VLAN 201 802.1p Cos=2VLAN 200 802.1p Cos=0

What SLAs Can I Expect?

• Multiple SLAs per port: Best effort, CIR/PIR or voice on a VLAN basis• Multiple SLAs per VLAN: Best effort, CIR/PIR or voice on a class basis

(classified based on L2 COS, IP ToS, outer/inner VLAN)

VoiceVoiceBusiness Critical

Best Effort

BusinessCritical

+Voice

Data All Other DSCPVoice Control DSCP 24/26Voice Control DSCP 24/26Voice DSCP 46Voice DSCP 46

VLAN203

VLAN203

BestEffort

+Voice

BestEffort

+Voice

Data All Other DSCPVoice Control DSCP 24/26Voice Control DSCP 24/26Voice DSCP 46Voice DSCP 46

VLAN204

VLAN 200 Best Effort ERMS802.1QTrunk

ERMS802.1QTrunk

ERS UNI802.1QTrunk

ERS UNI802.1QTrunk

ERS Service Class

ERMS Service Class

U-PE


End to End Classification/Marking Model:How Is Traffic Classified and Marked Between Domains?

DiffServ CodePoint (DSCP) 802.1p MPLS EXPMPLS EXP 802.1p DiffServ Code

Point (DSCP)

534

21

0

802.1p COS

534

Real TimeAVVID Voice Transport

AVVID Call Control

Interactive Video

21

Business CriticalCIR

PIR

0Best Effort

MPLS EXPClasses of Service




IntelligentEdge

MultiserviceCore

Efficient Access


EdgeEfficientAccess

SiSi


CE CE



• Queuing behavior APPEARS consistent across EVERY hop• CIR and PIR in same queue ensures no packet re-ordering• Best effort doesn’t always have to be discarded in favor of CIR• Traffic engineering based on offered load determines proper queue

allocations; this will require experience to tune properly

Per Hop Queuing:CoS MPLS Value Mapping

WRRWRR

UNI UNI

SP Network

10%

80%CIRand PIR

BestEffort

4%

(PQ)PriorityQueue00

22

55

33

77Signalingand Mgt.

11

77SP Network Mgt66Unused

55AVVID Voice Transport

44Interactive Video33AVVID Call Control

22Business Critical (CIR)

11PIR (planned)00Best Effort

MPLS EXP

802.1p COSQueue

SP Network

Critical

VoIPSNMP Alarms

Best Effort

VoiceSignaling 5%


802.1Q Tunneling Enhancement(CoS Mutation)

• QoS marking preserved also on Q-in-Q interfaces

• Multiple Classes of Service bundles on the sameQ-in-Q interface

VLAN 100CoS 5

VLAN 100CoS 5

VLAN 152CoS 5

Data.1QTAG

VLAN 100CoS 5

SMACDMAC.1QTAG

VLAN 152CoS 5

Data.1QTAG

VLAN 100CoS 5

SMACDMAC

802.1Q Trunk802.1Q Tunneling

Data.1QTAG

VLAN 100CoS 5

SMACDMAC

CoS Mutation Table

7766554433221100



IntelligentEdge

MultiserviceCore

Efficient Access


EdgeEfficientAccess

SiSi


CE CE


NETWORK AVAILABILITY



Network Availability

Optimize

d

Network

Design

Protoco

ls

Redundan

cy

Resiliency

Hardware

Redundancy

NetworkNetworkAvailabilityAvailability



Network Availability

Access Rings

Dual Homing of CE

EtherChannel UNI

Multiple Tiers of Aggregation

Multiple CEConnections

to a PESP Network

EthernetAccessDomain



Pseudowires


Unidirectional Link Detection (UDLD)

• Cisco proprietary protocol

• Detects uni-directional links due to GBIC failures or fiber strands misplaced (tx and rx swapped)

CoreEdgeAggregationAccess

Port in Forwarding StatePort in Blocking State

FF

BB

MPLS



FF


• Link might become uni-directional

Without UDLD:• Spanning tree loops might occur

• It takes time to detect a change in the forwarding topology



FFGBICRx Failure

STP Loop MPLS


FF


• Link might become uni-directional

With UDLD:• The affected interfaces are error-disabled

• Spanning tree detects immediately the change in the forwarding topology



GBIC Failure

Err-disable

MPLS



FF


Recommendations:• UDLD in “aggressive” mode

• UDLD enabled on all non-UNI physical interfaces



GBIC Failure

Err-disable

MPLS


Spanning Tree PortFast

• After the link comes up, the port moves into forwarding state by-passing the intermediate STP states

• To be enabled on the edge ports (UNI)



MPLS

PortFast Enabled

BLK > FWD

BLK > LSTN > LRN > FWD30 Seconds

PortFast Disabled



EtherChannel

• PAgP (Cisco proprietary) or IEEE 802.3ad • To provide link redundancy• To increase the aggregate bandwidth• To load-balance the traffic based on sMAC/dMAC



MPLS


EtherChannel

• To provide link redundancy

Without EtherChannel:• Link redundancy is offered by spanning tree protocol, which blocks one link



MPLS

FF FF

FFBB



EtherChannel


Without EtherChannel:• When one physical link fails, spanning tree identifies the alternate

forwarding path



MPLS

FF

BB FF

Link Failure< 1 sec


EtherChannel


With EtherChannel:• When one physical link fails, the logical port “stays up”

(single port EtherChannel)



MPLS

FFLink Failure

FF



EtherChannel


With EtherChannel:• Traffic is switched across the active link within < 200 msec, without spanning

tree protocol intervention



MPLS

FF

< 200 msecFF


EtherChannel

• To increase the aggregate bandwidth towards “C” endpoint

Without EtherChannel:• Since one redundant link is blocked by spanning tree, the link can only

accommodate 1 Gigabit of traffic traffic loss



MPLS

FF

BB

1 Gb Capacity 1 Gb Capacity

1 Gb Capacity

FF

FF 50% Loss

A B

C



EtherChannel

• To increase the aggregate bandwidth towards “C” endpoint

With EtherChannel:• By bundling 2 physical interfaces, the logical link can accommodate up to 2

Gigabits of traffic no data loss



MPLS

FF

FF

1 Gb Capacity 1 Gb Capacity

2 Gb Capacity

0% Loss

A B

C


EtherChannel

• To load-balance the traffic based on sMAC/dMAC

Without EtherChannel:• All the traffic will traverse a single link, since the redundant path is blocked

by spanning tree



MPLS

FF

BB

FF

FF



EtherChannel

• To load-balance the traffic based on sMAC/dMAC

With EtherChannel:• Traffic is load-balanced across the links in the EtherChannel, accordingly to

the criteria configured (sMAC or dMAC)



MPLSsMACdMACsMACdMAC


IEEE 802.1w/1s: Recommendation #1

• Configure primary and secondary root to provide root redundancy and load-balancing across multiple path

• One switch will be the primary root for one set of instances



MPLS

FF

BB

Primary STP Root

Secondary STP Root



IEEE 802.1w/1s: Recommendation #1 (Cont.)

• Configure primary and secondary root to provide root redundancy and load-balancing across multiple path

• One switch will be the primary root for one set of instances and the other switch will be the primary root for the other set of instances



MPLS

BB

FF

Secondary STP RootSecondary STP Root

Primary STP Root



• Same MST configuration in terms of VLANs-instance mapping, revision numbers, name on all the devices part of the same L2 domain:



MPLS

Name [Networkers-Config]Revision 1Instance Vlans mapped-------- -----------------------------------0 1-99,167-199,267-1499,1566-40941 100-1662 200-26615 1500-1565




• Make sure that the port cost of the preferred path is lower than the port cost of alternate ports, also in case of a single port EtherChannel configuration



MPLS10000Po1Cost

150003/1Cost



Scenario #1• Physical link in a 2 ports channel fails

• Port cost of the channel is re-calculated



MPLS20000Po1Cost

Link Failure

150003/1Cost





• Port cost of the channel is re-calculated

• Port 3/1 has a lower port cost spanning tree re-converges



MPLS20000Po1Cost

150003/1Cost

BBFF

FFBB




• Port cost of the channel is recalculated



MPLS20000Po1Cost

300003/1Cost

Link Failure





• Port cost of the channel is recalculated

• Port 3/1 has a higher port cost spanning tree does not reconverge



MPLS20000Po1Cost

300003/1Cost

FF

BB


Spanning Tree Loop Guard

• BPDUs sent by the root are not received by the access switch (unidirectional link)

Or

• CPU overloaded on the root switch BPDUs are not sent at the proper rate (BPDUs are skewed)



MPLS

RPRP

BPBP

Primary STP Root

Secondary STP Root

BPDUs

BPDUs




Without Spanning Tree Loop Guard:• The access does NOT receive BPDUs, its ports will become

designated transitioning into FWD when the previous root information is aged out



MPLS

RPRP

BPBP

Primary STP Root

Secondary STP Root

BPDUs

BPDUs

DPDP

DPDP



Without Spanning Tree Loop Guard:• The access does NOT receive BPDUs, its ports will become

designated transitioning into FWD when the previous root information is aged out

STP Loop



MPLS

Primary STP Root

Secondary STP Root

FF

FF

STP Loop




With Spanning Tree Loop Guard:• The access does NOT receive BPDUs, its ports will transition into loop-

inconsistence (i.e. BLK) when the previous root information is aged out

To Be Enabled on the Non-Root Switch



MPLS

STP RootLoop-GuardInconsistent


Route Processor Redundancy +(RPR+)/ Fast Software Upgrade (FSU)

• Provides supervisor redundancy

• Line cards are not reloaded nor re-initialized during the supervisor switchover

• Dynamic protocols are re-started after theswitchover

• The Cisco IOS® image is downloaded on the standby supervisor



Stateful Switchover (SSO)/Non-Stop Forwarding (NSF)

• SSOActive and standby supervisor have the configuration synchronized Protocol processes are created on both active and standby supervisorsWhen the primary supervisor fails, the redundant supervisor become active maintaining the switching information previously learnt and without restarting the L2 protocols (CDP, DTP, STP, 802.1Q, Port Security, … )

• NSFRouting protocols such as EIGRP/OSPF/BGP and IS-IS are not restarted nor re-initialized after a primary supervisor failure


Network Resiliency Model: Summary

VLAN 5VLAN 20 VLAN 30VLAN 40

802.1w/1s802.1w/1sRPR+/FSUSSO/NSF

UDLDSpanning Tree PortFast

UDLDLOOP GuardPAgP/802.3ad

Customer—SPBoundary

802.1QTrunk UNI

802.1QTrunk

CPE

Access

CoreSP

IP/MPLSNetwork







Agenda



Why Is Queuing Needed in the Campus?Oversubscription: Uplink Congestion

SiSi SiSi

SiSiSiSi

Access

Distribution

CoreInstantaneousInstantaneous

InterfaceInterfaceCongestionCongestion

Typical 20:1Typical 20:1Data OverData Over--

subscriptionsubscription

Typical 4:1Typical 4:1Data OverData Over--

subscriptionsubscription

= Data= Voice



77 66 55 44 33 22 11 00

Classification Tools:IPv4 IP Precedence and DiffServ Code Points

• IPv4: Three most significant bits of ToS byte are called IP Precedence (IPP)—other bits unused

• DiffServ: Six most significant bits of ToS byte are called DiffServ Code Point (DSCP)—remaining two bits used forflow control

• DSCP is backward-compatible with IP precedence

IDID OffsetOffset TTLTTL ProtoProto FCSFCS IP SAIP SA IP DAIP DA DataDataLenLenVersionVersionLengthLength

ToSToSByteByte

DiffServ Code Point (DSCP)DiffServ Code Point (DSCP) Flow Ctrl

IPv4 Packet

IP Precedence Unused Standard IPv4

DiffServ Extensions


QoS Design: Provisioning for Voice, Video and Data

• Latency ≤ 150 ms

• Jitter ≤ 30 ms

• Loss ≤ 1%

Smooth, Drop Sensitive Delay Sensitive, UDP Priority

Voice

One-WayRequirements for Voice and

Video

Bursty, Greedy, Drop Sensitive, Delay Sensitive, UDP Priority

Video

Smooth/Bursty, Benign/Greedy, Drop Insensitive, Delay Insensitive,

TCP Retransmits

Data

• 17-106 kbps guaranteed priority bandwidth per call

• 150 bps (+ layer 2 overhead) guaranteed bandwidth for voice-control traffic per call

• Minimum priority bandwidth guarantee required is:

Video-Stream + 20% e.g. a 384 kbps stream would require 460 kbps of priority bandwidth



QoS Design: Classification and MarkingMarking Recommendations to the Enterprises

Network ManagementNetwork Management

Call SignallingCall Signalling

Streaming Video

Transactional Data

Video ConferencingVideo ConferencingVoiceVoice

Application

Bulk Data

L3 Classification

1826 2426 24

3234344646

161610

AF21AF31 CS3AF31 CS3

CS4AF41AF41

EFEF

CS2CS2AF11

233

44455

22

IPP

1

233

44455

221

Scavenger 8CS11 1

Best Effort 000 0

Routing 48CS66 6

Mission-Critical Data 25-3 3

DSCPPHB CoS


QoS Design: Classification and MarkingCollapsing the Classes of Service

Network ManagementNetwork Management

Call SignalingCall Signaling

Streaming Video

Transactional Data

Video ConferencingVideo ConferencingVoiceVoice

11 ClassQoS Baseline Model

Bulk Data

Best Effort

IP Routing

Mission-Critical Data

Scavenger

Time

Critical Data

7 Class Model

Video


Best Effort

VoiceVoice

Bulk Data

Network Control

Critical Data

5 Class Model

Video


Best Effort

VoiceVoice




SP-Enterprise QoS Model Summary

• At the ingress of the SP network, the 11 enterprise classes of service get mapped into 4-5 SP classes

• The enterprise DSCP marking scheme is translated into a SP CoS marking scheme, which does not change the enterprise DSCP values

• Egress shaping on the enterprise CPEs required to increase the goodput

• Enterprise jitter, latency and drop requirements are guaranteed by the SP QoS model







Agenda



CPE Redundancy Considerations

• CPERouterSwitch

• Resiliency mechanismEtherChannelSpanning treeFlexlinkHot Standby Routing Protocol (HSRP)

• Attachment to service providerDual-attached Dual-homed



CPE Attachment Considerations

SP Network SP Network

SP Network

Dual-HomedDual-Attached

Customer Location

HSRP


Example #1: Router or Switch Dual Attached with EtherChannel

• ERS service flowing between RTR #1 and RTR #2

• RTR #1 uses 192.168.1.1/30 on Port Channel #1 • RTR #2 uses 192.168.1.2/30 on the Port Channel #1

ERS Service

IP/MPLSNetwork

RTR #1

Service Provider Network

U-PE #1 RTR #2U-PE #2

PortChannel #1

FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

PortChannel #1

FE 1

FE 2



ERS Service

IP/MPLSNetwork

RTR #1



PortChannel #1

FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

PortChannel #1

FE 1

FE 2

Example #1: Router or Switch Dual Attached with EtherChannel (Cont.)

1. On RTR #1, FE 1 fails in the port channel

11


ERS Service

IP/MPLSNetwork

RTR #1



PortChannel #1

FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

PortChannel #1

FE 1

FE 2 22

Example #1: Router or Switch Dual Attached with EtherChannel (Cont.)

1. On RTR #1, FE 1 fails in the port channel

2. On RTR #1, traffic converges onto FE 2Service outage is less than 200ms due to using EtherChannel



Example 2:Switch Dual Homed Using Spanning-Tree

• Ethernet Multipoint Service is configured between SW #1 and SW #2• The customer uses VLAN 100 for the service between SW #1 and SW #2• SW #1 uses 192.168.1.1/30 for VLAN 100 and allows VLAN 100 on FE #1

and FE #2• SW #2 uses 192.168.1.2/30 for VLAN 100 and allows VLAN 100 on FE #1

and FE #2

IP/MPLSNetwork

SW #1


SW #2

FE 1

FE 1

FE 1

FE 2

Spanning-Tree*

* BPDU’s Are Only Seen by CPE, not the Service Provider

U-PE #1 U-PE #2N-PE #1 N-PE #2

FE 1

FE 2

EMS Service

FF

BB BB

FF


IP/MPLSNetwork

SW #1


SW #2

FE 1

FE 1

FE 1

FE 2

FF

Spanning-Tree*

* BPDU’s Are Only Seen by CPE, Not Service Provider (Because of theEMS Configuration)


FE 1

FE 2

EMS Service


1. Failure occurs

11

BB

FF

BB



IP/MPLSNetwork

SW #1


SW #2

FE 1

FE 1

FE 1

FE 2

BB

Spanning-Tree*

* BPDU’s Are Only Seen by CPE, Not Service Provider (Because of theEMS Configuration)


FE 1

FE 2

EMS Service


1. Failure occurs2. 2nd link becomes active and traffic reverts

to the alternate path

22

FF

FF

FF


Example #3: Router or Switch Dual Attached: FlexLink*

• ERS service flowing between RTR #1 and RTR #2• FlexLink is configured on U-PE #1 for FE 2 to backup FE 1 • FlexLink is configured on U-PE #2 for FE 2 to backup FE 1• RTR #1 uses 192.168.1.1/30 on Port Channel #1 • RTR #2 uses 192.168.1.2/30 on Port channel #1

* FlexLink—a Feature Used to Backup Another L2 Interface and Provide 100ms or Less Convergence

ERS Service

IP/MPLSNetwork

RTR #1



FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

FE 1

FE 2



Example #3: Router or Switch Dual Attached: FlexLink (Cont.)

1. On RTR #1, FE 1 fails in the Port Channel

ERS Service

IP/MPLSNetwork

RTR #1



FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

FE 1

FE 2

11


Example #3: Router or Switch Dual Attached: FlexLink (Cont.)

ERS Service

IP/MPLSNetwork

RTR #1



FE 1

FE 2

FE 1

FE 2

N-PE #1 N-PE #2

FE 1

FE 2 22

1. On RTR #1, FE 1 fails in the Port Channel

2. On RTR #1, traffic converges onto FE 2Failure recovery is 100ms or less due to FlexLink



Example 4:Router and HSRP

• Dual exit paths are needed out of the enterprise network• Two routers are used with diverse paths as exit points• RTR #2, #3, #4, #5 track interfaces connecting to the service provider for failure• HSRP is used between RTR #2 & RTR #3 for an exit path for RTR #1• HSRP is used between RTR #4 & RTR #5 for an exit path for RTR #6

FE 1

IP/MPLSNetwork


U-PE #1 U-PE #2

FE 1

N-PE #1 N-PE #2

EnterpriseNetwork

EnterpriseNetwork

HSRPHSRP HSRPHSRP

RTR #1

RTR #3

RTR #2

RTR #6

RTR #5

RTR #4

ERS Service


FE 1

IP/MPLSNetwork


U-PE #1 U-PE #2

FE 1

N-PE #1 N-PE #2

EnterpriseNetwork

EnterpriseNetwork

HSRPHSRP HSRPHSRP

RTR #1

RTR #3

RTR #2

RTR #6

RTR #5

RTR #4

ERS Service


1. Failure occurs

11



22

FE 1

IP/MPLSNetwork


U-PE #1 U-PE #2

FE 1

N-PE #1 N-PE #2

EnterpriseNetwork

EnterpriseNetwork

HSRPHSRP HSRPHSRP

RTR #1

RTR #3

RTR #2

RTR #6

RTR #5

RTR #4

1. Failure occurs

2. 2nd path is available; traffic uses 2nd path to reach remote destinations

ERS Service


22


CPE Redundancy Summary

• There are different attachment options to consider when connecting to the service provider network, such as:

Dual—attached

Dual—homing

• Depending on the desired connectivity option to the service provider, various resiliency options are provided, such as:

EtherChannel

Spanning Tree

FlexLink

HSRP

• A new feature “FlexLink” may provide the fastest switchover time with 100ms or less


Q AND A



Associated Sessions

• ACC-1000/ACC-1N0-1: Introduction to Layer 2 Transport and Tunneling Technologies (L2VPNs)

• ACC-2000: Layer 2 Transport and Tunneling (L2VPN) Application and Deployment

• ACC-2001: Design Considerations for Sizing and Scaling Metro Layer 2 Services

• OPT-2045: Extending Metro Ethernet Across SONET/SDH Transport Infrastructure

• ACC-3001: Troubleshooting Layer 2 Transport and Tunneling (L2VPN) Technologies



Recommended Reading

• Metro Ethernet [1-58705-096-X]

• Cisco Self-Study: Building Cisco Metro Optical Networks (METRO) [1-58705-070-6]

• DWDM Network Designs and Engineering Solutions [1-58705-074-9]

• Optical Network Design and Implementation [1-58705-105-2]


Network Availability Recommendations

EnableSpanning Tree Loop Guard

EnableSpanning Tree Root Guard

EnableEnablePAgP/802.3ad

EnableEnable*EnableEtherChannel

Enable on UNISpanning Tree Port Fast

Enable

ML-Series Cisco 7600Catalyst 3550Feature

PVRSTP

EnableEnable802.1w/.1s

*Other Considerations Should Be Taken into Account When Enabling EtherChannel between the PE-AGG and N-PE within the ML Topology; Review the ML-DiG for More Information



Network Availability Recommendations (Cont.)

EnableFSU

EnableRPR+

Enable

ML-Series Cisco 7600Catalyst 3550Feature

Resilient Packet Ring

EnableEnableUDLD


Complete Your Online Session Evaluation!

WHAT: Complete an online session evaluation and your name will be entered into a daily drawing

WHY: Win fabulous prizes! Give us your feedback!

WHERE: Go to the Internet stations located throughout the Convention Center

HOW: Winners will be posted on the onsiteNetworkers Website; four winners per day



Documents

METROPOLITAN ETHERNET DESIGN FUNDAMENTALS · 2017-10-10 · Metro Ethernet Connectivity Ethernet Wire Service • Features Point-to-point connectivity Carrier network transparency