Core Network Evolution Stefan Kollar

8/3/2019 Core Network Evolution Stefan Kollar

http://slidepdf.com/reader/full/core-network-evolution-stefan-kollar 1/54

1© 2010 Cisco and/or its affiliates. All rights reserved.

Core Network Evolution and

IPoDWDM: 40G 100G andBeyond

Stefan Kollar

Consulting Systems Engineer CCIE #10668

[email protected]



© 2010 Cisco and/or its affiliates. All rights reserved. 2

OTN – Technical Foundation

OTN – Properties and Impact on IP Layer

MPLS-TP – Technical Foundation

MPLS-TP Forwarding and OAM

MPLS-TP Deployments

40G/100G Design Considerations

40G/100G Deployment Considerations in 10G Optical Networks

100G - and beyond




Global IP Traffic GrowthIP traffic will increase fivefold from 2008 – 2013

Source: Cisco Visual Networking Index—Forecast, 2008-2013

Other ServicesSONET/SDH

Data Centre

Private Line

IP packets willDominate

IP RoutedServices

L2 Packet

Services

87% Consumer

Traffic Video



© 2010 Cisco and/or its affiliates. All rights reserved. Cisco PublicPresentation_ID 4





• OTN defined a fixed “hierarchy”of payloads: from OTU1 (2.5G) toOTU3 (40G). Now ODU0 (1G)and OTU4 (100G) are beingadded

• OTN started as a pure wrapper around WDM client signals toimprove reach and manageability

• Recently it has developed into acomplex multiplexing structurethat enables a service layer aswell as TDM bandwidth mgmt

Frame Payload (OPU)

ODU0 (coming) 1,238,954 kbit/s

OTU1 2,488,320 kbit/s

OTU2 9,995,276 kbit/s

OTU3 40,150,519 kbit/s

OTU4 (coming) 104,355,975 kbit/s







The OTN Multiplexing Hierarchy• OTN Hierarchy

ODU0: ~1.22Gbps : GEODU1: ~ 2.7Gbps : STM-16

ODU2 : ~10.7Gbps : STM-64ODU3 : ~43Gbps : STM-256ODU4: ~112Gbps : 100GE

ODU2e:~11.1Gbps : 10GE LAN PHYODU3e:~40Gbps : 10GE onto 40G

• ODU-FLEX

Similar concept to VCATs in SONET/SDH but uses OTN containers

Increments are variable (10Gbps, 1Gbps) but only one increment size per “link”

1.25Gbps increments is the most commonly discussed ODU-FLEX service.

GMP stands for the Generic Mapping Procedure that is currently under definition in Q11

Client

O D U - F L E X

GFP + idles

GFP

GFPGFP

TS

TS

TS

TS

G M P

Client

OD U-F L E X

GFP + idles

GFP

GFPGFP

TS

TS

TS

TS

GMP

O D U - F L E

X

TS

TS

TS

TS

G M P

TS

TS

TS

TS

G M P

HO ODUHO ODU HO ODU HO ODU

Classifier

ODU-FLEX Switch ODU-FLEX Switch




Supported Ethernet Service Types per G.8011.x

OADM / Mesh DWDM node

G.709 formatted signal, OTUm

Multi-degree ROADMCross Connecting Lambdas

Dropping full lambdas

OTN HO Electrical CrossConnectGrooming and aggregation

Sub-lambda HO interfaces(SONET, OTN, Ethernet, ESCON)

Ethernet Switching Fabric

Ethernet interfaces (e.g. OC-3/STM-1)

Supported Ethernet Service Types per G.8011.x

Point-to-PointEthernet Private Line (EPL) Type 1

Ethernet Private Line (EPL) Type 2

Ethernet Private Line (EPL) Type 2 – timingtransparent

Ethernet Virtual Private Line (EVPL) Type 1, 2, 3

Point-to-MultipointEthernet Private Tree (EPT)

Ethernet Private LAN (EPLAN)

Ethernet Virtual Private Tree (EVPT), Type 1, 2, 3

Ethernet Virtual Private LAN (EVPLAN), Type 1, 2, 3




OTN – Properties andImpact on IP Layer




Encapsulation and OAM&Pover optical spans(G.709 transponders)

Point to Point Multiplexing(G.709 muxponders)

OTN Grooming and Switching(OEO Cross Connects)OTNOEO

OTNOEO

OTNOEOOTNOEO

OTNOEO

OTNOEO

OTNOEO

OTNOEO

OTNOOO

OTNOOO

OTNOOO




Individual IFs

• Works• Cost

Channelized

POS/OTN• No chOTN• Cost

Ethernet w/

VLANs• Shaping• Metrics




Primary Optical Path

Back Optical Path

Characteristics

Optical level over-provisioning : 100%IP level over-provisioning : 0%Overall over-provisioning : 100%Complexity : High – need hold timersComplete IP level protection: No

Span Failure

IP working : Yes

Primary Optical Path

Back Optical Path

Patch Cord, transponder

or router card failureIP working : No

Primary Path failure




The Contrasting Topologies

L3 Routing Nodes

OTN ODU-FLEX switch

Internet

Internet

Business DataCentre

Business DataCentre

Internet

Internet

Business DataCentre

Business DataCentre

Video andInternet cache

Video andInternet cache

IP/TVHead End

IP/TVHead End

Hierarchical SolutionPhysical and Logical Topology the same

Bypass Solution

EndUser

EndUser




What’s the Impact at the Network Level ? Links and Over Provisioning

1Gbps Initial Demand1Gbps increments

1Gbps Initial Demand10Gbps increments

Network Wide Implications – 100 Node network

Worse over-provisioning : Links * b/w incrementNetwork wide link upgrades = Links * Link upgrades

Worse case over-provisioning =200 * 10Gbps = 2000Gbps

Number network wide physical link upgrades = 200 * 0 = 0

Network Wide Implications – 100 Node network

Worse over-provisioning : Links * b/w incrementNetwork wide link upgrades = Links * Link upgrades

Worse case over-provisioning =5000 * 1Gbps = 5000Gbps

Number network wide logical link upgrades = 5000 * 7 = 35000

Note : This does not take into account physical link upgrades

Link Level Over provisioningLink Level Over Provisioning

Network Efficiency and provisioning needs to account for total number of links, traffic growth, provisioning efficiency and upgrade frequency.





Client to OEO

with SR optics

Client withSR Optics

Client withIntegratedOptics

Photonics Bypass

Transponder Short Reach

PhotonicOptical Approach

Major IP demands are very large – often the driving force behind DWDM upgradesFull interfaces directly to the photonic layer of NG/ROADM Intelligent DWDM system

Cut-through / bypass at the photonics layer

Eliminates expensive high bandwidth opti-electrical components

More pronounced with 40Gbps and 100Gbps

Transport OTNOEO Approach

Optical and in particular IPoDWDM is the most cost effective highb/w inter router connection




MPLS-TP – Technical Foundation




• Evolution of SONET/SDH transport networks to packet switchingdriven by

Growth in packet-based services (L2/L3 VPN, IPTV, VoIP, etc)

Desire for bandwidth/QoS flexibility

• New packet transport networks need to retain same operationalmodel

• An MPLS transport profile being defined at IETF (in collaborationwith ITU-T)




18

1: [RFC 5317]: Joint Working Team (JWT) Report on MPLS Architectural Considerations for a Transport Profile,Feb. 2009.

Definition of MPLS “Transport Profile” (MPLS-TP) protocols,based on ITU-T requirements

Note: IETF decided to support single MPLS-TP OAM solution.IETF Chair stated at IETF 79 (11/2010) and IETF 80 (3/2011)

Derive packet transport requirements

Integration of IETF MPLS-TP definition into transport networkrecommendations

IETF and ITU-T agreed to work together and bring transport requirements into the IETFand extend IETF MPLS forwarding, OAM, survivability, network management, andcontrol plane protocols to meet those requirements through the IETF StandardsProcess.[RFC5317]1

ITU-T withdrawal of T-MPLS draft G.8114 in Jan. 2008.




• MPLS-TP is proper subset of MPLS proper

• So far, no „MPLS-TP only“ functionality standardized

MPLS Solution Space

• ECMP

• MP2Pt

• LDP, IP

MPLS-TP Solution Space

• PHP default disabled

MPLS-TP Only Solution Space




• Connection-oriented packet switching model

• No modifications to MPLS data plane

• Interoperates/interworks with existing MPLS and pseudowire control and dataplanes

• No LSP merging or PHP

• LSPs may be point to point (unidirectional, co-routed bidirectional or associated

bidirectional)

• LSPs may be point to multipoint (unidirectional)

• Networks created and maintained using static provisioning or a dynamic controlplane: LDP for PWs and RSVP-TE (GMPLS) for LSPs

• In-band OAM (fate sharing)

• Protection options: 1:1, 1+1 and 1:N

• Network operation similar to existing transport networks

See RFC 5654




MPLS-TP Forwarding Plane




• Static• Bidirectional

• Co-routed (same forward and reverse paths)

• In-band Generic Associated Channel (G-ACh)

• Ultimate hop popping (no explicit/implicit null)

• No ECMP

• Contained within a tunnel

MPLS-TPLSP

G-AChMPLS-TPTunnel




MPLS-TP

• No IP routing required in control andforwarding planes

• Node will still source/terminate IP packets(e.g. SNMP, NTP)

• Link numbers required on each MPLS-TPinterface

• Two interface configuration modelsIP-enabled (uses ARP)

IP-less (no ARP)

• IP-enabled requires interface configurationfor

Local IPv4 address

Remote IPv4 (next-hop) address

• IP-less requires configuration of Destination MAC address

• Same OAM messages for IP-enabled andIP-less interface configurations




MPLS-TPTunnel

ProtectLSP

G-AChG-ACh

WorkingLSP

• MPLS-TP tunnels abstracted as Tunnel-tp

interface

• Tunnel holds a working LSP and a protectedLSP

Working

Protect (optional)

• Tunnel may be configured with a bandwidthallocation

• Tunnel operationally UP if at least one LSPoperationally UP (and not locked out)

• LSP operationally UP if OAM (Continuity Check)session operationally UP

• LSP requires static configuration of LSP labelimposition (output label and output link)

• LSP requires static configuration of LSP labeldisposition (input label)

• LSP must be co-routed (no embedded check)




• Same OAM functions for LSPs, pseudowires and sections

• In-band OAM packets (fate sharing)

• OAM functions can operate on an MPLS-TP network without acontrol plane

• Extensible framework with current standardization focus onfault and performance management

• Independent of underlying technology

• Independent of PW emulated service




• OAM capabilities extended using a generic associated channel (G-ACh) based onRFC 5085 (VCCV)

• A G-ACh Label (GAL) acts as exception mechanism to identify maintenancepackets

• GAL not required for pseudowires (first nibble as exception mechanism)

• G-ACh used to implement FCAPS (OAM, automatic protection switching (APS),signaling communication channel, management communication channel, etc)

ACH

OAMPayload

GAL

Label

Associated Channel Header

Generic Associated Channel Label (GAL)

PW AssociatedChannel Header

(ACH)

ACH

OAMPayload

Label

PW Label0 0 0 1 Version

RFC 5586

RFC 5085

IETF

LSP

G-ACh

PWG-ACh

Reserved Channel Type




• Relies on a disjoint working and a disjointprotect path between two nodes

• Provides 1:1 protection (only one activeLSP) in revertive mode

• Functionally similar to path protection inIP/MPLS

• Protection switching can be triggered by

Detected defect condition (AIS/LDI, LKR)

Administrative action (lockout)

Far end request (lockout)

Server layer defect indication (LOS)

Revertive timer (wait-to-restore)

PE1 PE2

P2

P1

Working LSP(Up, Active)

Protect LSP(Up, Standby)

PE1 PE2

P2

P1

Working LSP(Down, Standby)

Protect LSP(Up, Active)

Working LSP(Up, Active)

Protect LSP(Up, Standby)

Working LSP(Down, Standby)

Protect LSP(Up, Active)

Before Failure

During Failure




Function Description Tool

Continuity Check Checks ability to receive traffic BFD

ConnectivityVerification

Verifies that a packet reaches expected nodeBFD (proactive)

LSP Ping (on-demand)

Diagnostic Tests General diagnostic tests (e.g. looping traffic) New

Route Tracing Discovery of intermediate and end points LSP Ping

Lock Instruct Instruct remote MEPs to lock path (only test/OAM traffic allowed) New

Lock Reporting Report a server-layer lock to a client-layer MEP New

Alarm Reporting Report a server-layer fault to a client-layer MEP New

Remote DefectIndication

Report fault to remote MEP BFD

Client FailureIndication Client failure notification between MEPs PW Status

Packet LossMeasurement

Ratio of packets not received to packets sent New

Packet DelayMeasurement

One-way / two-way delay (first bit sent to last bit received) New

IETF




MPLS-TP deployements




Flexible Edge and Services Architecture

Multiservice Core!Aggregation! Edge! Core!Static MPLS-TP Access

IP/MPLS Access

(L2 and CE only)

Ethernet Access

IP/MPLS + Dynamic MPLS-TP IP/MPLSIP/MPLS

Pseudo-Wire Switching

MPLS-TP IP/MPLSL3 IP Edge and Service Placement

Flexible IP Edge and Service Placement – Agg, Edge or Core

IP/MPLS in Core / Edge / Aggregation

Use MPLS-TP toolbox to enhance dynamic IP/MPLS domain

Variety of Access options – static MPLS-TP, Ethernet, IP/MPLS

Common protocols and control plane aggregation to aggregation

Circuit Emulation + Ethernet




Centralised Edge and Transport Architecture

Multiservice Core!Aggregation! Edge! Core!MPLS-TP Access

IP/MPLS Access

(L2 and CE only)

Ethernet Access

IP/MPLS (L2 and CE only) IP/MPLS

Centralised IP Edge and Service placement : Edge, Core

IP/MPLS in Core / Edge / Aggregation

Variety of Access options – static MPLS-TP, Ethernet, IP/MPLS

Common protocols and control plane aggregation to aggregation

Ethernet Access / IP/MPLS L2 Aggregation : Widely deployed model

IP/MPLS

L3 IP + Services Placement





Centralised Edge and Transport Architecture

Aggregation! Edge! Core!

Ethernet Access

Static MPLS-TP IP/MPLSIP/MPLS

Static MPLS-TP Access

L3 IP + Services Placement

Centralised IP Edge and Service placement - Edge, Core

IP/MPLS in Core / Edge

Static MPLS-TP in Aggregation

Variety of Access options : Ethernet / Static MPLS-TP

Common forwarding protocols aggregation to aggregation

End to end transport operations using pseudo-wire switching





40G/100G Design Considerations




Solutions for Implementing 100G DWDM

•

All lambdas upgraded to 100Gbps• Sub-100G services provided by OTN OEO

Advantages

All lambdas on a fibre are 100G

Disadvantages

100TXP investment upfront

Need an additional OTN OEO

All 10G TXPs are obsolete

10G and 100G DWDM

Coexistence

10G and 100G lambdas co-exist on same fibre

Packet uses 100G, everything else 10G

Advantages

Only high demand clients upgraded to 100G

Protects existing 10G DWDM investment

Lowest cost per bit (100G TXPs>10 x10G TXPs)

Disadvantages

Need a guard band between 10G and 100G frequencies

Not appealing in ULH environments

100G lambda

10G lambda

100G lambdas

OTN OTN

10G SR

100G SR

OTN Multiplexing




• 40 Gig and above rates must meet minimum requirements:Target 10 Gig distances—1500 Km reach

Not simply a Greenfield technology, but plug and play over existing10Gig networks

Must be as open as possible, operate over third party DWDM networks

Must operate over both 100GHz as well as 50GHz spacingsPower and footprint must be reasonable, can not redesign Router/transport shelf due to blade

• To achieve must leverage/control:

1. Optical Impairments

2. Modulations schemes




• AttenuationLoss of signal strength

Limits transmission distance

• Chromatic Dispersion (CD)

Distortion of pulsesLimits transmission distance

Proportional to bit rate

• Optical Signal to NoiseRatio (OSNR)

Effect of noise in transmission

Caused by amplifier

Limits number of amplifier




• Polarization Mode Dispersion(PMD)

Caused by nonlinearity of fiber geometry

Effective for higher bitrates (10G)

• Four Wave Mixing (FWM)

Effects in multichannel systems

Effects for higher bit rates

• Self/Cross Phase Modulation(SPM, XPM)

Effected by high channel power

Effected by neighbor channels

Spreaded Pulse asIt Leaves the Fiber




• Cisco expects 100G Deployments to:

Target 10G distances—1500 - 2000 Km reach

Plug-and-play over existing 10Gig networks

Must be spectrally efficient- 50GHz Grid

Power / density / cost / performance trade off

• As Bit Rate increases the above becomes more challenging

Simplify deployment of 100Gig into 10Gig Systems




Transmitter Decrease Speed – reduce $$

Increase Modulation - Increase spectral efficiency

Increase Optical efficiency – Increase spectral efficiency

Receiver

Move from Direct Detection to Coherent detection

Compensate for Optical impairments in Electrical Domain(DSP) – reduce $$

Forward Error Correction (FEC)

Move to Higher coding gain FECs – Increase reach

100Gig was our first challenge – overcame with PM-QPSK

and new FECs




• Need to go slower

Optical impairments are directly related to signaling rates

• Need to increase modulation efficiency

Signaling speed decreases & Information Rate increases

NRZ to ODB to (D)PSK to (D)QPSK

• Need to increase optical efficiencySplit signal over two polarizations (PM – Mod Scheme)

1 bit/symbol

NRZ

0 1

1 bit/symbol

PSK

1-1

2 bits/symbol

QPSK

00

1011

01




• RX Laser behaves as Local Oscillator to provide a Polarizationreference

• 90° Hybrid:

• Converts Phase modulation in Amplitude modulation

• Signal Processor :

• Recovers Polarization

• Compensates CD and PMD electronically




• Utilizing Cisco’s advanced Optics and FEC

• Distances up to 2000Km and beyond

• Operates over existing Infrastructure at both 50 / 100GHz

100G PM-QPSK – At and / or exceeds 10Gig System Performance

PM-QPSK allows 100G operation over existing and greenfield

networks at 10Gig distances




40G/100G DeploymentConsiderations in 10G Optical

Networks




3rd

PARTY DWDM SYSTEM MUST SUPPORT ALIEN WAVELENGTHS!

Alien/foreign wavelength is any 3rd party ITUwavelength operating over an existing DWDMinfrastructure.

G698.2 – Standard for “Alien/Foreign waves”defines:properties for signal sources and sinksproperties for DWDM links for “blacklinks” (i.e. alien wavelengths)




Design Considerations

• 40G receiver differs from 10GNoise and Impairment Limits

40G IPoDWDMTransponder

(DPSK+)10G Transponder

LaunchPowers

0 dBm 0 dBm

Rx Windows 5 to –18 dBm 0 to –23 dBm

OSNR (.1nm) ~ 14.5 dB ~ 15 dB

CD +/- 750ps/nm +/- 2000ps/nm

PMD 2.5ps 10ps




• One of the biggest challengers in Poland (~350k broadband users)

• Implemented XR12000 core 1.5 year ago (one of the firstXR12000 production networks in the world)

• BB explosion has made their traffic grow quicker than expected =>need to upgrade main nodes

• Customer A expected big CapEx/OpEx savings




• IPoDWDM

CapEx/OpEx reductionEliminates Transponder Shelf

4:1 Capacity Savings

• Conducted successful tests on 40G IPoDWDM

on Warsaw -> Poznań link (614 km)

link designed for 10G optical




100G

Where are we today?




• IEEE 802.3ba: 40Gb/s and 100Gb/s Ethernet Task Force

40G and 100G Ethernet

Physical interfaces for Backplane, Copper, Fiber PMDs

• IEEE 802.3bg: 40Gb/s SMF Ethernet Task Force

40G Serial PMD optimized for carrier applications

• ITU Study Group 15: Optical and Transport Networks

OTU4 frame format

Single mapping for 40GE/100GE into OTU3/OTU4

OTL protocol enabling OTU3/4 over multi-lane (low cost) optics

• OIF: 100G Long-distance DWDM Transmission

Industry consolidation around a single 100G DWDM solution

Ratified

Ratified

Ratified

Ratified




• Customer demands are driving the need for 100Gig and beyond

Video – HD / 3D

Video Conferencing – HD / 3D

Gaming – HD / 3D

• Packet will dominate

28% of population connected

14% of population broadband

• 100Gig is deploying NOW

Content Providers

Tier One SPs and MSOs

• Mass deployments 2nd Half 2012 early 2013

Question is not “When 100Gig?” but rather “What is after 100Gig?”




• Higher data rates200Gig, 400Gig, 1T?

• Need to investigate other modulation techniquesPM-16QAM, PM-64QAM, …. or CO-OFDM ?

• Need deeper look at FECAdvanced FEC

What other algorithms are there

• Need of intelligent DWDM layer Flex spectrumControl planeAdvanced operations, troubleshooting andprotection mechanisms

• Must a channel really fit into 50GHz spacing?Or should it be gridless?




Information distributed over afew Sub-Carriers spaced asclosely as possible forming a1,000Gbps Super-Channel

Each Sub-Carrier transportinga lower Bit Rate, compatiblewith current ADCs and DSPs

-200 -150 -100 -50 0 50 100 150 200 f [GHz]

| S c h ( f ) | 2

10x 100Gbit/s Sub-Carriers

close-to-Baud-ratespaced

Super-Channel #1 Super-Channel #2 Super-Channel #3




• Sub-Carrier spacing: 1.2 times theBaud Rate

• Different approaches for 1,000Gb/s:CP-QPSK: 10 Sub-Carriers at 111 Gbit/seach

back-to-back sensitivity 12 dB

CP-8QAM: 8 Sub-Carriers at 138.75 Gbit/s eachback-to-back sensitivity 16.1 dB

CP-16QAQM: 5 Sub-Carriers at 222 Gbit/seach

back-to-back sensitivity 19.1 dB

• System Configuration:

Span of 90km each (ITU-T G.652)Span Insertion Loss: 25dB

CP-16QAM

CP-8QAM

CP-QPSK

1.041.08

1.121.20

1.441.81

1.041.08

1.121.2

1.441.8

1.04

1.081.12

1.21.44

1.8

with subcarrier spacings



Thank you.

Documents

Core Network Evolution Stefan Kollar