Optical Packet-Switched WDM Networks: a Cost and … · Summary Optical packet-switched networks Network architectures - Point-to-point WDM IP networks - IP networks with optical

Optical PacketOptical Packet--Switched Switched WDM Networks: WDM Networks:

a Cost and Energy Perspectivea Cost and Energy Perspective

Rodney S. Tucker

ARC Special Research Centre for Ultra-Broadband Information Networks (CUBIN)

University of Melbourne

[email protected]

Acknowledgements: Jayant Baliga, Kerry Hinton, Rob Ayre, Gangxiang Shen, Wayne Sorin

Australian Research Council, Cisco

SummarySummary

Optical packet-switched networks

Network architectures- Point-to-point WDM IP networks- IP networks with optical grooming- Optical burst switching- Optical packet switching

CAPEX and network architectures- Scaling

Energy consumption and network architectures- Core and access networks

Disclaimers: - Numbers given here are approximate - YMMV- OPEX not included

E-mail me for copies of these slides

Impact on network growth

Edge Routers Optical PacketOptical Packet--Switched WDM NetworkSwitched WDM Network

1 hop

Is this simple model realistic?

IP Packets Core Router

OXC

Patent(s) Pending and Copyright © LumetaCorporation 2007

The Reality: Map of the InternetThe Reality: Map of the Internet

Not Included:• Enterprise Networks• Content Distribution Networks• Data Centres

Number of Hops in the InternetNumber of Hops in the Internet

0 5 10 15 20 250

0.02

0.04

0.06

0.08

0.1

Number of Hops, k

[]

Pr

Hk

=

Source: P. Van Mieghem,“Performance Analysis of Computer Systems and Networks”, Cambridge (2006)

2006 Data

Optical Burst

Switching (OBS)

Evolution of Optical PacketEvolution of Optical Packet--Switched NetworksSwitched Networks

Capacity

Time

Point-to-Point WDM (P2P)

Optical Circuit

Switching (OCS)

Stage 1

CAPEX? Energy Consumption? Optical

Packet Switching

(OPS)

““ConventionalConventional”” WisdomWisdom

1. Point1. Point--toto--Point WDM NetworkPoint WDM Network

Router provides sub-wavelength groomingby statistical multiplexing

LightpathFiber O/E/O Converters

Core Router

O/E/O Converters

Access Routers

Edge Routers

Core Router

1 hop

OXC

2. Optical IP Network2. Optical IP Network

2. Optical IP Network2. Optical IP Network

Lightpath

OXC/ROADM

Core Router

Fiber

• Optical bypass • Sub-wavelength grooming

O/E/O Converters

O/E/O Converters

Access Routers

3. Waveband IP Network3. Waveband IP Network

MUX

• Waveband grooming• Optical bypass • Sub-wavelength grooming

Lightpath

FiberWaveband

OXC/ROADM

Access Routers

O/E/O ConvertersCore

Router

Consider three network architectures:

Cost and Scalability of Optical NetworksCost and Scalability of Optical Networks

Compare CAPEX

Parthiban et al., 2003

Number of users: 10 million

P2P WDM Optical IP Waveband IP

Input ParametersInput Parameters

Capacities:Router: 5 – 90 Tb/sOXC: 1000 portsLightpath: 40 Gb/sFiber: 250 lightpaths

250 λ

1 λ = 40 Gb/s

1000 1000

Average Access rate per user: 1 - 100 Mb/s10 million users

Component costs [Ferreira 02, Sengupta 03]OXC, router – chassis, portFiber, amplifiers, lightpath terminations

IgnoreAccess network, search engines, data centers, etc.Optical impairment cost – e.g. regeneratorsProtection & restorationMultiple domainsCost reductions as technology maturesOPEX

Results Results –– Network CostNetwork Cost

1

102

104

1010 100

Average Access Rate (Mb/s)

Cos

t per

Use

r ($)

Parthiban et al., 2003

P2P WDM

Optical IP

Waveband IP

Today’s Internet (~ 100 kb/s) (Access rate per user in a busy period)

Slope = 1

Cumulative Internet Capex (North America)

0

10

20

30

40

50

60

70

80

90

2000 2005 2010

TotalCore and

Metro

Access

Optical transport

Year

Cum

ulat

ive

Cap

ex $

Bill

ions

Based on data from Nemertes Research, 2007

~$400 / user

Sanity CheckSanity Check

150 million users

Nemertes, November 19, 2007: “User demand could outpace network capacity by 2010$137 billion global infrastructure investment needed”

• Optical IP network – Can save costs in today’s network– Optical bypass reduces router ports

• Waveband IP network– Eliminates the bottleneck in number of ports and lightpaths– Least costly for high access rates

• There is no such thing as a free lunch– If you want more bandwidth, you will have to pay for it

ObservationsObservations

Energy Consumption of the NetworkEnergy Consumption of the Network

Power In

• Greenhouse Impact

• Managing “Hot Spots”- Getting the energy in- Getting the heat out

• Energy-limited capacity bottlenecks

Why worry about energy consumption?

• OPEX

Hot spot

Metro

Core

Edge Edge

Curb Curb Curb Curb

Core Core

Access

Core

ONU ~ 5-10W

OLT - 100W

CRS-1 ~ 10 kW / rack

12816 Edge ~ 4 kW

0.1 - 1000 Mb/s to the user

Fibre Amps

WDM

Passive Optical Network

Packet over

Sonet

Energy Model of Simple IP NetworkEnergy Model of Simple IP Network

Baliga et al., 2007

1000

Pow

er (W

/use

r)

% o

f Ele

ctric

ity S

uppl

y

Baliga et al., 2007

0

20

0.5

25

5


15

4 862

1.0

10

20 hops

Power Consumption of IP NetworkPower Consumption of IP Network

Total

Core

Access +Metro

SDH/WDM Links

Today’s Internet (~ 100 kb/s)

2008 Technology

10000

Pow

er (W

/use

r)

% o

f Ele

ctric

ity S

uppl

y

Baliga et al., 2007

0

150

2.5

200

50


100

40 806020

5.0

7.5

10

Total

Core

Access +Metro SDH/WDM

20 hops

Power Consumption of IP NetworkPower Consumption of IP Network

2008 Technology

• Access network dominates energy consumption at low rates– Standby mode?

• Core network dominates at higher rates– Reduce hop count?

• What is the bottleneck in the core? Speed or energy?

– Optical packet switching

• Optical transport (WDM) consumes relatively little energy< 5% of energy> 25% of CAPEX

• Annual CAPEX / Annual energy OPEX > 2

Observations / QuestionsObservations / Questions

1

3

2

OpticsElectronics

1

F

1

K

1

Switch Fabrics

F

Buffers

F Demutiplexers F

Multiplexers

Fibers

Forwarding Engine

J

1

SwitchFabric

J

O/E Converters

Reduced bit rate

Electronic RouterElectronic Router

Router Throughput

Pow

er c

onsu

mpt

ion

(W)

Source: METI, 2006, Nordman, 2007

Power Consumption in RoutersPower Consumption in Routers

10

100

1,000

10,000

100,000

1,000,000

P = C2/3

where P is in Wattswhere C is in Mb/s

10 nJ/bit

100 nJ/bit

11 Pb/s

P ~ 10

1 Tb/s1 Gb/s1 Mb/s

?

Ener

gy p

er B

it(n

J)Energy per Bit in RoutersEnergy per Bit in Routers

1

10

100

1,000

10,000

100,000

10 nJ/bit

100 nJ/bit

0.1

1 nJ/bit

Router Throughput

1 Pb/s1 Tb/s1 Gb/s1 Mb/s

?

Year

Source: K. Brill, The Uptime Institute

Heat LoadHeat Load

1 nJ/bit

Air Cooling Limit

Switch Fabric

Buffer

I/O

Switch Control

Data Plane Control Plane

Line Card

Energy in HighEnergy in High--End Electronic RouterEnd Electronic Router

Routing Engine

Routing Tables

Power supply

inefficiency

Fans and blowers

35%11%32%7% 5% 10%Fraction of Total

Source: G. Epps, Cisco, 2007

Buffer

O/E

Buffer

O/E Forwarding Engine

Energy/bit 0.7 nJ 1.0 nJ0.5 nJ3.2 nJ 3.5 nJ1.1 nJ

Forwarding Engine

Total

Metro + AccessCore

0.1 1 10 100

Ener

gy p

er b

it (J

)

10-6

10-3

10-8

10-5

10-7

10-4

20 hops

~1 μJ/b

WDM Links


Network Network EnergyEnergy Consumption per BitConsumption per Bit

Energy in Electronic Integrated CircuitsEnergy in Electronic Integrated Circuits

CMOS GatesCwire

⎡ ⎤= + ⎣ ⎦∑ ∑ 21Energy2gate wireE C V

=Power Energy x Bit Rate

CMOS IC

Diversion: Diversion: CapacitanceCapacitance of the Core Networkof the Core Network

30 F (North America)NetworkC ≈

300 F (Global)NetworkC ≈

Source

→ = × 8~ 150 million users 200 nF 1.5 10NetworkC x

50% efficiency

Energy per bit per user: 212E C V⎡ ⎤= ⎣ ⎦∑

= = → ≈∑1 μJ2, 200 nF2

V E C

V Destination

Capacitance per user

C1 C2

Energy Consumption in Access NetworksEnergy Consumption in Access NetworksNEC CM7710T

Splitter PON

FTTN

PtP

WiMAX

Cabinet

Edge Node

Cisco 12816

NEC CM7700S

ZyxelVES-1616F-34Cisco

4503

NEC VF200F6

NEC GM100

Axxcelera ExcelMax CPE

AxxceleraExcelMax BTS

Cabinet

Baliga et al., OThT6

Access N/W


Baliga et al., OThT6

Energy Consumption in Access NetworksEnergy Consumption in Access Networks

• Wireless access consumes more energy than optical access• PON FTTH is “greener” than FTTN• “Standby mode” shows significant potential (OThT6)

Ener

gy p

er b

it (J

)WiMAX

FTTN

PtP

PON10-7

10-5

10-8

10-6

1 10 100 1000

Stage 1

Time

Optical Burst Switching

(OBS)

Point-to-Point WDM

(P2P)

Stage 2Optical Packet

Switching (OPS)

Capacity


Optical Circuit

Switching (OCS)

CAPEX and Energy Barrier

All-optical sub-wavelength grooming

Burst Assembly Router

Core Router

Edge Router

Stage 2 Evolution Stage 2 Evolution –– OCS to OBSOCS to OBS

Access Routers

Edge Node

OXC

ElectronicsWDM Link

Optical Burst Switches

Optical Burst Switched NetworkOptical Burst Switched Network

t

Optical Burst Switch No Buffering

IP Packets

Burst Assembly

Router

IP Packets (duration < μs)

Data Bursts (duration < ms)

Headers

Offset t

C. Qiao and M. Yoo, JHSN, 1999

Optical Burst SwitchingOptical Burst Switching

Source Destination

D

AT

A

Offset

Burst length< ms

Header

Time

Distance

Switch 1 3 2 4 5

Nor

mal

ized

Cos

t

20

With wavelength conversion

10-6 10-5 10-4 10-3 10-2 10-1

5

0

10

Average Path Blocking Probability

15

Normalized Cost: OBS vs. IPNormalized Cost: OBS vs. IP

Waveband IP

OBS

Without wavelength conversion

1 Gbit/s Access Rate

Optical IP

Parthiban et al., OFC 2005

Why is OBS more Costly?Why is OBS more Costly?

Waveband

MUX

OBSDEMX

Waveband

Wavelengths Wavelengths

Bursts Bursts

• Requires increased number of lightpaths for a given blocking probability

• Switch technology requires fast reconfiguration time: More costly per port than “slow” OXC’s (MEMS etc.)

Burst

WavebandWBS

Waveband Route

λ

t

Pro’s:• Requires fewer OXC ports

Con’s:• OXC ports must be wideband• Requires waveband (i.e. multi-channel) wavelength conversion• Dispersion issues

Waveband

Solution: Waveband Burst Switching (WBS)Solution: Waveband Burst Switching (WBS)

Y. Huang et al., OFC 2004, Pathiban et al., OFC 2006

Average Path Blocking Probability

Nor

mal

ized

Cos

t

5

Optical IP

Waveband IP

10-2 10-1

WBS (WC) + Deflection Routing + Burst Segmentation

WBS with wavelength conversion

Waveband Burst Switching (WBS)Waveband Burst Switching (WBS)

3

1

7

Parthiban et al., OFC 2006

1 Gbit/s Access Rate

10-5 10-4 10-3 10-2 10-1

Stage 1

Stage 2

Time

Optical Burst

Switching (OBS)

Optical Packet

Switching (OPS)

Point-to-Point WDM

(P2P)

Optical Circuit

Switching (OCS)

Stage 3Capacity


• Will a viable optical buffering technology emerge?

• Can OPS reduce energy consumption?

Optical Packet Switches

With Buffering

Access Router

Optical PacketOptical Packet--Switched NetworkSwitched NetworkThe “Holy Grail” of Optical Networking

OXC

Demutiplexers

Optical Packet Switch (OPS)Optical Packet Switch (OPS)

1

F

1

K

Wavelength-Interchanging

Cross Connect + Header

Replacement

F

Output Buffers

1

Forwarding Engine

Input SynchronizersMutiplexers

Electronics Optics

12

Delay

Delay

Delay LineVariable

Delay Line

Cross Point

Recirculating Loop

Staggered Delay Line

Optical Buffer StructuresOptical Buffer Structures

Delay

Delay

Control

Cross Point

Cross Point

.

Optical Fiber BuffersOptical Fiber Buffers

Cisco CRS-1 with 1000 ports, 250 ms buffering per port

OPS with 1000 ports, 250 ms buffering per portoptical fiber delay lines

Total buffer capacity of 104 Gigabits

~ 103 RAM chips

Cost < US$ 50k

Buffer power dissipation < 1 kW

Total fibre length = 40 Gm

150 times distance from Earth to Moon!

Total fibre length = 1000 km~5 μs (~20 packets) buffering per port

Loss per port = 0.2 dB

Loss per port = 33,000 dB

Optical fiber delay lines

Planar waveguide or slow light delay lines

(0.1 dB/cm)

Reduced Buffer SizeReduced Buffer Size

Minimum feasible buffer size?

Loss Energy

Tucker, PS 2007

Holographic BuffersHolographic Buffers

Input image

Output image

Reference beam

Orlov et al., Proc IEEE, 2004Psaltis, CLEO 2002Ashley et al. IBM J. Res. Dev., May 2000

Read speeds up to 10 Gb/s

Holographic Medium: Photorefractive material (e.g LiNbO3, polymers)

Write speeds up to 1 Gb/s

Storage density

Retention time

Access Time Write Time

∞ > 500 μs1/λ3 ~ 50 μs

Show stopper

Comparison of Optical Buffer TechnologiesComparison of Optical Buffer Technologies

Challenge

Technology Fiber Planar WG, Slow Light

Optical Resonator Holographic CMOS

Access Time

Structure-dependent

Structure-dependent Small ~ 50 μs 200 ps

Retention Time > 500 μs < 5 μs 1-100 ns ∞ 64 ms

Capacity (Packets) > 2,000 < 20 << 1 ∞ ∞Energy/bit ~ 1 fJ ~ 1 pJ ~ 1 pJ ~ 1 pJ ~ 1 fJ

Physical Size

Very Large Medium Medium Small Very

Small

Chirp Sensitivity No Small Large Large No

Tucker, PS 2007

Optical Interconnects

ChipsAWG’s

Wavelength Converters

Packet Switch CrossPacket Switch Cross--Connect TechnologiesConnect Technologies

AWG-Based CMOS

SOA-Based

~10 pJ/bit ~5 pJ/bit

>100 pJ/bit

3-stage Clos

(2 pJ/bit)

(1 pJ/bit)

(1.5 pJ/bit)

(10 pJ/bit per gate)

Tucker, JLT 2005, OSN 2008

Projected Energy Consumption

Benes

Benes Array

• No viable optical buffering technology in sight

• Optical switch fabrics may become competitive with CMOS

• Not clear whether optical packet switching will solve the energybottleneck problem

Observations on Optical Packet SwitchingObservations on Optical Packet Switching

Summary

• Optical bypass reduces CAPEX and energy consumption- Promising future for optical cross-connects and ROADMs

• Energy bottleneck in routers is looming- More significant than the so-called “electronic speed bottleneck”

• Can optical packet switching overcome the energy bottleneck?- Optical buffering is currently a show-stopper

• Think “Energy per Bit”

Documents

Optical Packet-Switched WDM Networks: a Cost and … · Summary Optical packet-switched networks Network architectures - Point-to-point WDM IP networks - IP networks with optical