Optical Packet Switching : Myth, Fact, and Promise · July 4, 2016 OECC / PS2016 3 Osaka Univ. Packet switching : Underlying concept P. Baran,”The beginnings of packet switching:

1 Osaka Univ.July 4, 2016 OECC / PS2016

Ken-ichi Kitayama

[email protected] School for the Creation of New Photonics Industries,

formerly with Osaka University, Japan

Optical Packet Switching : Myth, Fact, and Promise

2 Osaka Univ.

Acknowledgements

Y.Huang, P.J.Argibay-Losada, Y.Yoshida (presently with NICT), and A.Maruta

I’d also like to thank my collaborators of ;R.Takahashi, NTT LaboratoriesNEC CorporationKyushu University

This work has been partly supported by the R&D program,“High-Performance Hybrid Optoelectronic Packet Router-Towards Green High-Capacity Data Center Networks-”(2011~2016), funded by NICT.

July 4, 2016 OECC / PS2016


Packet switching : Underlying concept

P. Baran,”The beginnings of packet switching: Some underlying concept,” reproduced in IEEE Commun. Magazine, July, 2002.


Circuit switching vs. packet switching

Call lossQueuing delay=0

Circuit switching

Arrival

POTS QoS guaranteed Stream data

Packet lossQueuing delay

Packet switching

Arrival

Buffer

Internet Best effort service:

Only connection guaranteedby TCP/IP

Burst data

Cost＞＞

H. Miyahara, Osaka Univ.-NTT Meeting (2001.11.05).

If not busy

Busy

5 Osaka Univ.

Outline

IntroductionNiche application of OPS to DC networks

Myths Immature architecture Lack in optical RAM buffer

Promise : Enabling technologies at hand Prototyping energy-efficient optoelectronic packet router OPS/OCS/VOCS DC network performs well against

mice & elephant Summary

July 4, 2016

OPS: Optical packet switchingOCS: Optical circuit switchingVOCS: Virtual OCS

OECC / PS2016

6 Osaka Univ.

Outline





July 4, 2016


OECC / PS2016

7 Osaka Univ.

Transforming DC from hardware-centric to software-centric

7July 4, 2016 OECC / PS2016

DC

traf

fic

TimeModified after A. Fuetsch, Plenary talk, OFC2016.* Cisco Global Cloud Index: Forecast and Methodology, 2012–2017.

Software-centricSDDC

Hardware-centric

Res

ourc

e ut

ilizat

ion

8 Osaka Univ.

Fat-tree network

….

Transforming DC fabric design

Core

Aggregation

ToR

Scale-up approach (10G⇒40G⇒100G) The path and bw are fixed end to end Traffic concentrates at the core switch Blocking the ports by STP* is a waste

Leaf & spine network

….

Spine

Leaf

ToR

Scale-out approach (10G⇒10GxＮ) Path & bw are changeable end to end Load-balancing is possible STP is replaced with IS-IS* or BGP*

* STP : Spanning tree protocol* IS-IS : Intermediate system-to-Intermediate system* BGP: Border gateway protocol

To be continued

July 4, 2016 OECC / PS2016


Leaf

Spine

ToR

W EW E

Transforming DC fabric design

N|S

N|S

Bimodal traffic distribution of short and long packet flows. East-to-west long-lived data flows, so-called elephant flows.

and south-to-north short-lived data flow, so-called mice flows. Single switching protocol cannot perform well for the co-existence.

Leaf & spine network

10 Osaka Univ.

Hybrid switching schemes

OECC / PS2016July 4, 2016

Category EPS OCS OPSTDM WDM SDM

Current DC ✔

Helios, C-Through

✔ ✔

LION ✔1

Lightness ✔ ✔ ✔ ✔

COSIGN3 ✔ ✔ ✔2 ✔

- ✔ ✔

1. AWGR-based wavelength routing.2. MCF switch which can switch individual cores.3. SDN controlled

K.Kitayama et al., J. Lightwave Tech., vol. 33, pp. 1063-1071, 2015.

11 Osaka Univ.After C. Kachris et al., IEEE Commun. Mag., Sept., 2013

Large-capacity electric router

Helios, C-ThroughElectrical packet sw Optical circuit sw+

Electrical packet sw

Adopting optical packet sw

LIONSOptical packet sw

A Large-scale high-speed optical sw

R&D efforts for photonics and optics in DC

July 4, 2016 OECC / PS2016

Lightness, COSIGNOptical circuit swOptical packet sw +

12 Osaka Univ.

Optical circuits break the network topology !


6-D torus network with 4,096 (46)-node.Note : 885 ExPs traverse 22% of 4,096 routers.

Typical operation range

0 40 80 120 160 200Load from servers [Gbps]

Pac

ket L

oss

Pro

babi

lity

1

10-1

10-2

10-3

10-4

10-5Pure OPS

885 ExP111 ExP11 ExP

30-50 %

3D-torus topology

13 Osaka Univ.

Power consumption of DCs

July 4, 2016 OECC / PS2016

2007 [Billion kWh] 2020 [Billion kWh]

Data centers 330 1,012Telecoms 293 951Cloud 623 1,963

~280NC power plants

~280NC power plants

Power & heat management

35%

14 Osaka Univ.

Power consumption of DCs

July 4, 2016 OECC / PS2016

~280NC power plants

~280NC power plants

Buffers

10%

IP look-up & forwarding engine

Control plane

I/O

Source: G. Epps, Cisco, 2007

5%

32%7%

11%

Switch fabric10%

Power & heat management

35%

15 Osaka Univ.

Outline





July 4, 2016


OECC / PS2016

16 Osaka Univ.

Conventional e-router architecture

8 xe-switching

planes

Segmented into 8 short fixed-length packets and switch at low-speed in parallel

Input buffering &Label processing

Line card Line card

Large buffer required to store packets for the re-assembly

Fill Interframegap (IFG)

Large buffer required to store all the input packets for segmentation

O/E E/O O/E E/O O/E E/O

Input packets

July 4, 2016 OECC / PS2016

Output buffering


Opticalswitch

Input packets

t

Header

Labelprocessor

OPS architecture To be continued

Huge bw of optical components can be fully exploited. Packet payload is transparently forwarded w/o

segmentation and w/o o-e-o conversion.


OPS architecture

Buffer

Opticalswitch

O/E E/O

No optical RAM won’t be available in the near future. e-RAM is available but it causes substantial delay. Small size fiber-delay-line (FDL) is practically

implementable.

19 Osaka Univ.

Outline





July 4, 2016


OECC / PS2016

20 Osaka Univ.

Small size FDL work well !!!

July 4, 2016 OECC / PS2016

w/o FDL

To be continued

P.J. Argibay-Losada et al., JOCN, Vol.7, No.7, 2015.

A single FDL increases the application throughput from 160Gb/s to 240Gb/s

Fixed-length packet @40~100Gbps in 44km, 12-node MAN

TCP Stop&wait (SAW)

21 Osaka Univ.

Small size FDL work well !!!

No appreciable improvement between 16~256 FDLs.

July 4, 2016 OECC / PS2016

P.J. Argibay-Losada et al., JOCN, Vol.7, No.7, 2015.

Fixed-length packet @40~100Gbps in 12-node MAN

w/o FDL

22 Osaka Univ.

Outline





July 4, 2016


OECC / PS2016

23 Osaka Univ.

Hybrid optoelectronic packet router (HOPR)

July 4, 2016 OECC / PS2016

8 x 8optical switch

Controller

Label processor

ToR

CMOSRAM

100-GbpsBM Rx

100-GbpsBM Tx

ToR ToR ToR

100Gbps optical packet

10GbE

25Gbpsx 4λ

Header Payload

CMOS RAM buffer

Targets : Power <100mW/Gbps

(=nJ/bit) Operation at > 100~400 Gbps Latency <100 ns

Targets : Power <100mW/Gbps

(=nJ/bit) Operation at > 100~400 Gbps Latency <100 ns

Fiber delay line buffer

To be continued

R.Takahashi et al., JOCN, Vol. 7, No.12, 2015.

24 Osaka Univ.

Enabling device technologies at hand

24

Fiber delay lineShared buffer

8x8Optical switch

ControllerLabel processors

CMOS 100GRx

100G Tx

1x8 EAM gate array 8x8 optical switch module

Broadcast & select optical switch

Burst-mode EDFA

Self-clocked High-speed, sensitive Op.

OCTA: Optically clocked transistor array

Label separating circuit

Self stabilization Compact low power module

Spin分極型

可飽和吸収体SOA

BPFBackward pulse

Packet envelope

ATT

SA

Input pulse from OCG

4

1Output pulse train

Optical clock pulse train generator

24

25G,2ch 1.6G, 32chDually-operating SPC

Serial-to-parallel converter

Burst-mode APD-TIA

High sensitivity Quick response from the 1st bit

Low-jitter latch clock generatorJuly 4, 2016 OECC / PS2016

1.6G,16ch 25G,1ch, Differential-trigger PSC NRZ output Tolerance to optical trigger

Parallel-to-serial converter

Tunable transmitter

New type（PRR）TLD Integrated with 25G EAM

（25-G Burst-mode APD-TIA） High, flat output power

25 Osaka Univ.

8x8 broadcast & select switch

25

In

2Out 1 3 4 5 6 7 8

1x8 MMI

EAM

Pas

sive

w

aveg

uide

250 mm38.5 mm

47.5

mm

Switching speed < 10 ns, Extinction ratio < -40 dB, Power consumption < 4 W

1xN SW

1xN SW

1xN SW

O-gateO-gateO-gateO-gate

SOA vs. EAM+EDFASOA EAM+EDFAPattern effectPoor SNRHigh powerHigh WDL/TDLCurrent controlVery compactExtinction, Speed

No pattern effectSmall NFLow powerLow WDL/TDLVoltage controlCompactExtinction, Speed

July 4, 2016 OECC / PS2016

26 Osaka Univ.

Current Previous in 2009

Data rate 100 Gbps 10 Gbps

Switch 8 x 8 (B&S) 8 x 8 (AWGR)

Throughput 1.28 Tbps 160 Gbps

Power consumption

110 W 86 [mW/Gbps]

360 W [2.25W/Gbps]

Latency 100 ns 380 ns

Constructed in 2009Next HOPR (3-D torus)

Fiber Delay Line

Shared buffer

8x8Optical Switch

Controller

ToR

CMOS100G

Rx100G

Tx

ToR ToR ToR

10 GbE…

Label Processors

70 WAggregation Switch

40 W

100 Gbps, 8x8Optical packet switch

5 W / 100G-port

Target specification of HOPR prototype

July 4, 2016 OECC / PS2016

27 Osaka Univ.

F. Buffers

10% A. Power & heat management

B. IP look-up & forwarding engine

D. Control plane

C. I/O

Energy consumption : EPS vs. HOPR

Source: G. Epps, Cisco, 2007

July 4, 2016 OECC / PS2016

5%

35%32%

7%

11%

E. Switch fabric10% A:16%

B:22%

C:44%

D:13%

E:3% F:2%

Source: R. Takahashi, NTT, 2016

Electronic router HOPR

28 Osaka Univ.

Outline





July 4, 2016


OECC / PS2016

29 Osaka Univ.

Best mixture of protocol : OPS and OCS


Best effort service Relatively small data Best effort service Relatively small data

QoS required Set of large/ stream data

Optical packet switching Optical circuit switching

High resource utilization due to statistical multiplexingPacket loss

High resource utilization due to statistical multiplexingPacket loss

Dedicated optical pathQoS guaranteedOptical path destroys the NW topology

Dedicated optical pathQoS guaranteedOptical path destroys the NW topology

＋

ToRToR ToRToR ToRToR ToRToR10GbE

Internet

Gateway

Open Flow controller

HOPR

ToR

100Gbps=25Gbpsx 4λ

6-D torus network6-D torus network

30 Osaka Univ.

Contention resolution strategy

OECC / PS2016

S1 S2

S4 S5 S6

S8 S9

S3

迂回ルート

ExP

(6, 6, 6)

(4, 6, 4)

S7

Deflection routingDeflection routing

July 4, 2016

Ordinary packet pathExpress path

Destined node HOPR

2X

1X

Breaking path loopBreaking path loop

,

, 1 1, 1

FDL

HOPR Contention!

ToR

HOPR

Destined node

Buffering strategy Buffering strategy

RAM

31 Osaka Univ.

Parameters in numerical simulations

July 4, 2016 OECC / PS2016

HOPR Number of FDLs D 4 Number of 10 Gbit/s links A 20 Number of ports between buffer and switch M 2

Size of each input queue in the shared buffer, Q 10

Network Dimension N 6 Number of nodes 46 = 4096 Additional TTL 50 Traffic injected by each server & Simulation Traffic pattern Uniform5 Injection process Bernoulli5 Packet length 1500B (Deterministic) Ordinary packet load 0.1~1.0 Express flow length L 10 packets Express packet load ' 0.01, 0.001 Cycle peroid (Time-slot) 120 ns (=1500B @100Gbps)

Y.C.Huang et al.,JOCN, Vol.7, No.12, 2015.

32 Osaka Univ.

Drastic reduction of packet dropping

OECC / PS2016

W/o contention resolutionW/o contention resolution With contention resolutionWith contention resolution

July 4, 2016

Note : 885 ExPs traverse 22% of 4,096 routers.

6-D torus network

0 40 80 120 160 200Load from servers [Gbps]

Pac

ket L

oss

Pro

babi

lity

Pure OPS

885 ExP885 ExP

1

10-1

10-2

10-3

10-4

10-5

Pure OPS

Y.C.Huang et al.,IEEE Phtonics J., Vol.8, No.1, 2016.

Pac

ket L

oss

Pro

babi

lityExtended work to leaf & spine fabric, TuF2. Extended work to leaf & spine fabric, TuF2.

33 Osaka Univ.

transport using void

Further improvement by virtual OCS (VOCS)

OECC / PS2016

FDL

A B C D

SW

Buffer Buffer

10G

10G 10G

10G

100GOPS + OCSOPS + OCS

FDLBuffer

SW

Buffer

OPS + VOCSOPS + VOCS Packet

July 4, 2016

Y.C.Huang et al.,Photon Netw.Commun., 2015. (https://www.researchgate.net/publication/284458652).

Pack

et lo

ss p

roba

bilit

y

Pure OPS & VOCS

34 Osaka Univ.

transport using void

Further improvement by virtual OCS (VOCS)

OECC / PS2016

FDL

A B C D

SW

Buffer Buffer

10G

10G 10G

10G

100GOPS + OCSOPS + OCS

FDLBuffer

SW

Buffer

OPS + VOCSOPS + VOCS Packet

July 4, 2016

Y.C.Huang et al.,Photon Netw.Commun., 2015. (https://www.researchgate.net/publication/284458652).

Pack

et lo

ss p

roba

bilit

y

Pure OPS & VOCS

35 Osaka Univ.

Summary


Niche application of OPS would be DC networks.

Optical RAM buffer is a long shot in the near future, but FDL can improve the performance.

Enabling device technologies for extremely energy-efficient optoelectronic packet router are at hand. But it’s a challenge to gain their cost-competitiveness.

OPS/OCS/VOCS performs best against mice & elephant data flows.


E-mail: [email protected]://www.kitayama.tech (personal)

Documents

Optical Packet Switching : Myth, Fact, and Promise · July 4, 2016 OECC / PS2016 3 Osaka Univ. Packet switching : Underlying concept P. Baran,”The beginnings of packet switching: