Compass-EOS for Linkmeup

Alex Clipper 17/1/15

Agenda

Conventional chip review

D-Chip review

Standard carrier-grade router architecture

Compass-EOS router architecture

Dev-test methodologies

Questions

D chip on the map

3600

412 mm2

4

Zooming in on the challenge

Data Transfer Between CMOS Processors

The Conventional Way

Electrical transmission via Copper

Loss is frequency dependent

High Power consumption as additional electronics are needed to compensate for the signal loss

Limited Interconnect length

The Compass Way

Using photons in fibers to transfer the data

Parallel optics are used to achieve high BW

Small interconnect footprint (10% of the total chip area)

Low Power consumption

Negligible Loss, supports long distances (>200m)

CMOS

Serial Links

Traffic-in Package

Tx Matrix Rx Matrix

FibersTraffic-out

CMOS

CMOS

Fibers

Compass Multi Terabit interconnect

First Commercial Implementation – The D-chip

Tx

Rx

icPhotonics™ - Fastest Optical Interconnect

1.34Tb/s Full Duplex Bandwidth

Order of magnitude higher Chip

I/O Density. 64Gb/s per mm2

Passive optical links that stretch to

Hundreds of Meters vs.

Centimeters with Electronics

• Direct Coupling to CMOS Chip

• Low energy consumption: 10pJ/bit

• Dozens of Patents Covering Technology & Processes

• Flexible form factor

• Ready for Mass Production

Laser

MatrixPhoto

Detector

s

Standard

I/O

Digital

CMOS chip

Direct

optical

interface

1.34 Tb/s In

Fiber Optic Bundle

Compass Optical Interconnect

Half bundle illuminated

1.34 Tb/s In

The enabling Technology

2 x 350G

2 x 1344G

Low Power – 2.7T at < 15 W

High Density – 2.7T in 40 mmSQ

Future Proof – Protocol Agnostic Technology

Very Scalable – 20T+ per Slot

BW density 30X

ASR9k LC Architecture

A simple Multicast scenario, fabric links normalized at 100G

Multicast in conventional architectures

Ingress

buffer

Switch

Fabric

Forward

From

FabForward

Egress

buffer

ToFab

Buffer

Ingress

bufferForward

From

Fab ForwardEgress

buffer

ToFab

Buffer

Ingress

bufferForward

From

Fab ForwardEgress

buffer

ToFab

Buffer

Ingress

bufferForward

From

Fab ForwardEgress

buffer

ToFab

Buffer

30G Multicast stream

100Gbps

100Gbps

100Gbps

100Gbps

30G Unicast stream

30G Unicast stream

30G Unicast stream

Congestion

• From fab resource on egress card congested

Back pressure

• Back pressure (BP) is signaled to all ingress

cards

X

X

X

• In response to BP, all ingress cards reduces

transmit rate towards the congested card

30G Multicast stream

X

• Multicast traffic to non-congested egress

cards can in worst case also be impacted !

X

?!?

?!?

Affected stream direction (streams with packet loss)

Operation of a distributed router system

Line cards comprises several

Silicon processors performing

different functions (Queuing,

Forwarding etc.)

Data needs to be exchanged

between these Silicon chips.

The interconnect can be chip

to chip, board to board or

chassis to chassis.

Data transfer rate between

processors is a significant

factor when it comes to router

performance

MAC

NPU

Egress

Buffers

D-chip

100Gbps

100Gbps

100Gbps

30G Multicast stream

30G Unicast stream

30G Unicast stream

30G Unicast stream

30G Multicast stream

MAC

NPU

Egress

Buffers

D-chip

MAC

NPU

Egress

Buffers

D-chip

MAC

NPU

Egress

Buffers

D-chip

• Egress interface is congested

Congestion

• Egress interface queuing have full visibility of all

traffic to this egress destination and handle

packets according to operator SLA priorities.

• No impact on Traffic destined for ports on

same or other cards

A simple Multicast scenario, Backplane links normalized at 100G

r10004 architecture

Compass EOS - Evolving Router Architectures

Ingress

bufferForward

ToFab

Buffer

From

FabForward

Egress

buffer

Ingress

bufferForward

ToFab

Buffer

From

FabForwardEgress

buffer

Egress

buffer

Ingress

buffer

Egress

Buffer

Ingress

buffer

Forward

Forward

ToFab

Buffer

From

Fab

ToFab

Buffer

From

Fab

ToFab

Buffer

From

Fab

ToFab

Buffer

From

Fab

Compass EOS – Router Architecture

Ingress

Forward

engine

Egress

Forward

engine

Ingress

Buffer

Queing &

Egress

Buffer

Ingress

Forward

engine

Egress

Forward

engine

Ingress

Buffer

Queing &

Egress

Buffer

To

switch

Buffer

From

switch

Buffer

To

switch

Buffer

From

switch

Buffer

Input ports

Input ports

Output ports

Output ports

D

D

•Queing

•Egress Processing

•Egress Buffer

•Queing

•Egress Processing

•Egress Buffer

•“Switch fabric” Router The Compass way

• Chip to Chip Optical inter-connect

• True Output Switched Router

• Switch Fabric eliminated

• No ingress side queuing

• Maintaining all of the functionality

DP components – the LC

iNP

Interlaken 150G

Interlaken 100G

Interlaken 150G

Interlaken 100G

eNP

D

eBufferiBuffer

iBufferInterlaken 60G

Interlaken 60G

Interlaken 60G

Interlaken 60G

D

eBuffer

Interlaken 100G

Interlaken 150G eNP

Interlaken 100G

Interlaken 150G

iBuffer

iBufferInterlaken 60G

Interlaken 60G

Interlaken 60G

Interlaken 60G

IL 30G

L2 FPGA

PMCSFP+SFP+

L2 FPGA

L2 FPGA

L2 FPGAPMC

SFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

PMCSFP+SFP+

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

IL 30G

CPU Module10GE

TCAMTCAM

iNP

TCAMTCAM

Compass-1: Overview

800Gbps system

6 RU , 19” shelf

4 x Line Cards

Optical BP

Front to Back air flow

Distributed DC Power Supply

80x10GE, 8x100GE

Summary Full Mesh advantages (Real Output Queue)

Ease of QoS configuration

What you configure is what you get

Packet drop takes place where you have full visibility

Better Multicast scale

Better system security

More accurate SLA

Predicted and deterministic behavior

D iNP eNP

D iNP eNP

CPU

macmac

macmac

mac

D iNP eNP

D iNP eNP

CPU

macmac

macmac

mac

D iNP eNP

D iNP eNP

CPU

macmac

macmac

mac

eNP iNP D

eNP iNP D

CPU

macmac

macmac

mac

CPU MEM MEM

CPU MEM MEM

CPU MEM MEM

CPU MEM MEM