Switch FabricArchitecture

Jack RegulaJanuary 12, 2001

PLX Switch Fabric Architecture

� Agenda� Communications Infrastructure Trends

� What Is a Switch Fabric?

� How Does it Work?

� The Compelling Nature of Switch Fabric

� Platform Segmentation

� Architecture Overview

Edge

Access

CoreBase Station

Controller

WirelessTower

ATMSwitch

DSL

Remote AccessConcentrator (RAC)

DSLAM

v.90

Switch

Switch

Add/Drop MUX Add/Drop MUX

WirelessLAN Gateway

EthernetSwitch

Router

Client

Client

EnterpriseStorage

Laptop

DWDM

TerabitSwitch Router

TerabitSwitch Router

DWDM

Digital SetTop Box

Cable ModemSwitch

MediaGateway

DigitalPhones

IP Fax

Communication Infrastructure Trends

Edge

Access

Core

Increasing Bandwidth

Base StationController

WirelessTower

ATMSwitch

DSL

Remote AccessConcentrator (RAC)

DSLAM

v.90

Switch

Switch

Add/Drop MUX Add/Drop MUX

WirelessLAN Gateway

EthernetSwitch

Router

Client

Client

EnterpriseStorage

Laptop

DWDM

TerabitSwitch Router

TerabitSwitch Router

DWDM

Digital SetTop Box

Cable ModemSwitch

10G ENet G ENet 1

0/100

OC-3 OC-12 OC-48 OC-192

MediaGateway

DigitalPhones

IP Fax

What is a Switch Fabric?

� An interconnected network of switchingdevices� The fabric contains many input and output ports� The fabric transports the data from inputs to outputs

� A fast, concurrent & transparentmechanism to transfer data� High-speed, point-to-point links connecting switching

devices� Many switching devices working in parallel� Sophisticated protocols to ensure reliable, ordered delivery

of data in accordance with QOS

� A highly-available network� Error detection mechanisms with correction via retry� Fabric redundancy for fault tolerance� Hot swap/live maintenance and expansion“Switch Fabric”

Fundamental Building Block for Networking Equipment

What is a Switch Fabric Controller?

To anotherswitching device

To anotherswitching device

Fabricinterface

To CPU or I/O

SwitchingDevice

Businterface

Fabricinterface

“Switch Fabric”

� A chip with fabric I/F and bus I/F� Connects to other switching devices to

form a switch fabric� Connects a CPU or I/O subsystem into the

fabric� Implements the switch fabric protocols

Why Switch Fabric is Compelling

● Higher BW● More ports/slots● Higher-availability● Longer distance● Time-to-Market

Enables Next-GenerationEquipment and End-Users

● Logical extension ofPCI roadmap

● New revenue streams● Higher ASP

Silicon Vendors

Equipment Vendors

Corporations

Switch Fabric

● More BW for employees● Connect remote/mobile users● Gain new subscribers● E-commerce!

Switch Fabrics in Current Platforms

Lucent NX64000 Multi-Terabit Switch/Router

“The NX6400 innovative switch fabric delivers6.4 Tbps switch capacity per chassis….”

Lucent CBX500 WAN Switch

“The CBX500 Multiservice WAN Switchis a high performance cell switch basedon crosspoint technology as its core switch fabric”

Cisco MGX8260 Media Gateway

“The system also provide additionalscaling through the use of inter-changeable switch fabrics growingthe capacities from 5 Gbps to 15 Gbps”

Nortel Versalar Switch Router 2500

“This high performance forwarding is achieved with a scalable, non-blockingswitch fabric……”

Next-Generation PlatformsDemand More Complex

Switch Fabrics

Evaluating a Switch Fabric■ Sufficient Non-blocking Throughput and Port Count ?■ Is It a Complete Solution for High Availability?

◆ Transparent fault and error recovery◆ Line card hot swap◆ Live fabric expansion and maintenance◆ Fabric redundancy

■ Does it Preserve Transaction Order?◆ Producer-consumer model◆ Cells/transactions delivered in order, one time only, even if fault or transient

error

■ Legacy Support:◆ Can you use same PCI cards and packaging◆ Does your software require modification or augmentation?

■ Is it transparent?◆ Boot, configuration, operation, fault recovery

■ How far does it get you?◆ Chip to chip, board to board, shelf to shelf, box to box

■ Is it available?

Conceptual Switch Fabric Model

Switch FabricASIC

Switch FabricASIC

Switch Fabric ASICCPU

I/O

I/O

Switching Devices

Switch FabricASIC

Switch FabricASIC

Switch FabricASIC CPU

I/O

I/O

1. Transports data from multiple sources to multiple destinations2. Increases scalability (bandwidth, I/O ports, distance)3. Improves reliability/availability (self-healing, fail-over)

Bus transactions �

exchange of switch fabric cells

Conceptual Model 2:Virtual PCI-PCI Bridges

VIRTUAL PCI

SwitchFabric

Controller

PR

IMA

RY

SE

CO

ND

AR

Y

SwitchFabric

Controller

PR

IMA

RY

SE

CO

ND

AR

Y

SUBORDINATE PCI

PRI

SEC

SECSUB

PRI

bus no.

reg.

HOST PCI

PLX Switch Fabric Architecture

Dual -Counter Rotating Rings

FabricInterface

CellProcessing

Engine

BusInterface

15 feetdistance

PCI

PCI

PCI

PCI

PCI

PCI

PCI

cPCI

cPCI

cPCI

LayeredArchitecture

PCI v2.2 Interface32-bit, 33/66MHz

PCI Local Bus

PCI Local Bus

Performance� Bandwidth of Link = L,

N=Number of Nodes

� Best Case BW = 2LN

� Worst Case BW = 4L

� Uniform Traffic Distribution:BW =8L(6.4 GB/sec; 51.2 Gbps)

GBP

GB

P

GBP

GBP

GBPG

BP

GBP

GBP

PLX Switch Fabric Controller Architecture

Application: Embedded PCI / cPCI

Host

Bridge

PCI

CPU

MEM

PCI

PCI

PCI

PCI

PCI

PCI

PCI

PCI

PCI

PCI

PCI

PCI

16-slot, 6U CompactPCI Fabric

J5

J4

J3

J2

J1

User I/O ConnectorsUser I/O Connectors

Low Profile 32-bit PCI to

SF Bridge

Low Profile 32-bit PCI to

SF Bridge

Low Profile 32-bit PCI to

SF Bridge

Low Profile 32-bit PCI to

SF Bridge

Application #2: CompactPCI Expansion

HostBr.

CPU

cPCI

cPCI

cPCI

cPCI

cPCI cPCI

cPCI

cPCI

cPCI

cPCI

cPCI

PCI

cPCI

Domain BDomain A

Dual Host Solution

PCI

PCI

PCI

PCI

PCI

MEM

CPU

Host

Bridge

PCI

MEM

CPU

Host

Bridge

PCI

PCI

PCI

PCI

Edge & Access Equipment

PCI

DataControl

DataEdge & Access Equipment

PLX Switch Fabric

Control

Short Design Cycle

Investment Preservation

Compatibility

Short Design Cycle

Investment Preservation

Compatibility

Performance (10x bandwidth)

Scalability (100x)

High Availability

Performance (10x bandwidth)

Scalability (100x)

High Availability

PLX Switch Fabric

Application Example

ControlModule

ControlModule

STM-1

STM-1GigEthernet

GigEthernet

Multi-ChannelVoice

Multi-ChannelVoice

Multi-ChannelVoice

Multi-ChannelVoice

Multi-ChannelVoice

Chip

to

Chip

Backplane

Media Gateway

Flexibility to Expand

Shelf

to

Shelf

PLX Switch Fabric Architecture Summary■ Bandwidth and Scalability

◆ 51.2 Gbps◆ 224 PCI Bus Segments at 32/64 bits, 33/66 MHz◆ 15 foot maximum distance for chassis to chassis

■ A Complete Solution for High Availability!◆ Transparent fault and error recovery◆ Line card and fabric hot swap◆ Integrated fabric redundancy◆ Multi-host support with fail-over

■ Transaction Order Fully Preserved!◆ Producer-consumer model◆ Cells/transactions delivered in order,◆ Transactions complete exactly once; even if fault or transient error

■ Complete Legacy Support!◆ 100% compatible with all PCI specifications◆ Use same PCI cards, packaging and software◆ Fully Transparent: boot, configuration, operation, fault recovery

Scalability and High-Availability on PCI

PLX Switch Fabric Architecture Summary� Scalability

� 224 PCI bus segments� 51.2 Gbps aggregate bandwidth� 15 feet distance chassis-to-chassis

� High Availability� Self-healing via retries & redirection� Live node insertion� Multi-host fail-over via SW

� PCI Compatibility� 100% compatible with all PCI Specifications� 33MHz or 66MHz PCI interface� Auxiliary Signal support for 4 PCI devices� External EEPROM for config and BIOS

� Electrical/Mechanical/Thermal� IBM 0.25u CMOS� 2.5Vcc supply, 5V/3.3V tolerant I/o� 388-ball PBGA, 27mmx27mm

Scalability and High-Availability on PCI

PLX Switch Fabric Architecture Schedule

� Chips & Tools Availability� Samples Q1 ‘01

� Rapid Development Kit (RDK) Q2 ‘01

� Rapid Development System (RDS) Q2 ‘01

The Leading Supplier of High Speed InterconnectSilicon and Software to the Networking and

Communication Industry

PLX Switch Fabric ArchitectureLive Demonstration at Bus&Board

� Demonstration of switching capabilities,throughput

� Suite 327

� Monday 5:00-8:00 p.m.

� Tuesday noon-5:00 p.m.