An Architecture for a Highly Available Disaggregated NOS built upon Open Networking Linux

Pat Moore, Director of Sales, Metaswitch NetworksJonathan Cumming, Architect, Metaswitch Networks

Track: Networking and SONiC

Metaswitch Networking Software

Some of our Customers

www.metaswitch.com/stacks

Architecture Protocols OCP Engagements

Yang Model Manager

Protocols

Control Plane Mgmt

Hardware Abstraction

Silicon SDK

Data-Plane

NETCONF / CLI / etc.

Lin

ux

Layer 2

LLDP

LACP

STP

ERPS

L2VPN

SyncE

ELMI

EVPN

Layer 3

BGP

OSPF

ISIS

RIP

IGMP

PIM

L3VPN

Seg Routing

MPLS

IP/MPLS

MPLS-TP

LDP

RSVP

OAM

BFD

LSP-Ping

Fault Mgmt

Perf Mon

ETH-CC/LB/ -

LT/DM/LM

SAI Demo at OCP 2016

SAI Tunnel Proposal

SAI Fast Rereoute Proposal

Disaggregation Today

x86

Linux

Standard, swappable HW Open OS Vibrant App Ecosystem Automated Flexible

Apps

OS

H/w

Compute

Hardware

NOS

Standard swappable HW Open NOS: SONiC, Stratum Vibrant App Ecosystem HA through network design

Data Center Networking

Proprietary, closed HW & SW No App Ecosystem Vendor lock-in HA in each switch/router

AppsNOS

Hardware

Telco Networking

Telco-grade Highly Available Router

• Single node in the network

• Survives hardware and software failures

• Minimizes impact of failure

• Redundant hardware using stacked whiteboxes

• Surviving unit takes over if one unit fails

NETWORKINGArchitecture of a Portable Telco NOS

CPUMemory Power FanFlash Merchant

Switch Silicon

ONL Linux LTS Kernel (v4.x) Switch Drivers

Co

ntr

ol P

lan

e

RIB LDB

BGP OSPF Other protocols

Man

age

me

nt

and

Ap

ps NETCONF

RESTCONFCLI SNMP Analytics

Monitoring

Automation

Orchestration

YANG-based management, as well third party monitoring and other tools

Control plane services separately runnable, multi-instantiable, restartable

Key network router databases and control plane modelling services

ONL provided abstraction layer

Linux kernel

Config DB

Platform hardware

NOS

IP/MPLS OAM

LIM FTM XCM

ON

L

ONL Platform Code Platform Dependent Code

ONL Platform Abstraction LayerOCP SAI

YANG

COTS and Standard Switch Hardware

NETWORKINGArchitecture of a Distributable NOS

Control plane extension to program distributed data plane

Drivers and time-critical function

NOS

CPUMemory Power FanFlash Merchant

Switch Silicon

ONL Linux LTS Kernel (v4.x) Switch Drivers

ON

L

ONL Platform Code Platform Dependent Code

ONL Platform Abstraction LayerOCP SAI

Co

ntr

ol P

lan

e

RIB LDB

BGP OSPF Other protocolsIP/MPLS OAM

LIM FTM XCM

Ap

ps

NETCONFRESTCONF

CLI SNMP Analytics

Monitoring

Automation

OrchestrationConfig DB YANG

COTS and Standard Switch Hardware

Dis

trib

uti

on Distributed Fabric Manager

LACP PDUY.1731 CCM

BFD ESMC

NETWORKINGDistributed Data Plane

COTS and Standard Switch Hardware

ON

LC

on

tro

l Pla

ne

Ap

ps

COTS and Standard Switch Hardware

ON

L

Distributed Fabric Manager

L3L2

RIB/FIB

VPN

Switch Drivers

SAIONL Platform Abstraction

LayerSwitch Drivers

SAIONL Platform Abstraction

Layer

CLI

Config DB

NETCONF

Neighbor Protocols

CCMNeighbor Protocols

CCM

Distributed data plane usingstacked ASICs or a Fabric backplane

Drivers and time-critical function

Mapping onto available hardware resources, synchronisation

Router-scoped state and protocol function

NETWORKINGData Plane Failure

COTS and Standard Switch Hardware

ON

LSwitch Drivers

SAIONL Platform Abstraction

Layer

Neighbor Protocols

CCM

Control Plane signals interface changes to protocol neighbors

Connectivity maintained over remaining interfaces

COTS and Standard Switch Hardware

ON

LC

on

tro

l Pla

ne

Ap

ps

Distributed Fabric Manager

L3L2

RIB/FIB

VPN

Switch Drivers

SAIONL Platform Abstraction

Layer

CLI

Config DB

NETCONF

Neighbor Protocols

CCM

NETWORKINGRedundant Control Plane

COTS and Standard Switch Hardware

ON

LC

on

tro

l Pla

ne

Ap

ps

COTS and Standard Switch Hardware

ON

L

Distributed Fabric Manager

L3L2

RIB/FIB

VPN

Switch Drivers

SAIONL Platform Abstraction

LayerSwitch Drivers

SAIONL Platform Abstraction

Layer

CLI

Config DB

NETCONF

Neighbor Protocols

CCMNeighbor Protocols

CCM

L3L2

RIB/FIB

VPN

Config DB

CLI NETCONF

Distributed Fabric Manager

Redundant Configuration and Management solution

Active and Backup Control Plane Locations

NETWORKINGControl Plane Failure

COTS and Standard Switch Hardware

ON

L

COTS and Standard Switch Hardware

ON

LSwitch Drivers

SAIONL Platform Abstraction

LayerSwitch Drivers

SAIONL Platform Abstraction

Layer

Neighbor Protocols

CCMNeighbor Protocols

CCM

L3L2

RIB/FIB

VPN

Config DB

CLI NETCONF

Distributed Fabric Manager

Co

ntr

ol P

lan

eA

pp

s

Distributed Fabric Manager

L3L2

RIB/FIB

VPN

CLI

Config DB

NETCONF

Configuration replayed on restart

Graceful restart + state replication to minimize disruption

Programming resynchronised once GR has completed

Neighbour Protocols maintain connectivity during control plane failure

Non-stop Forwarding

Telco L3VPN Scenario

• Reboot active unit

• Kill Control Plane of active unit

OSPF + LDP

BGP/MPLS L3 VPN

LACP + µBFD LACP + µBFD

OSPF OSPF

1

2

CE1 PE1 P PE2 CE2PING PING

1/ge1

2/ge1

1/ge2

2/ge2

r

Metaswitch NOS Toolkit components

Provider-Edge Router failover

PoC only

Case Study –Failover

NETWORKING

Demo

NETWORKINGNOS CookbookStep-by-Step guide to building a Portable NOS for Telcos

Leverages OCP Technologies

ONIE, ONL, SAI etc.

Standard, Flexible Architecture

Portability built-in from inception

Provides modern and open interfaces

Automation, Orchestration etc.

Supports open & closed source components

WhitePapers

Switch

Abstraction

Interface

www.metaswitch.com/cookbook

Call to Action

White Box HA -Facilitating Telco deployment

Metaswitch NOS Toolkit info:www.metaswitch.com/nos-toolkit

Download the NOS Cookbook: www.metaswitch.com/cookbook

SAI Project source code: github.com/opencomputeproject/SAI