Facebook Networking &the Open Compute Project (OCP)

March 16, 2016 - Open Networking Summit

Omar BaldonadoFacebook Network TeamOCP Networking Project Co-Chair

Launched Live Video

▪Supports millions of concurrent viewers for a single stream ▪Rolled out to dozens of countries in months

More than 80% of our daily active users are outside the US

software everywhere

fboss

and more…Hybrid controllers

Backbone & edge traffic engineering

Network analytics& simulationNetwork modeling

Circuit automation & testing

IPv6

Traffic shaping

Config automation & mgmt

Passive & active monitoring

100G

operations over features

FBOSS/Wedges in production#

of F

BOSS

/wed

ges

in p

rodu

ctio

n

Time

What happens when racks show up?

▪ Every week, every data center ▪ “Provisioning” ▪ RAM disks, addresses, images ▪ And what happens when they

disappear?

Upgrades

https://code.facebook.com/posts/145488969140934/open-networking-advances-with-wedge-and-fboss/

Changing the image every week (instead of twice/year)

fail fast over fail-proof

https://code.facebook.com/posts/1534350660228025/netnorad-troubleshooting-networks-via-end-to-end-probing/

NetNORAD

detect network interruptions and automatically mitigate

them within seconds

rememberthe whole network lifecycle

Open DataCenter Stack

Open Rack

Leopard Knox

Wedge Battery

Power

6-PackCold Storage

Cooling

Open DataCenter Stack

Open RackLeopardKnoxWedgeBatteryPower6-PackCold StorageCooling

Scalable

Efficient

Flexible

SpitfireServer (AMD)

Power Supply

BatteryCabinet

Freedom Servers

Data Center

Triplet Rack

2011

Watermark(AMD)

Mezzanine Card v1

Windmill (Intel)

2012

Winterfell Knox

Open Rack v1

Group Hug

2013

Micro Server (Panther)

Honey Badger

Cold Storage

Open Rack v2

Mezzanine Card v2

2014

Yosemite

2015

6-Pack

Wedge

OCP Networking as of March 2015

▪ One accepted switch ▪ Software building blocks ▪ Testing efforts starting

Takeaway: Disaggregation was here, but still ramping up!

What a difference a year makes

OCP networking hardware

▪ Full design packages ▪ Community review ▪ Testing program ▪ Disaggregation ▪ Hardware and software ▪ Multiple layers

11 OCP data center switches accepted

▪ 16x40G ▪ 48x10G ▪ 32x40G ▪ 36x40G ▪ 32x100G

▪ Facebook Wedge 100 ▪ Alpha 48x10G and 32x100G

Newly shared OCP specs - new DC switches

▪ 48x10G Mediatek/Nephos ▪ 32x100G Edge-core with Cavium

Newly shared OCP specs - new silicon

▪ Facebook “6-pack” - 128X40G ▪ Edge-core 256x100G, 512x100G

Newly shared OCP specs - chassis/modular

▪ Edge - based on Broadcom “Qumran” - deep buffers, expandable TCAM ▪ Access - 48x1G w/ stacking & POE options

Newly shared OCP devices - edge & access

▪ 2 indoor, 1 outdoor ▪ 802.11ac

Newly shared OCP devices - access points

OCP hardware needs… software

▪ Every OCP networking device supports choice in software

Software

OCP software - moving up the stack

▪ Initial work was in “building blocks” ▪ ONIE, ONL, SAI ▪ Still continuing ▪ Moving up to actual forwarding functionality

A growing ecosystem of software

▪ Multiple projects and providers emerging ▪ Open source and commercial ▪ Distributed and centralizedSAI

▪ Managing Wedge via “Metal-as-a-Service” ▪ Created an FBOSS snap ▪ OCP Hack-a-thon - created an Open Switch snap

OCP Wedge Demos:

▪ TORC - “Applications, Microservices, VNFs controlled by Top-of-Rack Controller” ▪ Used Wedge’s micro-server extensively ▪ Docker, Mesos Master, FBOSS, OpenNSL, ONL, OpenBMC, Calico

OCP Wedge Demos:

▪ “Evolving a Telcom operator network into an IT convergence network”

▪ Ported OpenSwitch to Wedge ▪ Ported Indigo to Wedge ▪ OpenFlow support ▪ Interested in SAI

OCP Wedge Demos:

What’s next for Facebook Networking & OCP?

▪ Working with the ecosystem and user community ▪ Reaching to new areas of the network with OCP Telco and TIP ▪ Code, code, code