ONIE / Cumulus Networks Webinar

v

Open Network Install Environment (ONIE) Webinar

Curt Brune, Member of Technical Staff

July 16, 2014

Webinar Agenda

ONIE Solves a Real Problem

ONIE Design Approach

ONIE Adoption

ONIE Technical Deep Dive

2 cumulusnetworks.com

Vision 10+ years ago the compute side of the datacenter dramatically changed and enabled high scale applications, but that change never made it to networking. It’s time for networking without constraints.

Mission To enable high capacity networks that are easy to deploy and affordable helping customers realize the full promise of the software-defined data center.

Cumulus® Linux®

Cumulus Linux is the first full-featured Linux OS for datacenter networking.

About Cumulus Networks


Ecosystem

Cumulus® Linux®

Dell

Other Network OS

Quanta Edge-Core

Penguin Computing

Automation Orchestration

MonitoringRouting

Unlock the Power of an Open Networking Ecosystem


ONIE – Open Network Install Environment

§ Modern, efficient installer §  Network OS installer

•  Discovery, transfer, and execution •  Implemented through Linux kernel with BusyBox (OpenWRT-like)

§ Industry support §  Enables open network hardware ecosystem §  Open source §  Well-documented

•  http://www.onie.org/ cumulusnetworks.com 5

Challenges Facing Bare Metal Switching

§ Confluence of Difficulties §  Each hardware platform is unique §  Data center operators want to automate provisioning §  End users want choice of HW and SW §  Similar to server industry moving from various

hardware platforms (SUN, SGI, x86) §  PXE was OK, but could be much more modern

cumulusnetworks.com 6

Create an Ecosystem for Bare Metal Switches

Bare Metal Hardware Platform

Operating System A


Operating System B


Operating System C

•  OS Vendors wrote their own platform dependent OS installer and integration layer

•  Hardware Vendors maintained separate SKUs for each combination of OS and platform

•  End Users could not change the OS, it was an appliance

Before ONIE The Appliance Model

Platform Dependent

Platform Dependent

Platform Dependent


Create an Ecosystem for Bare Metal Switches cont.


ONIE

Operating System A


Operating System B


Operating System C

•  Hardware Vendors maintain a single ONIE SKU for each hardware platform. Opens up the distribution channels

•  OS Vendors target a common installer format. Speeds time to market

•  End Users can pick and choose among a variety of Operating Systems. Automated mega-scale OS and hardware provisioning is now possible

ONIE Smooths out the Rough Edges

ONIE ONIE


ONIE Innovation

§ A New Approach §  More modern than PXE §  Pre-installed on the switch by the hardware vendor §  A mini operating system based on Linux §  Could have enhanced BIOS or U-Boot, but Linux is way

more capable: device drivers, networking, applications §  Support IPv6, HTTP/S, DHCP and existing TFTP infra §  Designed for Automation §  Additional meta-data during network installs


ONIE Adoption

§ Time line §  Late 2012, brainstormed initial features §  Early 2013, evangelized with HW partners §  May 2013, first public demo at OCP MIT Workshop §  Summer 2013, first products available from multiple

vendors §  Summer 2013, project incubated by OCP §  June 2014, project fully adopted by OCP


Hardware Vendors


Operating System Vendors


§ Standardize ONIE Implementations … §  UTSA OCP Certification Laboratory §  Collaborate on test plans §  Help develop test suites §  Help HW vendors certify their ONIE

implementation §  Benefit OS vendors §  Improve the quality of the ONIE project

OCP – ONIE Certification Program


Halftime

§ Break For Questions


Technical Deep Dive

§  How Does ONIE Work? §  Leverages the Linux kernel and BusyBox §  A few state machines to manage network interfaces

and image discovery methods §  Shell scripting is the programming language


ONIE – Anatomy of a Network Switch

( Management Interfaces ) ( Data Plane )

CPU SoC

DRAM Boot Flash

Mass Storage

Switching ASIC

Serial Console

Ethernet Mgmt Port

10Gb Port

40Gb Port … 10Gb

Port 40Gb Port

…

PCIe


ONIE – Uses Management Interfaces

( Management Interfaces ) ( Data Plane )

CPU SoC

DRAM Boot Flash

Mass Storage

Switching ASIC

Serial Console

Ethernet Mgmt Port

10Gb Port

40Gb Port … 10Gb

Port 40Gb Port

…

PCIe


ONIE – Bare Metal Install – First Time Boot Up

Boot Loader (HW Vendor Supplied)

ONIE (HW Vendor Supplied)

Installer (OS Vendor)

Boot Loader •  Low Level boot loader, configures CPU complex •  Loads and boots ONIE

ONIE •  Linux Kernel with Busybox •  Configures management Ethernet interface •  Locates and executes an OS installer •  Provides tools and environment for installer

OS Installer •  Available from network or USB •  Linux executable •  Installs vendor OS into mass storage

Network OS (OS Vendor Supplied)

Fetches

Installs


ONIE – Subsequent Reboots – Vendor’s OS is Already Installed

Boot Loader (HW Vendor Supplied)

ONIE (HW Vendor Supplied)

Boot Loader •  Low Level boot loader, configures CPU complex •  Loads and boots OS vendor’s installed OS

Network OS •  Configures Switching ASIC •  Runs Network Protocols •  Provides CLI

Network OS (OS Vendor Supplied)

ONIE •  Still exists, but is not used •  Available for uninstall / re-install operations


ONIE – Network OS Installer Discovery and Install Behavior

Configure Network Interface

Locate Installer

Run Installer

•  Uses DHCPv4, DHCPv6 •  Configures Ethernet interface for IPv4 / IPv6 •  Configures DNS and hostname

•  Determines the location of an installer executable •  Examines local file systems, e.g. USB flash drives •  Uses DHCP options, DNS Service Discovery, Multicast

DNS and IPv6 Neighbors

•  Downloads installer via URL •  Passes various environment variables to

installer •  Launches installer


ONIE – Discovery Waterfalls

§ Management interface §  IPv6 network discovery §  DHCPv4 §  Static IP address


1 § OS installation source §  Local USB §  HTTP §  FTP §  TFTP

2

§ File name search §  onie-‐installer-‐<arch> §  onie-‐installer-‐<vendor> §  onie-‐installer

ONIE – Installer Mode GRUB Menu


ONIE – Booting into Installer Mode


ONIE – IPv4 / IPv6 HTTP and TFTP Waterfall


ONIE – Image Request HTTP Headers


ONIE – Other Behaviors

§ Provides a mechanism for a network OS to invoke the above behaviors.

Reinstall Remove currently installed OS and return to the “out of box” provisioning state

Uninstall Completely wipe out everything, except ONIE

Rescue Reboot box into ONIE for repair, debug and forensics

Update Install a new ONIE version

Diag Run HW Vendor’s diag (optional)


ONIE – Ongoing Development

§ Support Multiple CPU Architectures §  PowerPC – Ready Today §  x86 – Ready Today §  x86 Virtual Machine §  Planning for ARM and MIPS §  Maintain ONIE behaviors across architectures


Resources

§ ONIE Websites §  Main Page: http://www.onie.org/ §  Wiki: http://www.opencompute.org/wiki/Networking/ONIE §  Source Code: https://github.com/opencomputeproject/onie §  Documentation: http://opencomputeproject.github.io/onie/ §  Mailing List:

http://lists.opencompute.org/mailman/listinfo/opencompute-onie §  Blogs: http://cumulusnetworks.com/blog/ §  Twitter: @ProjectONIE @curtbrune


© 2014 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.

§ Thank You!


Technology

ONIE / Cumulus Networks Webinar