Embed Size (px)
Citation preview
v
Network Operations, Demo
Matt Peterson
NFD9 – February 11th, 2015
cumulusnetworks.com
Demo
§ OpenStack “in a {USB stick} box” § 2x 10G switches – Cumulus Linux 2.5.0 § 3 hosts – 1 controller, 2 compute; Ubuntu 14.04.1
§ L2 MLAG § Mature Nova-Net (Neutron - even in Juno has limited HA) § Common bridge between switches § LACP on host side
Background - Matt
§ Office of the CTO § Began Cumulus journey February 2013 § Help build customer engineering team § Support cross-functional projects, rabble-rousing
▪ Past and present ASN’s § 33612, 15211, 19230, 30321, 12276 § “web 2.0”, financial, NGO, temporary event, internet exchange
Historical Perspective
§ NOC technician in 1999 § Monitor alerts, wake people up, try to
stay awake, dream of sudo rights, ...
▪ Silos were the norm
§ Diverse skills: database, storage, network, tools
§ Each with a unique measurement of success
!
Fast forward to 2010’s
§ Built Infrastructure teams @Square & @Tumblr § NetEng not scoped as FTE role, generalist as the norm § Excess of 10k+ switch ports, 10+Gbps external traffic
▪ DevOps revolution § Linux, scripting, networking, virtualization, database, etc.
knowledge § Build MPLS backbone, web 2.0 site.. No; but VLAN’s, KVM,
MySQL replication, etc. all within scope of supporting
Today
§ System administrators have morphed into Infrastructure Admins § “The site is slow”.. Walk through JavaScript, DNS, HTTP
response time, database latency, disk, network, etc.. § All layers of the stack are within scope
▪
DevOps
§ Networking is the last holdout of the DevOps revolution
§ DevOps is a vague and well abused term § my definition: common mindshare & shared reasonability
▪ DevOps revolution § Preserve “traditionalists” data plane protocols AND § “Modern” approach to control / management plane
Loading dock to Production
§ ONIE - “PXE that doesn’t suck” § Born from decade of PXE baggage,
talented embedded engineers, & disaggregation goal across the industry
§ Provision a NOS from a MacBook + ethernet cable (note lack of console)... to an established CMDB driven environment
§ Backwards compatible w/ PXE setup (DHCPv4 and TFTP)
PPC example: U-Boot initialization, ONIE as factory default U-‐Boot 2013.01.01-‐gec6bfbe (Mar 20 2014 -‐ 18:33:37) -‐ 3.0.1.7 CPU0: P2020E, Version: 2.1, (0x80ea0021) Clock Configuration: CPU0:1000 MHz, CPU1:1000 MHz, Board: Accton ES4654BF-‐ZZ (cpld_ver: 0.4) I2C: ready ... <snip> Hit any key to stop autoboot: 0 Loading Open Network Install Environment ... ... <snip> ONIE: Using DHCPv4 addr: eth0: 192.168.0.11 / 255.255.255.0 ONIE: Starting ONIE Service Discovery ONIE: Executing installer: http://192.168.0.1/onie-‐installer ... <continued>
GET /onie-‐installer HTTP/1.1 Host: 192.168.0.1 User-‐Agent: onie/1.0 (Linux-‐3.2.35-‐onie+2014.01.02; BusyBox-‐v1.20.0) Connection: close ONIE-‐SERIAL-‐NUMBER: AD19004107 ONIE-‐ETH-‐ADDR: 70:72:CF:96:69:F9 ONIE-‐VENDOR-‐ID: 259 ONIE-‐MACHINE: as4600_54t ONIE-‐MACHINE-‐REV: 0 ONIE-‐ARCH: powerpc ONIE-‐SECURITY-‐KEY: ONIE-‐OPERATION: os-‐install
ONIE – Image Request Headers
Verifying image checksum ... OK. Preparing image archive ... OK. Dumping image info... Description: Cumulus Linux OS-‐Release: 2.0.2-‐a8ec422-‐201404161914-‐final ... <snip> Data Archive Contents ===================== -‐rw-‐r-‐-‐r-‐-‐ build/Devel 83218432 2014-‐04-‐17 02:43:39 sysroot.squash.xz -‐rw-‐r-‐-‐r-‐-‐ build/Devel 1680 2014-‐04-‐17 02:43:39 uboot_env.tar.xz -‐rw-‐r-‐-‐r-‐-‐ build/Devel 3914264 2014-‐04-‐17 02:43:39 uImage-‐powerpc.itb ... <continued on next slide>
ONIE – Image Extraction
Formating /dev/sda1... done. Formating /dev/sda3... done. Validating sha1 for uImage-‐powerpc.itb... done. Validating sha1 for sysroot.squash.xz... done. Installing OS-‐Release 2.0.2-‐a8ec422-‐201404161914-‐final into image slot 1 ... Copying sysroot into /dev/sda6... done. Verifying sysroot copy... OK. Copying kernel uImage into /dev/sda5... done. Installing OS-‐Release 2.0.2-‐a8ec422-‐201404161914-‐final into image slot 2 ... Copying sysroot into /dev/sda8... done. Verifying sysroot copy... OK. Copying kernel uImage into /dev/sda7... done. Validating sha1 for uboot_env.tar.xz... done. Updating u-‐boot environment variables... done. Updating u-‐boot environment variables. Success: System provisioning complete. Rebooting...
ONIE – OS Installation
U-‐Boot 2013.01.01-‐gec6bfbe (Mar 20 2014 -‐ 18:33:37) -‐ 3.0.1.7 CPU0: P2020E, Version: 2.1, (0x80ea0021) Clock Configuration: CPU0:1000 MHz, CPU1:1000 MHz, Board: Accton ES4654BF-‐ZZ (cpld_ver: 0.4) I2C: ready ... <snip> Hit any key to stop autoboot: 0 ** local boot ** Booting primary image... Using Accton Technology Corporation AS4600_54T machine description Cannot reserve gpages without hugetlb enabled Linux version 3.2.46-‐1+deb7u1+cl2+1 (build@monster-‐04.cumulus)...
Booting on OS Install, ONIE muted from boot process
Loading dock to Production
§ ZTP - “everyone has this” § DHCP response URL or USB stick § Shell script executed as root § Pick your lang: bash, python, ruby, perl
▪ Common use cases: § Install license key § SSH key for Ansible, Puppet cert install, etc
Zero Touch Provisioning – Example DHCP configuration
ddns-‐update-‐style none; default-‐lease-‐time 600; max-‐lease-‐time 7200; authoritative; log-‐facility local7; option cl-‐provision-‐url code 239 = text; subnet 10.99.0.0 netmask 255.255.255.0 { range 10.99.0.10 10.99.0.250; option routers 10.99.0.1; option domain-‐name-‐servers 10.99.0.1; option domain-‐name "example.net"; option cl-‐provision-‐url "http://server.example.net/first_boot.sh"; }
Zero Touch Provisioning – Example Shell Script
#!/bin/bash # CUMULUS-‐AUTOPROVISIONING # add non-‐CL repo, update apt cache, install ASCII art generator echo "deb http://http.us.debian.org/debian wheezy main" >> /etc/apt/sources.list apt-‐get -‐q -‐y update apt-‐get –q –y install figlet # set hostname SWHOSTNAME=demo-‐myswitch echo $SWHOSTNAME >/etc/hostname echo "127.0.0.1 $SWHOSTNAME" >> /etc/hosts invoke-‐rc.d hostname.sh start # update motd rm /etc/motd /usr/bin/figlet ”My Company" > /etc/motd thedate=$(/bin/date) echo –e ”\n\tBorn on $thedate\n" >> /etc/motd # install license wget -‐q -‐O /root/license.txt http://10.99.0.1/license.txt /usr/cumulus/bin/cl-‐license -‐i /root/license.txt exit 0
Zero Touch Provisioning – Completed
% ssh cumulus@demo-‐myswitch Linux switch01-‐sfo02 3.2.46-‐1+deb7u1+cl2+1 #3.2.46-‐1+deb7u1+cl2+ __ __ ____ | \/ |_ _ / ___|___ _ __ ___ _ __ __ _ _ __ _ _ | |\/| | | | | | | / _ \| '_ ` _ \| '_ \ / _` | '_ \| | | | | | | | |_| | | |__| (_) | | | | | | |_) | (_| | | | | |_| | |_| |_|\__, | \____\___/|_| |_| |_| .__/ \__,_|_| |_|\__, | |___/ |_| |___/
Born on Tue Jun 10 23:29:00 UTC 2014 cumulus@demo-‐myswitch:~#
Open architectures transform culture
§ Why no CLI? § Bash is the common Linux adopted interface,
we follow Linux’isms where possible by default § When Debian moves to systemd, we will; when
lldpctl added JSON support, we did; etc. § CL is modeled after traditional Linux
distributions, not traditional NOS’s - preserve the protocols, not the battle scars
Demo - review
§ Hosts cabled to out-of-band 1G mgmt net
§ USB sourced install on spine-01 via ONIE
§ Spine-02 & hosts (pre-seed) install over OOB net
§ LLDP provides cabled topology, template interfaces file
§ Puppet modules handle OpenStack components
Open architectures transform culture
§ Power and freedom is very liberating § Use only pre-install software § Bash scripts (no fancy frameworks) § PythonCGIServer (no Apache, rails, etc installed) § Parsing LLDP results (key=value, pre-JSON support) § Puppet for configuration files, daemon taming
▪ Removing “behind the scenes” covers, empowers our partners & users
© 2015 Cumulus Networks. Cumulus Networks, the Cumulus Networks Logo, and Cumulus Linux are trademarks or registered trademarks of Cumulus Networks, Inc. or its affiliates in the U.S. and other countries. Other names may be trademarks of their respective owners. The registered trademark Linux® is used pursuant to a sublicense from LMI, the exclusive licensee of Linus Torvalds, owner of the mark on a world-wide basis.
§ Thank You!
cumulusnetworks.com 21
