Cisco IOS XE Routers ASR 1000 & ISR 4000 & CSR 1000v

Cisco IOS XE Routers ASR 1000 & ISR 4000 & CSR 1000v

Jaromír Pilař, Consulting Systems Engineer, CCIE #2910

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public

•  Introduction, What is IOS XE

•  IOS-XE Software Architecture

•  IOS-XE Platforms Architecture •  ASR 1000, ISR 4000, CSR 1000v

•  IOS-XE Network Programmability

•  Summary

Agenda

2


Introducing IOS XE

•  2007 •  ASR1000 introduced as the first routing platform using IOS XE

software

•  2013 •  ISR4451 introduced as the first branch

routing platform using IOS XE software •  ISR4000 series routers inherited the new IOS architecture and

married with the previous innovations from the ISR G2 series of routers

•  2014 •  5 new ISR4000 series routers introduced to extend coverage

through all branch connectivity needs

•  2015 •  3 new ASR1000 chassis along with new MIP and Interfaces blades

With ASR1000 and ISR4000 series routers

3


What is IOS XE

BIG differences!

•  Linux is the underlying operating system for the chassis

•  IOSd runs as a process in Linux

•  Benefit from protected memory and process isolation

•  Very familiar CLI (some things are best kept the same)

•  Separation of control and data planes into discrete processes

•  Multicore support for data plane

•  Introduction of services plane in addition to control and data plane

How is it different than Classic IOS at 30,000 feet?

4


Linux? where?

•  Linux, yes, but the only interface with the system is via IOSd

•  IOSd presents the same CLI interface that everyone loves from other platforms like 7200, 7600, and ISR G2 routers

•  Because IOSd is running as a discrete process it has protected memory that is isolated from crashes in other processes and failures in other components in the system.

•  Individual software component upgrade opportunity

•  With “service internal” and “request platform software system shell” commands you can find Linux. Don’t do it without a good reason. Here be dragons and you taste good with ketchup. Requires one-day license from TAC since you go well with ketchup.

I don’t see a shell prompt anywhere!

5


Same CLI

•  In general configurations from Classic IOS platforms move forward to IOS XE without any changes

•  There are certain features like QoS, carrier grade NAT (CGN), WAAS, CME that when moved forward are going to have slight variations or need updating to take advantage of new features

•  Cisco Active Advisor can analyze configurations from Classic IOS platforms and provide updated configurations for IOS XE platforms https://ciscoactiveadvisor.com/

Not like IOS-XR that looks like you understand it until you don’t…

6


Divided we stand, united we fall!

•  Classic IOS is a single threaded monolithic blob of code that has served us well for a long time •  Impossible to separate control and data plane •  Processors aren’t getting faster so much, their number of cores are

growing though

•  Multi-core lets us to dedicate certain cores for control plane and others for data plane, i.e. no starving data plane for control plane

•  Furthermore, we can use one chip architecture for control and a separate for data plane for mix and match to meet needs

•  We have even created a services plane that can run alongside IOSd and not impact platform performance

Wait, isn’t it supposed to be the other way around?

7


Services plane

•  All platforms have multiple cores on the control plane, no truck rolls needed!

•  IOSd consumes one core with occasional use for extra cores for specific features

•  Remaining cores are given to a hypervisor which can run dedicated applications to provide appliance like services •  vWAAS •  EnergyWise •  SNORT •  WireShark

•  Single memory pool is used for Linux, IOSd, and the services plane

Coffee, tea, soft drinks, peanuts, watch your elbows please…

8


IOS XE software architecture •  IOS + IOS XE Middleware + Platform

Software

•  IOS runs as its own Linux process for control plane

•  Linux kernel with multiple processes running in protected memory

•  Fault containment, re-startability

•  ISSU of individual SW packages

•  With redundant data plane hardware packet loss can be as low as 50 ms at failover

Dat

a pl

ane

Con

trol-p

lane

IOS active

Platform Adaptation Layer (PAL)

Forwarding manager-FP

IOS standby

Chassis manager

Linux Kernel

Forwarding manager-RP Chassis manager

Linux Kernel

Forwarding engine client

Forwarding engine driver

Control messaging

9


Cha

ssis

IOS XE software architecture

Dat

a pl

ane

Con

trol-p

lane

IOS active



IOS standby

Chassis manager

Linux Kernel


Linux Kernel



Control messaging

Dat

a pl

ane

Con

trol-

plan

e IOS active



Linux Kernel




Control m

essaging

ISR4000 implementation

ASR1000 implementation

10


Dat

a pl

ane

Con

trol p

lane

IOS active


Forwarding manager - FP

IOS standby

Chassis manager

Linux Kernel

Forwarding manager - RP Chassis manager

Linux Kernel



Control messaging

IOS XE architecture building blocks •  Runs Control Plane •  Generates configurations •  Maintains routing tables (RIB,

FIB…)

•  Initialization of RP processes •  Initialization of installed cards •  Detects and manages OIR of

cards •  Manages system status,

environmentals, power, EOBC

•  Provides abstraction layer between hardware & IOS

•  Manages ESP redundancy •  Maintains copy of FIB and

interface list •  Communicates FIB status to

active & standby data plane FM

•  Maintains copy of FIBs •  Programs forwarding plane and

forwarding engine DRAM •  Statistics collection & RP

communication

•  Communicates with forwarding manager in control plane

•  Provides interface to QFP client & driver

•  All messaging done via IP datagrams in the kernel or over the backplane of the chassis

11


IOS-XE Platforms

12


ASR 1000 series

13

§  Integrated firewall, VPN, encryption, NBAR, CUBE

§  Scalable on-chip service provisioning through software licensing

§  Fully separated control and forwarding planes

§  Hardware and software redundancy §  In-service software upgrades

Instant On Service Delivery

§  Line-rate performance 2.5G to 200G+ with services enabled

§  Hardware QoS engine with up 128K queues per ASIC

§  Investment protection with modular engines, IOS CLI and SPAs for I/O

Compact, Powerful Router

Business-Critical Resiliency

ASR 1002-X ASR 1004 ASR 1006-X / 1006 2.5–36 Gbps

10-40 Gbps

10-100 Gbps

2.5 - 20 Gbps

10-200 Gbps

ASR 1013

One IOS-XE Feature Set

ASR 1001-X 40-200 Gbps

ASR 1009-X


ASR1000 building blocks

Midplane

ES

P

FECP

QFP Crypto Assist.

interconnect

PPE BQS ES

P

QFP

interconn.

PPE BQS

FECP

Crypto Assist.

interconnect

RP CPU

interconn. GE switch S

IP

SPA SPA

IOCP SPA Aggreg.

interconnect

RP CPU

interconn. GE switch

SIP

SPA SPA

IOCP SPA Aggreg.

interconnect

SIP

SPA SPA

IOCP SPA Aggreg.

interconnect

Route Processor Handles control plane Manages system Embedded Service Processor

Handles forwarding plane traffic

SPA Interface Processor Houses SPA’s Queues packets in & out

•  Route Processor (RP) •  Handles control plane traffic •  Manages system

•  Embedded Service Processor (ESP) •  Handles data plane traffic

•  SPA Interface Processor (SIP) •  Shared Port Adapters provide interface connectivity

•  Centralized Forwarding Architecture •  All traffic flows through the active ESP, standby is

synchronized with all flow state with a dedicated 10-Gbps link

•  Distributed Control Architecture •  All major system components have a powerful control

processor dedicated for control and management planes

14


Cisco Intelligent WAN Architecture

MPLS Unified Branch

3G/4G-LTE

Internet

Private Cloud

Virtual Private Cloud

Public Cloud

Application Optimization

Enhanced Application Visibility and Performance

AVC, NBAR, Netflow, HQOS, WAAS, Akamai

Secure Connectivity

Comprehensive Threat Defense

ZBFW, CWS, IPsec, Suite B

Intelligent Path Control

Application Aware Routing

Performance Routing

Transport Independent

Simplified Hybrid WAN

DMVPN

Enterprise IWAN - IWAN-App/APIC-EM SP-IWAN - vMS/NSO ORCHESTRATION


All these services without killing the box at branch ?


Sure…Use an ASR1K !


RP

Interconn.

Forwarding Processor (Dataplane)

FECP Crypto Assist

QFP Subsystem

Interconn.

Route Processor

(Controlplane)

Solution: Build ASR1k..but in an ISR disguise

SPA Agg.

SPA SPA

Interconn.

IOCP SPA Agg.

SPA SPA

Interconn.

IOCP

IOSd

Serviceplane

Forwarding Processor (Dataplane)

Forwarding CPUs

Route & Service Processor

(Controlplane)

NIM FPGE SM-X

MGF

ASR 1000

ISR 4000


Crypto

FECP

ASR1K ESP Architecture

GE, 1Gbps I2C SPA Control SPA Bus

ESI, 11.2Gbps SPA-SPI, 11.2Gbps Hypertransport, 10Gbps Other

RPs RPs RPs ESP SIPs

E-RP* PCI*

E-CSR

QFP

TCAM Resource DRAM Packet Buffer DRAM

Part Len / BW SRAM

SA table DRAM

Dispatcher Packet Buffer

DDRAM

Boot Flash (OBFL,…)

JTAG Ctrl

EEPROM

Temp Sensor

Reset / Pwr Ctrl

Packet Processor Engine

…

PPE1 PPE2 PPE3 PPE4 PPE5

PPE6 PPE7 PPE8 PPE40

BQS

Reset / Pwr Ctrl Interconnect

SPI Mux

Interconnect

Forwarding Engine Control Processor Manages board Programs QBS, PPE, Crypto Linux Kernel

Buffering Queuing & Scheduling Executes complex QoS scheduling (shapers, LLQ’s,…) Queues and schedules packets in due time

Quantum Flow Processor Overall packet forwarding

Packet Processor Engine Multicore CPU Routes and applies features to packets


ISR 4451 Hardware Diagram

Data Plane (10 core)



Control Plane (4 cores)

Ctrl SVC1

SVC2 SVC3

FPGE

DRAM

Multi Gigabit Fabric

DSP

SM-X

System FPGA

Peripheral Interconnect

DRAM

Console / Aux

Mgmt Ethernet

Flash

USB

4xPCIe DDR3 4xSGMI

DDR3

1xSGMI

10 Gbps/slot

NIM NIM

NIM

2Gb/slot

SM-X

10 Gbps XAUI

1 Control Plane Core RP and FECP-like roles

3 Services Core

10 Cores, 1 thread / core 5 fwd cores by default 4 remaining cores license activated

Inline Cryptography No Crypto Assist chip

No hardware TCAM

BQS on a core One Core dedicated to BQS Always active (5+1 or 9+1 cores)


Cisco ISR 4400 Architecture

Platform Controller

Hub

Control Plane (1 core) and Services Plane (3

cores)

System FPGA

Data Plane (6 or 10 cores)

Multigigabit Fabric

NIM

ISC

SM-X

FPGE

DRAM

4xPCIe

4xPCIe

10G XAUI

4 x 1 Gb/sec SGMII

1 Gb/sec SGMII

10 Gb/sec per slot

2 Gb/sec per slot

DRAM

Mgmt Ethernet

USB

Console/Aux

Flash

Service containers

IOSd

KVM WAAS

EnergyWise

21

3rd party


Cisco ISR 4300 Architecture

DRAM

Mgmt Ethernet

USB

Console/Aux

Flash

Platform Controller

Hub

Control Plane (1 core) Services Plane (3 cores)

Data Plane (4 cores)

System FPGA

Multigigabit Fabric

NIM

ISC

SM-X

FPGE

4xPCIe 10G XAUI

3 x 1 Gb/sec SGMII

1 Gb/sec SGMII

10 Gb/sec per slot

2 Gb/sec per slot

Service containers

IOSd

Note:4321 uses 2DP, 1CP & 1SP cores

mSATA

KVM WAAS

EnergyWise

22

3rd party


ISR4000 - World’s Broadest Service Offerings in One Box Simplified Services Integration

The Ultimate Converged Branch – No More Appliances

Native, Full Featured Security, AVC, WAN Opt, UC

Ease of Service Deployment – No Truck Rolls

Network, Compute and Storage

WAN opt Compute Storage UC Path Control App Visibility Security


Cisco Branch Router Evolution ISR 4431 & 4300 family Making a complete ISR 4000 family ISR 4451

First ISR based on IOS XE

ISR G2 family 800, 1900, 2900 & 3900 Taking ISR G1 architecture to the next level

ISR G1 family 1800, 2800, 3800 The first architecture custom designed for integrated services

Cisco 2500 Cisco’s first family of branch routers for 23 different deployments

Cisco 2600 Superseded 2500. Considered one of Cisco's premier products.

2014

2013

2009

2004

1998

1993

Not shown here: 700, 1600, 1700, 4000/4500, 3600 & 3700 series routers


Pay-As-You-Grow with Cisco ISR 4000 Series

ISR 4321 50-100 Mbps

ISR 4331 100-300 Mbps

ISR 4351 200-400 Mbps

ISR 4431 500-1000 Mbps

ISR 4451 1-2Gbps

Investment Protection Without Oversubscription

4-10X Faster Add performance and services anytime

Flexible consumption options

4x GE (all dual) 3x NIM 2x Enh SM 4x GE (all dual)

3x NIM 3x GE (all dual) 3x NIM 2x Enh SM 3x GE (dual+RJ+SFP)

2x NIM 1x Enh SM 2x GE (dual+RJ)

2x NIM


Cisco ISR 4000 Family I/O Design Management Interface •  out-of-band control plane •  connection directly to a

management network

Front-Panel GE •  RJ45/SFP GE Interfaces •  PoE+ available on some

models

Network Interface Modules (NIMs) •  Larger and more powerful

than EHWICs •  Up to 8 ports per module •  DSPs directly on modules

Optional Drive NIM for Embedded Applications •  RAID 1 for data protection •  Single HD (future) and

dual SSD options

USB Connections •  2 times type A for file storage •  USB type B console in addition to RJ45 console

and aux ports

Enhanced Service Modules •  Compatible with Cisco® ISR G2 •  Up to 10-Gbps connection to system •  Faster and more powerful than SMs


Connectivity Options Outside the office Inside the office

Analog Voice §  FXS, E/M §  SRST §  CME

Ethernet / Switching §  SM 16/24/48 port switch module §  SM routed ports (6xGE or 4xGE/

1x10GE) CU/SFP module §  NIM 4 and 8 port switch module §  NIM 1 and 2 port routed module

Cisco UCS® E-Series §  2, 4, 6 and 8-core Intel® Xeon®

processors §  Up to 3 TB storage and 48 GB

DRAM

§  T1/E1 §  FXO, PRI §  BRI

§  T1/E1, T3/E3, serial §  ADSL, VDSL

§  3G/4G

PSTN

WAN/ Internet

Backup


CSR 1000V - virtualized IOS XE •  Virtualized IOS XE

•  Generalized to work on any x86 system

•  Hardware specifics abstracted through a virtualization layer

•  Forwarding (ESP) and Control (RP) mapped to vCPUs

•  Bootflash: NVRAM: are mapped into memory from hard disk

•  Boot loader functions implemented by GRUB

•  Limitations •  No dedicated crypto engine – we leverage the Intel

AES-NI instruction set to provide hardware crypto assist.

•  No QFP – lower forwarding performance •  No HW Accelerators – Less efficient feature

processing

Control Plane Forwarding Plane

vNIC vCPU vMemory vDisk

Physical Hardware

Hypervisor (VMware / Citrix / KVM)

Chassis Mgr. Forwarding Mgr.

IOS

Chassis Mgr. Forwarding Mgr.

FFP Client / Driver

FFP code Linux Container

CPU Memory Disk NIC

28


IOS-XE Network Programmabilty

29


Embedded Event Manager

Syslog email notification

SNMP set Counter

CLI Applets SNMP get SNMP

notification Application

specific TCL

Policies Reload or switch-over

EEM Applets multi-event-correlation

IOS.sh Policies

Actions

Event Detectors

Syslog Event

Process Scheduler Database

Interface Descriptor

Blocks

Syslog ED

Watchdog ED

Interface Counter

ED

CLI ED

OIR ED

ERM ED

EOT ED

RF ED

none ED

GOLD ED

XML RPC ED

SNMP EDs

Remote: •  Notification Local: •  Notification •  Get/Set

NetFlow ED

IPSLA ED

Route ED

Timer EDs

•  Cron •  Count down

HW EDs

•  Fan •  Temp •  Env •  ...

CDP LLDP

ED

802.1x ED

MAC ED

Embedded Event Manager (EEM)

30


Device-Level API – RESTCONF Problem: How to programmatically interact with a Router – in a model-based, loosely coupled, easy to understand and standards-based way?

Available Nov 2015

Solution: Use RESTCONF from IOS XE3.17 (and XR 6.1) onwards

•  RESTful interface over HTTPS

•  JSON/XML Data Representation

•  Based on YANG Models •  IETF Standard Models where they exist •  Cisco Models where common across platforms •  Cisco Platform specific models

•  ASR1000, ISR4000, CSR1000, …

Try it out – available today

YOUR App

HTT

PS

JSON/ XML


Device-Level – Hosting Options YOUR App

YOUR App

Container •  Dedicated CPU/RAM/Storage •  Any OS in a KVM OVA •  Low Latency and Delay •  Virtualized, Elastic •  Fate Sharing, local Visibility

YOUR App

Blade •  Physical CPU/RAM/Storage •  Any OS and/or Hypervisor •  Lower Latency and Delay •  Modular •  Fate Sharing, local Visibility

Server / Controller •  Unlimited CPU/RAM/Storage •  Any OS and/or Hypervisor •  High Latency and Delay •  Extra Deployment •  Extra Footprint


UCS E-Series Portfolio S

cala

bilit

y

Feature Richness

Cisco UCS-E140S

•  Service Module •  Vmware, Hyper-V,

Citrix Certified •  Intel E3 4 Core

Processor •  vWLC, vWAAS,

Physical Security

Cisco UCS-E180D

•  Service Module •  Vmware, Hyper-V, Citrix

Certified •  Intel E5 8 Core Processor •  vWLC, vWAAS, Virtual

Desktops, Physical Security, Security applications

Cisco UCS-E160D

•  Service Module •  Vmware, Hyper-V, Citrix

Certified •  Intel E5 6 Core

Processor •  vWLC, vWAAS, Virtual

Desktops, Physical Security

Cisco UCS-EN120S

•  Service Module •  VMware and

Hyper-V Certified •  Network Compute

Applications – vWLC, vWAAS


Cisco UCS E-Series NCE – NIM Small form factor, Compact, Multipurpose Blade Housed in ISR 4Ks only– UCS-EN140N

•  Use Case •  Ideal for hosting 1-2 cisco network applications and other lightweight applications

•  Product Features •  Intel® Atom® 4-core processor

•  Up to 8GB of RAM

•  Up to 200GB SSD storage

•  Available on all ISR 4k routers

•  Cisco Integrated Management controller (Cisco IMC) comes standard for out of band management

•  Certified for Bare-metal OS like Microsoft 2012 R2 and Redhat Linux and Hypervisors like Vmware ESXi 5.5 and

Microsoft Hyper-V

Available Oct 2015


Virtual Service Containers Problem: Can I run my App inside a Router ?

Solution: Yes !

From IOS XE3.17 onwards

•  Option to enable unsigned containers

•  Any 3rd party KVM

•  Libvirt based format / YAML manifest file

•  Requires 4GB+ dedicated RAM

•  May require persistent storage (NIM-SDD)

•  ASR1000, ISR4000, CSR1000

Try it out – available today

virtual-service signing level unsigned

# virtual-service install name myapp package flash:myapp.ova # show virtual-service list # show virtual-service detail name myapp # virtual-service connect name myapp aux|console

YOUR App

Available Nov 2015


What is a Service Container? Service Containers use virtualization technology (LXC and KVM) to provide a hosting environment on Cisco routers/switches for applications which may be developed and released independent of platform release cycles. Virtualized environment on a cisco device. Use Case Cisco Virtual Services:

•  Lightweight Application Hosting •  Example: ISR-WAAS ( KVM ) •  Example: SNORT ( LXC )

Use Case Third Party Services:

•  KVM Hosted Applications

Container

Network OS

Virtual Service


Linux OS

IOSd Control Plane

Snort

KVM

IOS-XE Container Architecture Cisco Apps ISR-WAAS

Customer and 3rd Party Applications (KVM only)

Platform-Specific Data Plane AppNav

Virtual Ethernet


Useful App on LXC container

Product Overview §  Works directly on a Linux service container – Single core §  Open source intrusion prevention system for real-time traffic analysis §  Good Enough Security at the Branch to Meet Compliance needs §  IPS/IDS functionality with an IOS IPS look and feel

LXC


What do I need to add to an ISR4K system?

•  Service Containers (currently) REQUIRE additional DRAM beyond the 4GB system default •  Additional DRAM beyond 4GB will be available to a KVM application

•  Example: 8GB DRAM will have 4GB available to Service Containers •  Example: 16GB DRAM will have 12GB available to Service Containers

Memory

•  No storage is included by default and applications do not have access to bootflash. •  Options include internal MSATA SSD on 4300 Series, NIM-SSD or NIM-HD on all ISR4K. •  Smaller sizes and lower reliability SSD options at lower price will be available in late CY15.

Storage

Note: ASR1K/CSR requirements will be different.


NIM-SSD:

•  1 or 2 hot-swappable 200GB SSD drives

•  50GB, 100GB and 400GB options in CY15

NIM-HD:

•  1 hot-swappable 500GB or 1TB drive

•  Available late 2015

SSD-MSATA-200G:

•  Doesn’t consume a NIM slot!

•  Embedded 200GB SSD storage

•  Smaller sizes available in CY15.

•  Not available on 4431/4451

Storage Options


Summary

41


•  IOS XE is an evolution of IOS •  provides operational continuity •  configurations move forward •  IOS protocol troubleshooting moves forward

•  Data / control / service plane separation •  Functionality isolation, DOS protection •  Improved and predictable performance •  Cost efficiencies

IOS XE summary

42


•  Operational excellence •  QoS, High Availability, easy service enablement

•  Platform management •  Multiple processors, memories, busses to be

monitored

•  Common code and feature sets across multiple locations in the network •  Eases deployments, decreases incompatibilities

IOS XE summary

43


IOS XE Platforms

•  Physical – ISR 4000, ASR 1000

•  Virtual – CSR 1000v

•  Ideal for feature rich Intelligent WAN deployment

•  Platform for branch consolidation – routing, switching, application

Wide variety of options

Thank you

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 45

We’re ready. Are you?

Documents

Cisco IOS XE Routers ASR 1000 & ISR 4000 & CSR 1000v