Cisco live local high level aci

C97-730020 -01 © 2013 Cisco and/or its affiliates. All rights reserved. Cisco Confidential 1

Local Edition

Application Centric Infrastructure and the Nexus 9000


Key Takeaway

Application Centric Infrastructure (ACI) Introduction

ACI Fabric

Services and Hypervisor Integration

Application Policy Infrastructure Controller

Services for ACI


ACI Fabric

Non-Blocking Penalty Free Overlay

App DB Web

Outside (Tenant VRF)

QoS

Filter

QoS

Service

QoS

Filter


APIC


•  Extend the principle of Cisco UCS® Manager service profiles to the entire fabric

•  Network profile: stateless definition of application requirements   Application tiers

  Connectivity policies

  Layer 4 – 7 services

  XML/JSON schema

•  Fully abstracted from the infrastructure implementation   Removes dependencies of the infrastructure

  Portable across different data center fabrics

## Network Profile: Defines Application Level Metadata (Pseudo Code Example) <Network-Profile = Production_Web> <App-Tier = Web> <Connected-To = Application_Client> <Connection-Policy = Secure_Firewall_External> <Connected-To = Application_Tier> <Connection-Policy = Secure_Firewall_Internal & High_Priority> . . . <App-Tier = DataBase> <Connected-To = Storage> <Connection-Policy = NFS_TCP & High_BW_Low_Latency> . . .

App Tier DB Tier

Storage Storage

Web Tier

Application

The network profile fully describes the application connectivity requirements


All forwarding in the fabric is managed through the application network profile •  IP addresses are fully portable anywhere within the fabric •  Security and forwarding are fully decoupled from any physical or virtual network attributes •  Devices autonomously update the state of the network based on configured policy requirements

DB Tier

Storage Storage

Application Client

Web Tier App Tier

Application policy model: Defines the application requirements (application network profile)

Policy instantiation: Each device dynamically instantiates the required changes based on the policies

VM VM VM

10.2.4.7

VM

10.9.3.37

VM

10.32.3.7

VM VM

APIC


Actions: No new hosts or VMs Evacuate hypervisors Re-balance clusters

PetStore Event

PetStore Dev •  Leaf 1 and 2 •  Spine 1 – 3 •  Atomic counters

PetStore Prod •  Leaf 2 and 3 •  Spine 1 – 2 •  Atomic counters

PetStore QA •  Leaf 3 and 4 •  Spine 2 – 3 •  Atomic counters

VXLAN

Per-Hop Visibility Physical and

Virtual as One

ACI Fabric provides the next generation of analytic capabilities

Per application, tenants, and infrastructure: •  Health scores •  Latency •  Atomic counters •  Resource consumption

Integrate with workload placement or migration

Triggered Events or Queries

APIC


•  Elastic service insertion architecture for physical and virtual services

•  Helps enable administrative separation between application tier policy and service definition

•  APIC as central point of network control with policy coordination

•  Automation of service bring-up/tear-down through programmable interface

•  Supports existing operational model when integrated with existing services

•  Service enforcement guaranteed, regardless of endpoint location

Web Server

App Tier A

Web Server

Web Server

App Tier B

App Server

Chain “Security 5”

Policy Redirection

Application Admin

Service Admin

Ser

vice

G

raph

begin end Stage 1 …..

Stage N

Pro

vide

rs inst

inst

…

Firewall

inst

inst

…

Load Balancer

……..

Ser

vice

Pro

file

“Security 5” Chain Defined


•  Integrated gateway for VLAN, VxLAN, and NVGRE networks from virtual to physical

•  Normalization for NVGRE, VXLAN, and VLAN networks

•  Customer not restricted by a choice of hypervisor

•  Fabric is ready for multi-hypervisor

Virtual Integration Network Admin

Application Admin

PHYSICAL SERVER

VLAN VXLAN

VLAN NVGRE

VLAN VXLAN

VLAN

ESX Hyper-V KVM

Hypervisor Management

ACI Fabric

APIC

APIC

VMware Microsoft

Red Hat XenServer

VMware Microsoft Red Hat


Object-Oriented Centralized Automation

RESTful XML/JSON

Open Ecosystem Framework

Comprehensive Programmability and

System Access

Northbound API •  Rapid integration with existing

management frameworks

•  OpenStack

•  Tenant- and application-aware

Southbound API •  Publish data model •  Open source •  Enables application portability

*Only straight chains supported at FCS

System Management


Automation Tools

Orchestration Frameworks

NetQoS

SolarWinds

Tivoli Software

CA Technologies HP

Arbor Networks

NetBrain

VMware

Microsoft

XenServer

InfoVista

Red Hat KVM

Puppet Labs

Opscode Python

CFEngine

CloudStack

OpenStack

VMware

Nebula Eucalyptus

Microsoft XenServer Red Hat KVM


Key Takeaway


ACI Fabric



Services for ACI


Ado

ptio

n

True virtualization and abstraction requires hardware innovation

Server Virtualization

Network Virtualization

Intel/AMD Virtualization Support

ACI-Enabled Hardware


•  Industry’s most efficient fabric: ‒  1/10 Gb edge – High-density 40 Gb spine (100 Gb-

capable) ‒  1 million+ IPv4 and IPv6 endpoints ‒  64,000+ tenants ‒  220K+ 1/10 Gb hosts in a single tier 3:1

oversubscribed fabric •  Routed fabric – optimal IP forwarding ‒  Bridging (L2) and routing (L3) of VXLAN, NVGRE,

VLAN at scale ‒  No x86 gateways – physical and virtual ‒  Application agility – place and join without limits in the

fabric •  Full visibility into virtual and physical •  Common operations from hypervisor to compute,

to fabric, to WAN

Spine Inline overlay hardware database 288 x 40 Gb ports Higher capacity and lower cost

Fabric Optimization Improved utilization1588 timing and Latency ECMP-based approaches

Scale Intelligent caching Overlay hardware offload Improved analytics

APIC


Insieme Fabric Controller ACI Spine Nodes

ACI Leaf Nodes

•  ACI Fabric provides: ‒  Decoupling of endpoint identity, location, and associated policy, all of which are independent from the underlying topology

‒  Full normalization of the ingress encapsulation mechanism used: 802.1Q VLAN, IETF VXLAN, IETF NVGRE

‒  Distributed Layer 3 gateway to ensure optimal forwarding for Layers 3 and 2

‒  Support for standard bridging and routing semantics without standard location constraints (any IP address anywhere)

‒  Service insertion and redirection

‒  Removal of flooding requirements for IP control plane (ARP, GARP)

APIC


•  ACI Fabric is based on an IP fabric supporting routing to the edge with an integrated overlay for host routing ‒  All end-host (tenant) traffic within the fabric is carried through the overlay

•  The fabric is capable of supporting an arbitrary number of tiers and/or partial mesh if required

•  Why choose an integrated overlay? ‒  Mobility, scale, multi-tenancy, and integration with emerging hypervisor designs

‒  Data traffic can now carry explicit meta data that allows for distributed policy (flow-level control without requiring flow-level programming)

IP fabric with integrated overlay Each node will be

assigned loopback IP address(es) advertised

through IS-IS

IP un-numbered 40 Gb links

APIC


•  ACI Fabric decouples the tenant endpoint address - its “identifier” - from the location of that endpoint, which is defined by its “locator,” or VTEP address

•  Forwarding within the fabric is between VTEPs (eVXLAN tunnel endpoints) and takes advantage of an extender VXLAN header format, referred to as the eVXLAN policy header

•  The mapping of the internal tenant MAC or IP address to the location is performed by the VTEP, using a distributed mapping database

VTEP VTEP VTEP VTEP VTEP VTEP

Payload IP eVXLAN VTEP

APIC


VXLAN VNID = 5789

VXLAN VNID = 11348

NVGRE VSID = 7456

Any to Any

802.1Q VLAN 50

Normalized Encapsulation

Localized Encapsulation

IP Fabric Using eVXLAN Tagging

Payload IP eVXLAN VTEP

•  All traffic within the ACI Fabric is encapsulated with an extended VXLAN (eVXLAN) header

•  External VLAN, VXLAN, NVGRE tags are mapped at ingress to an internal eVXLAN tag •  Forwarding is not limited to, nor constrained within, the encapsulation type or

encapsulation ‘overlay’ network •  External identifies are localized to the iLeaf or iLeaf port, allowing re-use and/or translation

if required

Payload

Payload

Payload

Payload

Payload

Eth IP VXLAN Outer

IP

IP NVGRE Outer IP

IP 802.1Q

Eth IP

Eth MAC

Normalization of Ingress Encapsulation

APIC


10.1.1.10 10.1.3.11 10.6.3.2 10.1.3.35 10.1.1.10 10.1.3.11 10.6.3.2 10.1.3.35

•  ACI Fabric supports full Layer 2 and Layer 3 forwarding semantics; no changes required to applications or endpoint IP stacks

•  ACI Fabric provides optimal forwarding for Layer 2 and Layer 3 ‒  Fabric provides a pervasive SVI, which allows for a distributed default gateway ‒  Layer 2 and Layer 3 traffic are directly forwarded to the destination endpoint

•  IP ARP and GARP packets are forwarded directly to the target endpoint address contained within ARP or GARP header (elimination of flooding)

Distributed Default Gateway Directed ARP Forwarding

APIC APIC


•  The forwarding table on the Leaf switch is divided between local (directly attached) and global entries

•  The Leaf global table is a cached portion of the full global table

•  If an endpoint is not found in the local cache the packet is forwarded to the ‘default’ forwarding table in the spine switches (1,000,000+ entries in the spine forwarding table)

10.1.3.11 fe80::462a:60ff:fef7:8e5e 10.1.3.35

Proxy A Proxy A Proxy B Proxy B

fe80::62c5:47ff:fe0a:5b1a

10.1.3.35 Leaf 3 10.1.3.11 Leaf 1

Leaf 4 Leaf 6

fe80::8e5e fe80::5b1a

10.1.3.35 Leaf 3

Proxy A *

10.1.3.11 Port 9

Global station table contains a local cache of

the fabric endpoints

Local station table contains addresses of all hosts attached directly to

the iLeaf

Proxy station table contains addresses of all hosts attached

to the fabric


•  ACI Fabric tracks the congestion along the full path between the ingress leaf and the egress leaf through the data plane (real-time measurements) ‒  Congestion on switch-to-switch ports

(external wires) ‒  Congestion on internal ASIC-to-ASIC

connections (internal wires) •  Fabric load-balances traffic on a ‘flowlet’ basis ‒  Dynamic shedding of active flows from

congested to less congested paths •  Fabric prioritizes small (and early) flowlets ‒  Provides DC-TCP behavior without having to

modify host stacks ‒  Ramps up large TCP flows faster

APIC


•  Improve fabric capacity of the fabric (resulting in more VMs per port)

•  Improve application response over standard ECMP

Dynamic Load Balancing and Dynamic Flow Prioritization

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9

1

0.12 0.21 0.20

Nor

mal

ized

Ave

rage

Fl

ow C

ompl

etio

n Ti

me

Up to 80% improvement in application flow completion time Up to 60% improved utilization of the fabric capacity

Small Flows (0,100KB)

Medium Flows (100KB, 5MB)

Large Flows (5MB, Inf)

ACI Dynamic Load Balancing + Flow Prioritization

Standard ECMP Network


•  TEP-to-TEP counters ‒  Packet and Byte counts between all iLeaf TEPs

‒  Matrix of load to and from each iLeaf to all other iLeaves

‒  Always active; level of granularity is TEP to TEP

Odd Bank Even Bank

TEP-to-TEP Atomic Counters


Path 1 Path 2 Path 3 Path 4

Packets Sent from Leaf 2 to Leaf 5

Path 1 2068

Path 2 2963

Path 3 2866

Path 4 2506

Difference

Path 1 2

Path 2 0

Path 3 -3

Path 4 0

Packets Received on Leaf 5 Sent from Leaf 2

Path 1 2066

Path 2 2963

Path 3 2869

Path 4 2506

APIC


•  Matrix of latency measurements between all iLeaves is tracked at each iLeaf •  Per-port average latency and variance to up to 576 other iLeaves

  Maximum accumulation, sum of square, and packet count

•  Per-port 99% latency (recorded to up to 576 other iLeaves)   99% of all packets have recorded latency less than this value

•  48-bucket histogram

Boundary Clock

PTP Time Sync

External Clock Source (Pulse Per Second [PPS]) on Each Supervisor in the Spine Chassis


•  1 million+ IPv4 and IPv6 endpoints within a single fabric •  64,000+ tenants within a single fabric •  200,000+ 10 Gb ports •  Any service anywhere for physical and virtual •  Normalizes encapsulations for VXLAN, VLAN, NVGRE ‒  No need for additional software or hardware gateways to connect between physical and virtual ‒  No latency penalty and no throughput penalty

VM VM DB VM VM DB VM VM DB VM VM DB

QFP QFP QFP QFP

APIC


Key Takeaway


ACI Fabric



Services for ACI


•  Service automation requires a vendor device package. It is a zip file containing

•  Device specification (XML file)

•  Device scripts (Python)

•  APIC interfaces with the device using device Python scripts

•  APIC uses the device configuration model provided in the package to pass appropriate configurations to the device scripts

•  Device script handlers interface with the device using its REST or CLI interface

Device Package Device Specification <dev type= “f5”> <service type= “slb”> <param name= “vip”> <dev ident=“210.1.1.1” <validator=“ip” <hidden=“no”> <locked=“yes”>

APIC – Policy Element Device Model

Device-Specific Python Scripts

APIC Script Interface

Script Engine

APIC Node

Device Interface: REST/CLI

Service Device

APIC


Tenant X

Self-Service User – App Ops or Tenant Admin

•  Publishes service graphs •  Deploys service graphs

•  Uploads device package •  Deploys devices •  Registers and allocates devices to

the tenants •  Publishes service graphs

Device Package A Device Package B Device Package C

Managed Objects: •  Service graphs

•  Device and service configuration

Device A Device B Device C Device C Device A Device A

Provider Network Administrator

APIC


•  Integrated gateway for VLAN, VxLAN, and NVGRE networks from virtual to physical

•  Normalization for NVGRE, VXLAN, and VLAN networks

•  Customer not restricted by a choice of hypervisor

•  Fabric is ready for multi-hypervisor

Virtual Integration Network Admin

Application Admin

PHYSICAL SERVER

VLAN VXLAN

VLAN NVGRE

VLAN VXLAN

VLAN

ESX Hyper-V KVM


ACI Fabric

APIC

APIC

VMware Microsoft

Red Hat XenServer

VMware Microsoft Red Hat


•  Network policy coordination with virtualization managers

•  Automatic virtual endpoint detection and policy placement

•  Policies consistently implemented in virtual and physical

•  Network policy stays sticky with VM

Virtual Integration Hypervisor

Management

Web App DB

Application Profile

Network Policy Coordination

Web App DB

VM Attach/Detach

Notification PortGroup

VM Mobility Notification

PortGroups VM Networks

APIC

APIC VMware Microsoft Red Hat

XenServer

VMware Microsoft


The Fabric normalizes VLAN’s which allows re-use and efficient communication across VMM Domains

VXLAN is not required to address the 4K VLAN limitations (VXLAN ‘is’ supported if desired)

An EPG can be spread across multiple VMM Domains (common policy across Domains)

VMM Domain 1

VMM Domain 1

Hosts

vCenter

vShield

Web EPG App EPG

VM VM VM VM

VMM Domain 2

VMM Domain 1 4000 EPGs

Hosts

vCenter

vShield

DB EPG App EPG

VM VM VM VM VM


Key Takeaway


ACI Fabric



Services for ACI


•  Unified point of data center network automation and management:   Application-centric network policies

  Data model-based declarative provisioning

  Application, topology monitoring, and troubleshooting

  Third-party integration (Layer 4 - 7 services, storage, compute, WAN, etc.)

  Image management (Spine/Leaf)

  Fabric inventory

•  Single APIC cluster supports one million+ endpoints, 200,000+ ports, 64,000+ tenants

•  Centralized access to all fabric information - GUI, CLI, and RESTful APIs

•  Extensible to compute and storage management

Layer 4..7 System Management

Storage Management

Orchestration Management

Storage SME Server SME Network SME

Security SME App. SME OS SME

Open RESTful API

Policy-Based Provisioning

APIC

Citrix

Cisco F5 EMC

Corporation

NetApp Puppet Labs

Opscode Python

CFEngine Microsoft XenServer

CloudStack

OpenStack VMware Red Hat

KVM


•  Applications fully use clustered and replicated controller (N+1, N+2, etc.)

•  Any node is able to service any user for any operation

•  Seamless APIC node adds and deletes

•  Fully automated APIC software cluster upgrade with redundancy during upgrade

•  Cluster size driven by transaction rate requirements

•  APIC is not in the data path

Single Point of Management Without a Single Point of Failure

See What’s Inside

APIC Cluster Distributed, Synchronized, Replicated

APIC


•  ACI Fabric supports discovery, boot, inventory, and systems maintenance processes through the APIC ‒  Fabric discovery and addressing

‒  Image management

‒  Topology validation through wiring diagram and systems checks

APIC Cluster

Topology discovery through LLDP using ACI-specific TLVs (ACI OUI)

Loopback and VTEP IP addresses allocated from “infra VRF” through DHCP from APIC

APIC APIC APIC


Key Takeaway


ACI Fabric



Services for ACI


•  Application-centric definition of network services – decoupling of profile from actual implementation

•  Policy-driven infrastructure and service management

•  Scalable (endpoints, policies, tenants, applications)

•  Consistent model for physical, virtual, and cloud

•  Flexibility of software, combined with hardware performance

•  Extensible model that can be used by partners and other vendors across the network, compute, and storage space


Designed from Its Foundation to Be Application-Centric

Application/Workload Orchestration and Scheduler

Unified Information Model and API

Policy Controller Compute Policy Controller Storage Policy Controller Network Fabric

Endpoint Group (EPG)

Endpoint Group (EPG) Application Graph (EP, EPG, graph

edges)

Application Profile Compute Service Profile Network Profile Storage Service Profile = + +

Thank you.

Technology

Cisco live local high level aci