Unified Technology & Design · Myths & Misunderstandings Putting It All Together. ... Massively scalable server access or mid- market aggregation Available Oct. 2010 Industry’s

Darryl McKinnon

Business Development Manager, Unified Fabric, APAC

Unified Technology & Design

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialPresentation_ID 2

Application NetworkingSwitching Management Compute

Open /

Standards

App.

Performance

Energy

EfficiencySecurity Continuity

Workload

Mobility

CloudAutomationVirtualizationConsolidation

Partner Ecosystem

Efficient Agile Transformative

Security OSStorage

Unified

Network Services

Unified

ComputingUnified

Fabric

New Bus. Models,

Governance and Risk

Driving Profitability

New Service Creation and

Revenue Generation

Cisco

Lifecycle

Services

Policy

TECHNOLOGY

INNOVATION

BUSINESS

VALUE

SOLUTION

DIFFERENTIATION

SYSTEMS

EXCELLENCE

Data Center Business Advantage At the Heart of Business Innovation


Agenda

Unified Fabric : The Big Picture

I/O Consolidation w/ FCoE

Myths & Misunderstandings

Putting It All Together


Cisco Unified Fabric Continued Architectural Innovation

FY08

FY10

CONVERGENCE

SCALE

INTELLIGENCE

Fabric Path

OTV

FEX-link

VN-Link

DCB/FCoE

vPC

VDC

Architectural Flexibility

Workload Mobility

Simplified Management

VM-Aware Networking

Consolidated I/O

Active-Active Uplinks

Virtualizes the Switch

Deployment FlexibilityUnified Ports


Unifying the Data Center FabricMany Networks, One Infrastructure

UnifiedFabric

Complexity, Cost, Power

Simpler Operations,Increased Efficiency

Management & Control

Primary Network

SecondaryNetwork

SAN

HPC

LAN


Building a Better Network

Ubiquitous

ReliableCost-

Effective

Mature

Fibre

ChannelEthernet

Converged 10 GE

LAN

SAN


Priority Flow Control IEEE 802.1Qbb creates lossless Ethernet with classes of service

Bandwidth Management IEEE 802.1Qaz allows flexible bandwidth sharing for LAN and SAN

Data Center Bridging Exchange Protocol IEEE 802.1Qaz provides device-device communication on resources

FCoE IEEE DCB

Fibre Channel Over EthernetHow It Works

• Encapsulation of FibreChannel frames over Ethernet

• Enables Fibre Channel to run on a lossless Ethernet

Fibre Channel

Traffic

Ethernet


Standards for FCoE

FCoE is fully defined in FC-BB-5 standard including Multi-Hop

FCoE works alongside additional technologies to make I/O Consolidation a reality

T11 IEEE 802.1FCoE

FC on

other

network

media

FC on

Other

Network

Media

FC-BB-5

PFC ETS DCBX

802.1Qbb

DCB

802.1Qaz 802.1Qaz

Lossless

Ethernet

Priority

Grouping

Configuration

Veritifcation


How FCoE Fits InPFC: Priority Flow Control

VLAN Tag enables 8 priorities for Ethernet traffic

PFC enables Flow Control on a Per-Priority basis

Therefore, we have the ability to have lossless and lossy priorities at the same time on the same wire

Allows FCoE to operate over a lossless priority independent of other priorities

9

Ethernet Wire

FCoE


Allows you to create priority groups

Can guarantee bandwidth

Can assign bandwidth percentages to groups

Not all priorities need to be used or in groups

Ethernet Wire

FCoE

20%80% 80%20%

How FCoE Fits InETS: Enhanced Transmission Selection


Allows network devices to advertise their identities and capabilities over the network

Enables hosts to pick up proper configuration from the network

Enables switches to verify proper configuration

Provides support for:PFC

ETS

Applications (e.g., FCoE)Ethernet Wire

―Hello?‖―Hello?‖

―Hello?‖―Hello.‖

―Hello.‖

―Hello.‖

―I’ll show you

mine if you

show me

yours.‖

How FCoE Fits InDCBX: Data Center Bridging Exchange


All Standards for FCoE Are Technically StableStatus of the Standards

PFC

ETS

DCBX

Inv Dev Appr Pub

Technically Stable

FC-BB-5

Inv Dev Appr Pub

Inv Dev Appr Pub

Inv Dev Appr Pub

Technically stable in Oct, 2008

Completed in June 2009

Published in May, 2010

Completed in July 2010, awaiting publication

Completed in July 2010 (completing Approval Phase 3)

Completed in July 2010 (completing Approval Phase 3)


Key Takeaways

So in…SummaryAll Standards Relevant to FCoE Are Complete

The problem of putting Fibre Channel frames on Ethernet was standardized in FC-BB-5

The problem of multi-hop FCoE was standardized in FC-BB-5

The problem of making Ethernet lossless was solved with PFC

Therefore, FCoE standards are done. Baked. Completed. Ready-to-Rock-and-Roll

Conversations about how to solve additional problems are ongoing

Expect more features to be available over time

We are targeting readiness in Q2’11

13


Complete data center class switching portfolio

Consistent data center operating system across all platforms

Infrastructure scalability, transport flexibility and operational manageability

NX-OS Data Center Operating System

Data Center Network Manager (DCNM)

Nexus 2000Nexus 4000

Nexus 1000V

Nexus 7000 MDS 9000

Nexus 5000/5500

The Cisco Unified Fabric Family


Cisco Nexus 5500 SeriesUnified Ports

Multi-protocol Ethernet (1/10 GbE) + Storage (FC, FCoE, iSCSI, NAS)

Multi-Layer and Highly Scalable 48 & 96 port models in 1RU & 2RU

FEX-link - Over 900 100 M/1 GbE & 600 10 GbE ports

FabricPath & Layer 2 /Layer 3 Capable

Multi-purpose Traditional Ethernet, virtualized, unified, HFT/HPC pods Massively scalable server access or mid- market aggregation

AvailableOct. 2010

Industry’s Highest Density & Performance for Fixed Switches


Unified Fabric DesignFCoE Design objectives

To expand I/O consolidation

Blade Server environments (e.g. Nexus 4000 / UCS)

Core and backbone links and devices

Preserve SAN design best practices

Collapsed-core, core-edge or edge-core-edge designs

Oversubscription, Fan-in ratios, hop count practices honored

Preserve SAN and LAN management models

Deterministic management of FC flows through all devices

No opaque ‘LAN’ clouds transporting SAN traffic

SAN scalability

Build-up the edge, from 20% attach-rate up to 100%

Allow LAN and SAN to scale independently


Unified Fabric DesignUnified Edge

The first phase of the Unified Fabric evolution design focused on the fabric edge

Unified the LAN Access and the SAN Edge by using FCoE

Consolidated Adapters, Cabling and Switching at the first hop in the fabrics

The Unified Edge supports multiple LAN and SAN topology options

Virtualized Data Center LAN designs

Fibre Channel edge with direct attached initiators and targets

Fibre Channel edge-core and edge-core-edge designs

Fibre Channel NPV edge designs

The Unified Edge

Fabric A Fabric BLAN Fabric

FC

FCoE

FC

Nexus 5000

FCF – NPV Mode

Nexus 5000

FCF – Switch Mode


Fibre Channel

DriversEthernet

Drivers

Operating System

PCIe

Eth

ern

et

Fib

re C

han

nel

10

Gb

E

10G

bE

Link

Unified Fabric DesignUnified Edge

Converged Network Adapter (CNA) presents two PCI address to the Operating System (OS)

OS loads two unique sets of drivers and manages two unique application topologies

Server participates in both topologies since it has two stacks and thus two views of the same ‘unified wire’

SAN Multi-Pathing provides failover between two fabrics (SAN ‘A’ and SAN ‘B’)

NIC Teaming provides failover within the same fabric (VLAN)

Ethernet Driver

bound to

Ethernet NIC PCI

address

FC Driver

bound to FC

HBA PCI

address

Unified Wire

shared by both

FC and IP

topologies

Nexus Unified

Edge supports

both FC and IP

topologies

Nexus Edge participates in

both distinct FC and IP Core

topologies

Nexus 5000

FCF-A

Nexus 5000

FCF-B


VLAN 10,30

VLAN 10,20

Unified Fabric DesignUnified Wires at the Edge

A VLAN is dedicated for every VSAN in the fabric

The FCoE VLAN is discovered and signaled to the hosts using FIP

Trunking not required on the host driver – all FCoE frames are tagged by the CNA

FCoE VLANs must not be configured on Ethernet links that are not designated for FCoE

Maintains isolated edge switches for SAN ‘A’ and ‘B’ and separate LAN switches for NIC 1 and NIC 2 (standard NIC teaming)

! VLAN 20 is dedicated for VSAN 2 FCoE traffic

(config)# vlan 20

(config-vlan)# fcoe vsan 2

VSAN 2

STP Edge Trunk


Nexus 5000

FCF-A

Nexus 5000

FCF-B

VSAN 3


Optimal layer 2 LAN design often leverages Multi-Chassis Etherchannel (MCEC)

Nexus utilizes Virtual Port Channel (vPC) to enable MCEC either between switches or to 802.3ad attached servers

MCEC provides network based load sharing and redundancy without introducing layer 2 loops in the topology

MCEC results in diverging LAN and SAN high availability topologies

FC maintains separate SAN ‘A’ and SAN ‘B’ topologies

LAN utilizes a single logical topology Direct Attach vPC Topology

Unified Fabric DesignUnified Wires at the Edge with MCEC

MCEC

vPC Peers

vPC Peer Link


Nexus 5000

FCF-A

Nexus 5000

FCF-B


Migration to 10G FCoE in place of 4/8G FC links (Ethernet price per bit economics)

Edge switch running as FCF with VE ports connecting to FCF on Core switch

Must be careful of Domain ID creeping

FSPF forwarding for FCoE traffic is end-to-end

Hosts will log into the FCF which they are attached to (access FCF)

Storage devices will log into the FCF at the core/storage edge

FCF

FCF

FCoE Multi-TierNext Steps – FCoE uplinks to SAN Core

FCF

FCF

VE Ports

SAN BSAN A


FCoE Multi-TierFuture – Shared Core/Aggregation Devices

Different requirements for LAN and SAN network designs

Factors that will influence this use case

Port density

Operational roles and change management

Storage device types

Potentially viable for smaller environments

Larger environments will need dedicated FCoE ‘SAN’ devices providing target ports

Use connections to a SAN

Use a “storage” edge of other FCoE/DCB capable devices

Direct Attach

FCoE Targets

CORE

Multiple VDCs

FCoE SAN

LAN Agg

LAN Core

© 2010 Cisco and/or its affiliates. All rights reserved. Cisco ConfidentialPresentation_ID 23© 2010 Cisco Systems, Inc. All rights reserved.

Case Study: University of ArizonaCisco Nexus Family and MDS Deployment with FCoE

Nexus5000

10 GE CapableServers with CNAs

1 GE 10 GE

Cisco Nexus 7000/LAN Network

MDS SAN Network

Fibre Channel FCoE

―…Nexus helped us drive the kind of server density and server connectivity and throughput that we knew we would need when deploying these new systems at a dramatically reduced cost and complexity.‖

Derrick Masseth, University of Arizona, Senior Director for IT

Business and Network Challenges

More capacity supports higher traffic out of new Enterprise databases

Higher bandwidth needed due to increased virtualization (20 Virtual Machines per physical server)

Why Select Cisco Nexus® and Deploy Unified FabricLower TCO - Converged Network - a 50% cost savings vs. separated LAN/SAN infrastructure

Cost-effective Scalability - Consolidation to 10GbEconnections reduced need for cables, adaptors, switch ports, etc

Investment protection for existing native FC storage infrastructure while supporting end-to-end FCoE

No interruption of applications in-service software upgrades (ISSU)


The Realities of Unified Fabric

Management stays with the relevant team

Skills don’t change

RBAC—both teams still control their domains

Cross-functional teamwork increases

Network

Admin

Storage

Admin

VSANs

Zones

ACLs

VLANsArraysQoS


Different “Tribe” Perspectives

Unified Fabric will require additional communication

Fear of takeover by the network team or storage team

Fear of operational changes

Consider ―SLAM‖ team


Key Takeaways

FCoE standards are done.

Unified Fabric design involves preserving both LAN and SAN best practices.

LAN Virtualization (VLAN) and SAN Virtualization (VSAN) are key building blocks to designing Unified Fabrics.

Cisco’s Unified Fabric offerings and roadmap provide a solid platform for I/O consolidation and Virtualized Data Centers

Talk to Cisco about how to use this technology to solve problems


Additional Resources

Standards Sites

http://ieee.org

http://t11.org/fcoe

http://fcoe.com

Case Studieshttp://bit.ly/cisco_casestudies

Blog

http://blogs.cisco.com/datacenter

Book

I/O Consolidation in the

Data Center

Documents

Unified Technology & Design · Myths & Misunderstandings Putting It All Together. ... Massively scalable server access or mid- market aggregation Available Oct. 2010 Industry’s