Cisco Data Center Network Architecture

1© 2005 Cisco Systems, Inc. All rights reserved. Cisco PublicBusiness Ready Data CenterArchitecture

Cisco Data Center Network Architecture

Ivan S ChandraSystems Engineer [email protected]

2© 2005 Cisco Systems, Inc. All rights reserved.Business Ready Data CenterArchitecture Cisco Public

Agenda

• Data Center Challenges

• Cisco Data Center Network Architecture

• Summary

• Data Center Challenges


• Summary


Data Center Architecture Today—Dedicated Infrastructure

• Low utilization of compute and storage resources

• Multiple points of management

• Inconsistent security policies

• Too costly to scale

APPLICATION 1...N

USERSChallenges

DEDICATED COMPUTING

DEDICATED STORAGE

Reference: Cisco Internet BusinessSolutions Group, 2004

DEDICATED NETWORK


Data Center Architecture Evolving—Dedicated Infrastructure to Service Oriented Infrastructure

VIRTUALIZATIONVIRTUALIZATION

• Much better utilization

• Lower administration costs

• Reduction in assets

• Increased service availability

• Faster deploymentof new services

Benefits

Reference: Cisco Internet BusinessSolutions Group, 2004

INTELLIGENT INFORMATION NETWORK

APPLICATION 1...N

USERS

POOLED COMPUTING

POOLED STORAGE


Evolution of the Data Center InfrastructurePhased Approach

AUTOMATION

Storage

Network

Compute

Dynamic Provisioning and Information Lifecycle

Management (ILM) to Enable Business Agility

Business PoliciesOn-Demand

Service OrientedVIRTUALIZATION

StorageNetworkCompute

EnterpriseApplications

Management of Resources Independent of Underlying Physical Infrastructure to

Increase Utilization, Efficiency and Flexibility

Data Network

Server Fabric

Network

Centralization and Standardization to

Lower Costs, Improve Efficiency and Uptime

CONSOLIDATION

LANWANMAN

SAN

Storage Network

Intelligent Information

Network

HPCClusterGRID


Agenda

• Data Center Challenges and Trends


• Summary



• Summary


InstantInstantMessagingMessaging

UnifiedUnifiedMessagingMessaging

MeetingMeetingPlacePlace

IPCCIPCC IP PhoneIP Phone VideoVideoDeliveryDelivery

PLMPLM CRMCRM ERPERP

HCMHCM ProcurementProcurement SCMSCM

CollaborationCollaborationApplicationsApplications

Traditional Architecture / Service Oriented ArchitectureTraditional Architecture / Service Oriented Architecture

BusinessBusinessApplicationsApplications

NE

TW

OR

KE

DN

ET

WO

RK

ED

INF

RA

ST

RU

CT

UR

EIN

FR

AS

TR

UC

TU

RE

LA

YE

RL

AY

ER

Compute NetworkCompute NetworkCompute NetworkCompute Network

Storage NetworkStorage NetworkStorage NetworkStorage Network

ServerServer Fabric Fabric

ServerServerSwitchingSwitching

Storage Storage SwitchingSwitching

Data Center Data Center InterconnectInterconnect

MDS FamilySFS Family Catalyst Family ONS Family

DirectorFabric

ModularRackBlade

InfinibandSwitching

DWDM, SONET, SDH, FCIP

ServerServer

Network Virtualization ServicesNetwork Virtualization Services

StorageStorage ClientsClients

Data Data CenterCenter

Data Data CenterCenterBranchBranch

BranchBranchCampusCampus

CampusCampusTeleworkerTeleworker

TeleworkerTeleworkerWAN/MANWAN/MAN

WAN/MANWAN/MANEnterprise Enterprise EdgeEdge

Enterprise Enterprise EdgeEdge Places in the NetworkPlaces in the Network

Cisco Data Center Network Architecture Framework

INT

ER

AC

TIV

EIN

TE

RA

CT

IVE

SE

RV

ICE

SS

ER

VIC

ES

LA

YE

RL

AY

ER

Infrastructure ManagementInfrastructure Management

Se

rvic

es

Ma

na

ge

me

nt

Se

rvic

es

Ma

na

ge

me

nt

Advanced Analytics and Decision SupportAdvanced Analytics and Decision Support

Infrastructure Enhancing Services Infrastructure Enhancing Services

Compute ServicesCompute ServicesCompute ServicesCompute Services

Storage Fabric ServicesStorage Fabric ServicesStorage Fabric ServicesStorage Fabric Services

Security ServicesSecurity ServicesSecurity ServicesSecurity Services

Application Networking ServicesApplication Networking Services

Virtualization, Replication, Virtual Fabrics

Virtualization, Replication, Virtual FabricsRDMA,

Low Latency Clustering

RDMA, Low Latency Clustering

Firewalls, Intrusion Protection, Security Agents

Firewalls, Intrusion Protection, Security Agents

Application Delivery ServicesApplication Delivery ServicesApplication Delivery ServicesApplication Delivery ServicesWAAS, App Acceleration, WAAS, App Acceleration,

Optimization, Security and Server OffloadOptimization, Security and Server Offload

WAAS, App Acceleration, WAAS, App Acceleration, Optimization, Security and Server OffloadOptimization, Security and Server Offload

Ad

ap

tiv

e M

an

ag

em

en

t A

da

pti

ve

Ma

na

ge

me

nt

Se

rvic

es

Se

rvic

es

Network Infrastructure VirtualizationNetwork Infrastructure Virtualization

Ser

vice

sS

ervi

ces

Vir

tual

izat

ion

Vir

tual

izat

ion

Application Networking ServicesApplication Networking Services

Infrastructure Enhancing ServicesInfrastructure Enhancing Services


Data Center ServicesWhere?

DATA CENTER INTERCONNECT

SONET/SDHxWDM

Metro EthernetFCIP

SFS 7000

MDS 9500

AVS WAAS

Firewall ServicesDDOS Guard

Intrusion Prevention

EMBEDDED SECURITY SERVICESEMBEDDED SECURITY SERVICES

Secure Virtual Fabrics

STORAGE SWITCHING

High Performance Compute (HPC) Clusters Catalyst

ONS 15000

InternetInternetMPLS VPNMPLS VPNIPSEC/SSL VPNIPSEC/SSL VPN

Storage & Tape Arrays

Blade Servers UNIX/NT Servers Mainframes

EMBEDDED APPLICATION NETWORK SERVICESEMBEDDED APPLICATION NETWORK SERVICES

ServerLoad Balancing

SSL Off-load ApplicationMessage Services

Application Control Engine

EMPLOYEE / PARTNER / CUSTOMER ACCESS NETWORK

SERVER SWITCHING

Enterprise Applications

Fibre Channel

FICON

Infiniband

GE / 10GE

Management and ProvisioningFramework

Fabric AssistedApplications

Data ReplicationServices

StorageVirtualization

EMBEDDED STORAGE SERVICESEMBEDDED STORAGE SERVICES

Fabric Hosted Applications

SERVER FABRIC


DC Network Topology

Layers

Server Virtualization V

Remote DMA Services

Virtual I/O

Clustering Services

Compute Fabric Services

Fabric Gateway Services

Fabric Routing Services

Data Replication Svcs

Storage Virtualization

Virtual Fabrics (VSANs)


Content Caching

SSL Offloading

Firewall Services

Intrusion Detection

Server Balancing

Network Analysis

VPN Termination

File Caching

DOS Protection

Server Farms Server Clusters

Storage/Tape Farms

Core

Aggregation

Access

Edge

Core


DC Server Network


Remote DMA Services

Virtual I/O

Clustering Services








Content Caching

SSL Offloading

Firewall Services

Intrusion Detection

Server Balancing

Network Analysis

VPN Termination

File Caching

DOS Protection


Storage/Tape Farms

Edge

Core

Layers

Core

Aggregation

Access


DC Access LayerLayer 2, Layer 3 Server & Mainframe Connectivity

• L2 and L3 requirements

• Dual and single attached

• High performance, low latency L2 switching

• Mix of oversubscription requirements

• Many uplink options

• STP processing for configured VLANs only

• Utilizes services in the Aggregation Layer

L3 Access

Blade Chassis w/integrated

switchBlade Chassis

w/pass thru

Mainframe w/OSA

L2 w/clustering & NIC teaming

Enterprise Core

DC Aggregation

DC Core

DC Access


DC Aggregation LayerProviding Critical Point for Control and Application Services

• Aggregates traffic to DC core

• Large STP Processing load

• Aggregates advanced application and security functions

• Maintains connection and session state for redundancy

• Layer 4-7 services: FW, SLB, SSL, IDS

• High flexibility and Economies of Scale

L3 Access

Blade Chassis w/integrated

switchBlade Chassis

w/pass thru

Mainframe w/OSA

L2 w/clustering & NIC teaming

Enterprise Core

DC Access

Service Modules


DC Aggregation LayerServer to Server Communications Path

•The Aggregation module may provide the primary communication path for server to server traffic

•Non traditional traffic emerging

•Driving lower oversubscription and 10GE uplinks

•Servers now ship with PCI-X NIC’s and GE

•Plan bandwidth for future server true capacity

Access

What types of server to server traffic will exist?

Multi-tier interaction,Backup,Replication,

Cluster Messaging, storage over ip

Aggregation

DC Core


DC Core LayerHigh speed switching fabric for Aggregation Modules

• Interconnects AGG Modules

• Isolates failure domains

• Scales large STP diameters

• Improves 10GE scaling

• Plan & build DC core up front

N x 100 Servers N x 100 Servers

Agg Module1 Agg Module2

DC Core

Aggregation

GE/10GE GE/10GE

Access

Enterprise Core


Service Integration and VirtualizationEvolving towards Virtual Network Services

Dedicated Shared Virtualized

Cust1

Cust1

Cust2

Cust2

CustN

CustN

…Cust1

Cust1

High CapEx & OpEx

Cust2

Cust2

Cust1

Cust1

Cust2

Cust2

CustN

CustN

…

Concerns for privacy & security

Physical Resource Service context

Quasi Virtualized

Cust1

Cust1

Cust2

Cust2

CustN

CustN

…

Virtual Network Service


DC Consolidated InfrastructureIntegration & Virtualization

Core


ContentCaching

StatefulFirewalls

HighDensity

MultilayerLAN

Switch

Core


ContentCaching

StatefulFirewalls

HighDensity

MultilayerLAN

Switch

IPS farm


DC Storage Network


Remote DMA Services

Virtual I/O

Clustering Services








Content Caching

SSL Offloading

Firewall Services

Intrusion Detection

Server Balancing

Network Analysis

VPN Termination

File Caching

DOS Protection


Storage/Tape Farms

Core

Aggregation

Access

Layers

Edge

Core


Direct Attach Storage

• Direct Attached Storage (DAS)

• Storage is captive ‘behind’ the server, limited mobility

• Limited scalability due to limited devices

• No storage sharing possible

• Costly to scale; complex to manage

FC

Clients

Direct Attached Storage

Application Servers

Win2k Linux Unix Unix

Tape

FC

LinuxWin2k

SCSI

LAN


Storage Area Network

• Storage is accessed at a block-level via SCSI protocol

• High performance interconnect providing high I/O throughput

• Lower TCO relative to direct attached storage, storage can be shared

• Limited vendor interoperability

• Complex management

Separation of Storage from the Server

Storage Area Network (SAN)

DatabaseServers

BlockStorageDevices

Fibre Channel

SAN

Clients

LAN


Virtual SAN (VSAN)

A Virtual SAN (VSAN) provides a method to allocate ports within a physical fabric to create virtual fabrics

• Analogous to VLANs in Ethernet

• Virtual fabrics created from larger cost-effective redundant physical fabric

• Reduces wasted ports of island approach

• Fabric events are isolated per VSAN – maintains isolation for HA

• Hardware-based isolation - traffic is explicitly tagged across inter-switch links with VSAN membership info

• Statistics can be gathered per VSAN

Cisco MDS 9000Family with VSAN Service

Physical SAN islands are virtualized onto

common SAN infrastructure


DC Interconnect

Data Center Core

Aggregation

Access

Core

Access

Servers

Storage

SONET/SDHNetwork

DWDMNetwork

Campus Core

IBM

Metro Ethernet

DC Interconnect

WAN

GE

IBM GDPS

1/2 Gb FC/FICON


Server Fabric

Edge

Core


Remote DMA Services

Virtual I/O

Clustering Services








Content Caching

SSL Offloading

Firewall Services

Intrusion Detection

Server Balancing

Network Analysis

VPN Termination

File Caching

DOS Protection


Storage/Tape Farms

Layers

Core

Aggregation

Access


Server SwitchesRequirements being addressed

Virtualization Virtualization (I/O, Storage, (I/O, Storage, andand CPU) CPU)

Shared Resources Across Entire ClusterRouting, Aggregation, Load BalancingApp/OS to CPU provisioning

High Performance High Performance Server-to-Server Server-to-Server

InterconnectInterconnect

RDMAHigh Bandwidth Low LatencyInfiniBand today; PCI-Express and /or 10GigE when ready

Policy-Based Policy-Based Dynamic Dynamic Resource Resource MappingMapping

Performance Performance andand Control Control


Server Switch ApplicationsWhy Are Performance and Control Important?

Server Clustering

High Performance Computing (HPC)“Enterprise-Class” HPC Database Scalability

Utility or Grid Computing

Application ProvisioningServer Re-purposingServer Migration

Applications

I/O Virtualization

I/O ConsolidationI/O AggregationServer Consolidation


Today’s Enterprise Service ProvisioningA Scale-Out Example

SysAdmin racks new serverLoads O/S and Applications

NetOps connects Ethernet cabling, configures VLAN/Port Config

SLB Admin Adds Server to Pool

SecOps checks security policy, expands FW Port Range

NetOps ensures Branch connectivity/ Routable Subnet

StorageOps configures LUN, maps to Server

StorageOps provisions disk volume and resources

Assume you just want to add one server to a web-farm…

The challenge is one of ‘coordination delays’. This type of simple scale-out of an existing serve often takes enterprises 90-days.

New service turn-ups, after the application has been developed, often take 180+ days.

VFrame is designed to eliminate these delays and automate the provisioning of services


CSM Load Balancer

Servers

VFrame identifies right App / OS ImageFrom storage

VFrame translates policies to actions

and passes to infrastructure

Data Center AutomationVframe Data Center 1.0

Catalyst 6500

SAN

FWSM Firewall

Administrator

MDS 9500

Campus/ WAN/VPN

Data Center

Policy

Application: SAP

Performance

Security

Availability

Image

Accounting

Define application services and pass policy to VFrame

VFrame™

VFrame picks server with right criteria to run application and

boots server

VFrame gives new server right VLAN and

LUN info so it can find/be found by right

clients and storage

VFrame provisions security policies to

FWSM

VFrame provisions CSM to add new server to load balancing poolApplication Service Provisioned!

NAS


Agenda



• Summary



• Summary


Data Center Networking Action Plan

• Decide on the end-state data center:What should the data center be infive years?

• Identify main immediate challengesand initiatives:

Consolidation, business continuance, virtualization, on-demand, etc.

• Develop data center networking strategy: Data center and network stakeholders engage

Supports data center short- to long-term goals and initiatives

• Engage with Cisco and partners:Plan, design, deploy, implement, operateand optimize


Documents

Cisco Data Center Network Architecture