Data Center: Infrastructure Architecture SRNDSolutions Reference Network DesignMarch, 2004
Corporate HeadquartersCisco Systems, Inc.170 West Tasman DriveSan Jose, CA 95134-1706 USAhttp://www.cisco.comTel: 408 526-4000
800 553-NETS (6387)Fax: 408 526-4100
Customer Order Number: 956513
THE SPECIFICATIONS AND INFORMATION REGARDING THE PRODUCTS IN THIS MANUAL ARE SUBJECT TO CHANGE WITHOUT NOTICE. ALL STATEMENTS, INFORMATION, AND RECOMMENDATIONS IN THIS MANUAL ARE BELIEVED TO BE ACCURATE BUT ARE PRESENTED WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED. USERS MUST TAKE FULL RESPONSIBILITY FOR THEIR APPLICATION OF ANY PRODUCTS.
THE SOFTWARE LICENSE AND LIMITED WARRANTY FOR THE ACCOMPANYING PRODUCT ARE SET FORTH IN THE INFORMATION PACKET THAT SHIPPED WITH THE PRODUCT AND ARE INCORPORATED HEREIN BY THIS REFERENCE. IF YOU ARE UNABLE TO LOCATE THE SOFTWARE LICENSE OR LIMITED WARRANTY, CONTACT YOUR CISCO REPRESENTATIVE FOR A COPY.
The Cisco implementation of TCP header compression is an adaptation of a program developed by the University of California, Berkeley (UCB) as part of UCBs public domain version of the UNIX operating system. All rights reserved. Copyright 1981, Regents of the University of California.
NOTWITHSTANDING ANY OTHER WARRANTY HEREIN, ALL DOCUMENT FILES AND SOFTWARE OF THESE SUPPLIERS ARE PROVIDED AS IS WITH ALL FAULTS. CISCO AND THE ABOVE-NAMED SUPPLIERS DISCLAIM ALL WARRANTIES, EXPRESSED OR IMPLIED, INCLUDING, WITHOUT LIMITATION, THOSE OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT OR ARISING FROM A COURSE OF DEALING, USAGE, OR TRADE PRACTICE.
IN NO EVENT SHALL CISCO OR ITS SUPPLIERS BE LIABLE FOR ANY INDIRECT, SPECIAL, CONSEQUENTIAL, OR INCIDENTAL DAMAGES, INCLUDING, WITHOUT LIMITATION, LOST PROFITS OR LOSS OR DAMAGE TO DATA ARISING OUT OF THE USE OR INABILITY TO USE THIS MANUAL, EVEN IF CISCO OR ITS SUPPLIERS HAVE BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
Data Cemter Networking: Enterprise Distributed Data CentersCopyright 2004, Cisco Systems, Inc.All rights reserved.
CCIP, CCSP, the Cisco Arrow logo, the Cisco Powered Network mark, Cisco Unity, Follow Me Browsing, FormShare, and StackWise are trademarks of Cisco Systems, Inc.; Changing the Way We Work, Live, Play, and Learn, and iQuick Study are service marks of Cisco Systems, Inc.; and Aironet, ASIST, BPX, Catalyst, CCDA, CCDP, CCIE, CCNA, CCNP, Cisco, the Cisco Certified Internetwork Expert logo, Cisco IOS, the Cisco IOS logo, Cisco Press, Cisco Systems, Cisco Systems Capital, the Cisco Systems logo, Empowering the Internet Generation, Enterprise/Solver, EtherChannel, EtherSwitch, Fast Step, GigaStack, Internet Quotient, IOS, IP/TV, iQ Expertise, the iQ logo, iQ Net Readiness Scorecard, LightStream, MGX, MICA, the Networkers logo, Networking Academy, Network Registrar, Packet, PIX, Post-Routing, Pre-Routing, RateMUX, Registrar, ScriptShare, SlideCast, SMARTnet, StrataView Plus, Stratm, SwitchProbe, TeleRouter, The Fastest Way to Increase Your Internet Quotient, TransPath, and VCO are registered trademarks of Cisco Systems, Inc. and/or its affiliates in the U.S. and certain other countries.
All other trademarks mentioned in this document or Web site are the property of their respective owners. The use of the word partner does not imply a partnership relationship between Cisco and any other company. (0304R)
C O N T E N T S
Document Purpose vii
Document Organization viii
Obtaining Documentation viiiWorld Wide Web viiiDocumentation CD-ROM viiiOrdering Documentation ixDocumentation Feedback ix
Obtaining Technical Assistance ixCisco.com ixTechnical Assistance Center x
Cisco TAC Web Site xCisco TAC Escalation Center xi
C H A P T E R 1 Data Center Infrastructure Architecture 1-1
Data Center Architecture 1-1
Hardware and Software Recommendations 1-3Aggregation Switches 1-3Service Appliances 1-5Service Modules 1-5Access Switches 1-6Software Recommendations 1-8
Data Center Multi-Layer Design 1-9Core Layer 1-9Aggregation and Access Layer 1-10Service Switches 1-10Server Farm Availability 1-11Load-Balanced Servers 1-12
Data Center Protocols and Features 1-15Layer 2 Protocols 1-15Layer 3 Protocols 1-16Security in the Data Center 1-18
Scaling Bandwidth 1-18
Network Management 1-19
iiiData Center: Infrastructure Architecture SRND
C H A P T E R 2 Data Center Infrastructure Design 2-1
Routing Between the Data Center and the Core 2-1Layer 3 Data Center Design 2-1Using OSPF 2-3Using EIGRP 2-7Designing Layer 3 Security 2-8
Switching Architecture for the Server Farm 2-9Using Redundant Supervisors 2-9Layer 2 Data Center Design 2-10
Using Three-Tier and Two-Tier Network Designs 2-10Layer 2 and Layer 3 Access Design 2-11Using VLANs to Segregate Server Farms 2-12VLAN Scalability 2-13Using Virtual Trunking Protocol 2-14Choosing a Spanning-Tree Algorithm 2-14Using Loopguard and UDLD 2-15Using PortFast and TrunkFast 2-17Using a Loop-Free Topology 2-18Designing Layer 2 Security 2-19
Assigning the Default Gateway in the Data Center 2-21Using Gateway Redundancy Protocols 2-22Tuning the ARP Table 2-23
C H A P T E R 3 HA Connectivity for Servers and Mainframes: NIC Teaming and OSA/OSPF Design 3-1
Overview 3-1Ensuring Server Farm and Mainframe Availability 3-2Load Balanced Servers 3-4NIC Teaming 3-4Mainframe Sysplex 3-6
NIC Teaming Architecture Details 3-7Hardware and Software 3-8Deployment Modes 3-8
Fault Tolerance Modes 3-8Load Balancing Modes 3-12Link Aggregation Modes 3-13Layer 3 Multihoming 3-14
Interoperability with Security 3-16
ivData Center: Infrastructure Architecture SRND
Intrusion Detection 3-17Port Security 3-17Private VLANs 3-19
Mainframe OSA and OSPF Architecture Details 3-20Overview 3-20Attachment Options 3-21IP Addressing 3-22OSPF Routing on a Mainframe 3-23Sysplex 3-24
Configuration Details 3-26Speed and Duplex Settings 3-27Layer 2 Implementation 3-27
Spanning Tree 3-27PortFast and BPDU Guard 3-28Port Security 3-29Server Port Configuration 3-29
C H A P T E R 4 Data Center Infrastructure Configuration 4-1
Configuring Network Management 4-1Username and Passwords 4-1VTY Access 4-2SNMP 4-3Logging 4-3
VLAN Configuration 4-3
Spanning Tree Configuration 4-6Rapid PVST+ 4-6MST 4-7Protection From Loops 4-7
VLAN Interfaces and HSRP 4-8
Switch-To-Switch Connections Configuration 4-9Channel Configuration 4-9Trunk Configuration 4-10
Server Port Configuration 4-12Speed and Duplex Settings 4-12PortFast and BPDU Guard 4-13Port Security 4-13Configuration Example 4-14
Sample Configurations 4-14Aggregation1 4-14
vData Center: Infrastructure Architecture SRND
Aggregation2 4-18Access 4-21
G L O S S A R Y
I N D E X
viData Center: Infrastructure Architecture SRND
This publication provides solution guidelines for enterprises implementing Data Centers with Cisco devices. The intended audiences for this design guide include network architects, network managers, and others concerned with the implementation of secure Data Center solutions, including:
Cisco sales and support engineers
Document PurposeThe convergence of voice and video in todays enterprise networks has placed additional requirements on the infrastructure of enterprise data centers, which must provide the following services:
Hosting enterprise-wide servers
Supporting critical application services
Supporting traditional data services
These requirements are based on the applications supported rather than the size of the data center. The process of selecting the proper data center hardware and software versions that meet the necessary Layer 2, Layer 3, QoS, and Multicast requirements can be a daunting task. This solutions reference network design (SRND) provides design and implementation guidelines for building a redundant, scalable enterprise data center. These guidelines cover the following areas:
Data center infrastructure and server farm design
Server farm design including high availability
Designing data centers for mainframe connectivity
Enhancing server-to-server communication
viiData Center: Infrastructure Architecture SRND
Document OrganizationThis document consists of the following chapters:
Obtaining DocumentationThe following sections explain how to obtain documentation from Cisco Systems.
World Wide WebYou can access the most current Cisco documentation on the World Wide Web at the following URL:
Translated documentation is available at the following URL:
Documentation CD-ROMCisco documentation and additional literature are available in a Cisco Documentation CD-ROM package, which is shipped with your product. The Documentation CD-ROM is updated monthly and may be more current than printed documentation. The CD-ROM package is available as a single unit or through an annual subscription.
Chapter 1, Data Center Infrastructure Architecture
Provides background information, including hardware recommendations for designing a data center infrastructure that is secure, scalable, and resilient.
Chapter 2, Data Center Infrastructure Design Describes design issu, including routing between the data center and the core, switching within the server farm
Chapter 3, HA Connectivity for