Upload
donna-richard
View
216
Download
2
Tags:
Embed Size (px)
Citation preview
Fast Path Data Center Relocation
Presented by:AFCOM & Independence Blue Cross
April 8, 2011
Agenda• Some Background to Understand Why• Initial Planning (After Fear Subsides)• Critical Path Components• Dark Fiber DWDM Network Issue Identified• Summary of Project Results• Keys to Success• Overview of DR Network Design
Background Events• Completed Relocation of Primary Data Center in September 2009• Completed Implementation & Data Center Infrastructure Build-out of Insourced
DR Capability in January 2010 at Secondary Site (Including Cancelation of Outsource Contract)
• In Q1 2010: Learned the Location Supporting the DR Data Center (580) Could be Phased Out in 2011
• Completed Relocation of Entire Test, QA, Development Server Infrastructure to Primary Data Center in Q2 2010
• Q2 2010: Started a Detail Data Center Site Alternative Study to Evaluate Options • By July 2010, Two Viable Options Had Been Identified:
– Build-out within an existing IBC real estate footprint – Build-out within the Directlink Technologies facility
• By August, 2010: Learned the 580 Phase Out Would Need to be a Fast Path with a Target Decommission in March 2011
Realization and Fear• Realization: We Have Less than 7 Months to:
– Select a Site and Either Complete Engineering Design or Negotiate a Deal
– Build Out the Data Center– Re-route the Dark Fiber Network– Plan and Execute the Relocation of the DR Data Center – Relocate the Operations Command Center which was Also Located
in the DR Facility• Initial Reaction:
Feasibility Assessment – Fear 101• Document Key Factors
– Long Lead Purchases – Contract Negotiations– Fiber Path Build-out– Data Center Build-out– New Command Center Build-outs (2)– Relocation Events Integrated into Master Release Calendar– Cycle Time for Purchase Order Approvals
• Obtain Executive Support– Agree to a Fast Path Approval Process– Agree to Master Calendar Adjustments
• Publish High Level Timeline• Parallel Sub-Projects for Critical Path Activities
580 Relocation Timeline - Optimal
8/2/2010 4/1/2011
9/1/2010 10/1/2010 11/1/2010 12/1/2010 1/1/2011 2/1/2011 3/1/2011
8/2/2010 - 8/31/2010Long-Lead Items 1/7/2011
Move 1
9/1/2010
Site SelectedDeal Sealed
8/27/2010
Equipment OrdersApproved
8/24/10
Submit EquipmentOrders for Approval
11/15/2010
Equipment Delivered
8/27/2010Order Equipment
8/2/2010 - 8/31/2010Detail Planning
10/31/2010
Start Site Network Buildout
1/28/2011Move 2
2/25/2011Move 3
12/15/2010
Network ActiveAt New Site
1/8/2011 - 2/26/2011580 Relocations
3/31/2011
Decommission580 Site10/31/2010
Contract Complete8/2/2010
Go Decision
8/2/2010
Start Site Selection Negotiations
High-Level Timeline
Site Selected: Directlink Technologies
7
Key Decision Factors: Facility infrastructure design that could serve as our production data center Our dark fiber vendor was already in the facility, and could be added as a node on
our ring with minimal physical build-out Negotiated a deal based on our requirements, not a “one model fits all” approach Enabled IBC to meet the aggressive timeline before contract officially signed (e.g.
work through the Attorney dance time)
Building 2
Critical Path – Parallel Activities 1. Facility build-out including space design and configuration changes:
Installation of power distribution, cooling capacity, overhead cable tray, and NOC area
2. Fiber network build-out to support migration plan to transition to the new data center location without outage
3. Acquisition and implementation of additional optical equipment to support the new fiber network node
4. Acquisition and implementation of “seed” network infrastructure equipment to allow new data center to be an active node prior to relocation events
5. Installation of “seed” server cabinets and structured cabling within the new data center space
6. Relocation strategy development and event detail planning7. Relocate the primary Operations Command Center to another location
(1901 Market St.) prior to completing data center relocation
Fiber Network Issue Identified
• One Inter-Data Center Fiber Path Much Longer– Invalidated Previous Inter Data Center Fiber
Channel Configuration – Required Conversion from Native FC to FC/IP for
Storage Replication and LSAN Capabilities• Resulting in (just what we needed)– Another Critical Path, Parallel Activity
Project Results Summary
• Command Center Reconfiguration– Primary Command Center Relocated from 580 to
IBC Headquarters at 1901 Market Street– Active Secondary Command Center Built-Out in
the Directlink Facility
Project Results Summary• Data Center Preparation
– Approximately two miles of fiber cable– Approximately one mile of copper cable (reduced from over 20 miles of copper
installed at 580 site)– 1,200’ of ceiling mounted cable trays– 1,850 fiber terminations– 500 Ethernet connections– Temporary routings of backbone fiber to support the cutover to Reading– Installation of more than 40 new network devices supporting the fiber backbone
and core network– Rerouting telecommunication circuits supporting redundant connections for the
MPLS WAN and Extranet, Internet ISP, Bluesnet, IBC Sonet, ASP Services and Remote Access
– Installation of 100 equipment cabinets (some new, some as emptied)– Custom configuration of dual power utility feeds to 2 separate 750 kVA UPS systems– Installation of four 300 kVA PDUs – Installation of new 30 ton CRAH units– Installation of 185+ electrical circuits to provide dual A/B power to all IT equipment
Project Results Summary• Equipment Relocation
– 111 physical servers that support redundant production, redundant QA and standby DR
– 126 virtual servers supporting redundant production, redundant QA and standby DR– 39 Information Security appliances and network infrastructure devices all
redundant to Bretz configurations– z10 Disaster Recovery Mainframe – Mainframe disk configurations hosting replication of data from primary site– Mainframe virtual tape system and back-end ATL– Affiliate AS/400 Disaster Recovery configuration including mirrored storage and tape
subsystem– Informatics-Teradata configuration supporting QA-Test-Dev and DR for production
environment– A 16-cabinet AS/400 configuration, including mirrored storage and ATL supporting
QA-Test-Dev and DR for production environment– Distributed Systems SAN storage configuration supporting both active HA systems as
well as production data replication– Distributed SAN Infrastructure and ATL– Fully engaged application test teams to verify system status post-move
Keys to Success• Partner Type Relationship with Directlink
– Handshake deal to start facility construction tasks– Willingness to work through configuration changes w/o red tape
• Confidence in Network Strategy– Strategy Successfully Used for Production DC Relocation– Experience with Target Design and Foundation Technology
• Experienced Team to Design & Build Data Center Layout and Supporting Cable Plant– Layout to Optimize Active Cable Plant and Facilitate Future Expansion – Diligence in Documenting. Labeling and Pre-Testing All Connections
• Leverage Inventory Configuration Management and Relocation Experience– Use of Existing Inventory Management Repository Containing Key Equipment
Specifications Including Power and Connection Details – Development of Detailed Move Event Scripts and Supporting Staffing Plans
• Strong Project Management Oversight Managing All Critical Path Activities
Disaster Recovery Network
• Configuration Flexibility To Provide– The Ability to Safely Test DR Capabilities in an
Isolated DR Network– Fast, Seamless Integration Into the Production
Network in the Event of a Primary Site Failure