Improving Data Centre Operational Management › documents › SqMileImproving... · 2009-05-19 · Improving Data Centre Operational Management Square Mile Systems Gordon Petrie

Improving Data Centre Operational Management

Square Mile SystemsGordon Petrie

Enabling Best Practice in IT Infrastructure Management!gordon.petrie@squaremilesystems.comwww.squaremilesystems.comTel +44 (0)870 950 4651Mob +44 (0) 7855 743209

© Square Mile Systems 1

House keeping

• Fire evacuation• Toilets• Refreshments• Mobile phones• Presentation material


What can I gain from today

• Industry knowledge and benchmarking• Common challenges and barriers• How can I enable my own “best practice”• Square Mile integration approach

– Solutions suited to complex environments– Time to deploy – Cost / value tradeoffs– Practical use by multiple teams


Square Mile OverviewUK based – Cirencester, Glos

Focus on applying asset & configuration management techniques to large infrastructures & data centres

Develop software for end to end systems and service mapping

Integrate existing CMDB / knowledge sources & toolsets

Services covering design, data capture, process development, training

To be an industry leader and subject matter expert• BCS Service Management Specialist Group• BCS Configuration Management Specialist Group• BCS Data Centre Specialist Group• ITSMF CCRM Special Interest Group• BICSI• Data centre management courses

Business ProcessesDepartmental, Company

ServicesEnd user, infrastructure, supplier

ApplicationsPC, server, mainframe, SOA

Virtual InfrastructurePCs, Network, Servers, Storage, DBMS

Hardware InfrastructurePCs, Network, Servers, UPS, Storage, Other

Fixed Infrastructure(Cabling, Power, Cabinets, Rooms, Buildings)


AgendaManaging the Facility

10:10 Skills Transfer and Training Certification for Data Centre TeamsPaul Rivett, CableNet Training Services

10:30 Taking Control of the Data CentreNeil Buckley, Paradigm-ITSM

11:00 Data Centre Management - The Hosting Perspective Phil Lydford, e-shelter

11:30 Coffee & Tea11:50 Avoiding the Overload of Excel and Visio

David Cuthbertson, Square Mile Systems12:30 Buffet lunch

Managing the Interfaces1:30 Process Development and Support

David Cuthbertson, Square Mile Systems2:30 Coffee & Tea2:50 Linking the ITIL CMDB to Data Centre Management

David Cuthbertson, Square Mile Systems4pm Finish



Avoiding the Overload of Excel & Visio

• Changing Requirements• Improve control without administrative

overload• Reduce silos of data• Management reporting• Examples of “integrated” data and reporting


Changing Requirements

BEFORE AFTERNo. of Servers per cabinet 3-6 30-40Power Disipated per cab. 300-1000W 3kW - 25kWCurrent service to cabinet 16A 32 A or 3 phaseTypes of Equipment Servers Blade Servers

Monitor Power Distribution UnitsKVMs MidSpan Boxes

Power Strips Disk Arrays (Storage)UPS Smart Power Strips

Regular Power StripsNetwork types 100Base-T 1000Base-T or 10G Base FNo. of Data Cables

Per server 1 or 2 2 or 3Total 20-30 300 - 400


Common Management ObjectivesBusiness Processes

Departmental, Company1. Improve speed of change2. Reduce risk of change3. Reduce disruption time4. Know current & future demand5. Optimise performance6. Adequate fail-over or recovery7. Reduce operational costs








Rack Space

75-100%

50-75%

<50%

%Full


Power

>6kW

2-6kW

<2kW

Loading


CoolingRated

75-100%

50-75%

<50%

The Inevitable Data Chaos

Rack diagrams Network diagrams

Patching spreadsheetsServer asset list

Building wiring diagrams

Maintenance contracts

Service desk inventory

Floor plans

Hot/cold aisle diagrams Departmental billing/charging

Chassis/card layouts

PAT testing results

Project plans

User locations

Network InventoryCircuit breaker loading

Power feeds to cabinetsIP addressing spreadsheet

Desktop asset listComputer room layout

Naming conventions

Power phase summationEquipment room mapsLabelling standards

PABX port mapping SAN Architecture


Service desk CMDB

You have to reduce duplication!


Listings by device type

Service Descriptions

Technical relationships

Physical Relationships

Business Data Flows

Business Processes

Device Information

Documents, Diagrams, Flow charts

Database(s) Spreadsheets

How do we get there?Before After


Excel

Word

Visio

ExcelExcel

Visio

Visio

Word

Word

Word

WordVisio

Visio

Visio

Excel

CommonData

DifferentViews

Reduce effortShare informationRe-use dataStandardisationTrustedProject/Operations useHome/central working

Visio

Excel

Word

Comparing Toolsets

1

End to end reporting

2

Diagrams to suit need

3

Forward Planning


Space Power Cabling Asset Workflow Capacity

AssetGen Connect

The AssetGen Range


AssetGen ConnectPhysical Infrastructure

AssetGen CMDBLogical

Infrastructure



ApplicationsPC or server based

Virtual Infrastructure ComponentsPCs, Network, Servers, Storage, DBMS

Hardware Infrastructure ComponentsPCs, Network, Servers, UPS, Storage, Other


AssetGen Connect Examples

• Tracing data and power connectivity• Finding blade and virtual servers• Rack Diagrams• Hot spots• Network diagrams with multiple technologies• Ad hoc diagrams to suit a need• Impact analysis of component change


AssetGen ConnectThe “Physical” Infrastructure CMDB

Service Desk CMDB, Spreadsheets, databases, monitoring systems, discovery

tools, asset lists, recovery plans

Connect application

Database (SQL)

Web interface& reporting

User Interface

AssetGen Connect

netVizvisualisation

Physical/logicalMaps

Device Imports/Updates Device Exports

Typical Benefits

1. Common platform for asset & connectivity management

2. Combines data from multiple teams / systems into one

3. No effort to produce rack layout diagrams

4. Enables ad hoc logical diagrams to be produced on demand

5. Identifies and reserves capacity at device & port level

6. Traces connectivity across multiple links and ports

7. Workflow reserves and identifies forward schedule

8. End to end understanding across multiple technologies


So….

Managing data centre infrastructure requires:– Good understanding of the physical infrastructure– Control of assets and technologies– Good understanding of dependencies– Knowledge base(s) that are owned and coordinated

• Local to data centre• Common with external support teams

– Linkage with change processes– Clear ownership and responsibilities


Questions?



AgendaManaging the Facility

10:10 Skills Transfer and Training Certification for Data Centre TeamsPaul Rivett, CableNet Training Services

10:30 Taking Control of the Data CentreNeil Buckley, Paradigm-ITSM

11:00 Data Centre Management - The Hosting Perspective Phil Lydford, e-shelter

11:30 Coffee & Tea11:50 Avoiding the Overload of Excel and Visio

David Cuthbertson, Square Mile Systems12:30 Buffet lunch

Managing the Interfaces1:30 Process Development and Support

David Cuthbertson, Square Mile Systems2:30 Coffee & Tea2:50 Linking the ITIL CMDB to Data Centre Management

David Cuthbertson, Square Mile Systems4pm Finish


Managing the Interfaces

• Technology / Service • Infrastructure / Components / Applications• Projects / Operations / Strategy / Governance• Internal / Outsource / Supplier / Customer• Central / Local

Every organisation is different and evolving, along the demands on IT groups


Common Management Objectives





Business ProcessesDepartmental, Company1. Improve speed of change

2. Reduce risk of change3. Reduce disruption time4. Know current & future demand5. Optimise performance6. Adequate fail-over or recovery7. Reduce operational costs



Why have a data centre?

• Controlled environment suited to high availability computing

• Controlled environment means– Clear policies covering the data centre– Defined processes and roles covering all aspects

of design, change and management practices– Operational systems that provide coordination

across teams, processes and evidence of control


Common Data Centre “Problems”

• Informal practices and walking databases• Lack of clarity around ownership and

interfaces between projects/operations teams• Complexity of infrastructure documented in

many disparate formats– Power, networks, storage, space, servers, etc.– Data sets centred on teams– End to end view difficult to get

• Funds for projects, not for management© Square Mile Systems 27

Improving Data Centre Processes• Provisioning

– Virtualisation of servers, storage, networks• Management

– Decommissioning– Maintenance– Risk control– Change coordination– Charging & billing– Cost control– Security– Capacity– Optimisation– Crisis management


Provisioning Example


Assess

Plan

Implement

Test

Completion

Knowledge bases coveringSpaceEnvironmentAsset ConnectivityDevice configurationsSystem / service configurations

Tasks and Information Needs • For the assessment

– Requirements management– Capacity management– Forward planning

• For detailed planning– Device information– System information– Service information– Resource allocation– Timescales


ISO20000 / ITIL FrameworkService Delivery Processes

Security Management

Service Continuity & Availability Management

Service Level Management

Service Reporting

Capacity ManagementFinancial Management

Release ProcessesRelease Management

Resolution ProcessesIncident Management

Problem Management

BusinessRelationship ManagementSupplier Management

Control ProcessesConfiguration Management

Change Management Relationship Processes


Change Impact Analysis

• What is meant by impact analysis?• When do we undertake an impact analysis?• How do we know we have identified all the

potential impacts?• How long does it take?



Client Funds TransferISA Payment Request Handling

BACS-IP BACS-Sched

Cheque Printer

Claims Processing

HO_BACRT01

ISA Payment

ISA Payment Function

Transact

UK_VWBIRM004

Workflow Client

Audittrack BACPAY

BACS IP-VPN

Citrix Server

Claims ProcessingFunds Move

HO Secure LAN

LPAR1 LPAR2

Paylog

UK_BIRM_BLADE-02

UK_VWBIRM001

Workflow

Front Desk FW_FWS_01

Oracle FWS_03 Oracle LP1 Oracle LP2

Service Desk

SQL FWS_04

SW_FWS_05

UK_BHAM-010301 UK_BIRM_BLADE_01 Z800 2065

BT Global

Citrix Client

MFLAN

SW_FWS_01

UK_VWBIRM002

Customer Services

MF_RT01

MF_RT02

MF_SW01 MF_SW02

ISA Payment Request

Services, Processes

Software

Hardware

Virtual

Incident

Incident

Change

Change

The CMDB “Concept”• A knowledge base that is shared across IT teams

covering all configuration items (CIs) and their relationships with other CIs. Could cover….– Hardware .. servers, networks, desktops– Software .. modules, applications, systems– Services .. Email, electronic payments, credit checks– Processes .. Customer checks, sales, finance

• Why have a CMDB?– Remove duplication and effort across teams– Common references for reporting and logging– Speed up change processes– Enable end to end understanding


Different Views

Service View Hierarchy

Service

Service

£1 per cup = £6Licences for 10 cupsMaintenance contract

Cost View – Asset Management


Row A

Row B

Row C

Row D

Row D

Carrier Terminations

Comms

Servers

Servers

Servers

B1-04B1-13B2-01B2-18

Data Centre ManagementSystems & Network Management


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24

1 2

R P S

DESKPRO

S 900

1 2 3

4 5 6

7 8 9

* 0 #

ABC DEF

GHI JKL MNO

PQRS TUV WXYZ

OPEN

1

21 1x 1 7

3L T R -R

Developing a CMDB

Services

Processes

Software

Multiple Applications & ServersDEALVIEW INTERNAL AUDITS

GLOBAL INS FUNDING PAYMENTS CASH

MANAGER CHECKBAL INCOMINGFUNDS

SECURE GH

SECURE EDI INVEST


Can the Server Support the Database?







UK_FWS_Blade_01

SQL FWS_04

UK_FWS_04

© Square Mile Systems 38MF_SW02MF_SW01


ISA Payment ProcessPayment Request

ISA Payment Request HandlingClient Funds Transfer

HO_BACRT01Cheque Printer

UK_FWS_05

ISA Payment Transact BACS-IP BACS-Sched Claims Processing

Paylog Audittrack BACPAY Funds Move Claims Processing

LPAR1 LPAR2

HO Secure LAN

BACS IP-VPN

UK_FWS_03

UK_FWS_Blade_01

Citrix Server

Oracle FWS_03 SQL FWS_04 Oracle LP1 Oracle LP2

Z800 2065

FW_FWS_01SW_FWS_05

Front Desk

UK_FWS_04

UK_FWS_10

Workflow Client

Service Desk

Workflow

SW_FWS_01

MFLAN

BT Global

Citrix Client

MF_RT01

MF_RT02

MF_SW01 MF_SW02

Customer Services

ISA Payment Request

Services, Processes

Software

Hardware

Virtual

If you don’t go the CMDB route..• Expect 1000s of documents, spreadsheets, diagrams and databases with

varying degrees of accuracy, depth and trust which are not coordinated

• End to end understanding is not “owned” by any team and it remains a “management problem”

• More disruption will be caused by change than device failure (80%-98%)

• Cost of providing IT services increases as managing change ends up becoming bureaucratic and slow

• Not delivering the management controls or costs the business/customer wants so they will find a new way – outsource, restructure, in-source.


Linking the ITIL CMDB to Data Centre Management

• Defining team requirements• Alternative approaches• Show examples of CMDB integration• Cover implementation issues


Change Ownership


AssetGen Connect

AssetGen CMDB

Internal to Data Centre

External to Data Centre



ApplicationsPC or server based

Virtual Infrastructure ComponentsPCs, Network, Servers, Storage, DBMS

Hardware Infrastructure ComponentsPCs, Network, Servers, UPS, Storage, Other


Define the NeedDependency Mapping

Security & Risk PlanningConsolidation Risk

Systems ArchitectureAvailability PlanningBusiness Continuity

Single Points of Failure

“Top Down”

The CMDB

MaintenanceAccurate updating

Correct classificationSystem reporting

Validating servicesConsistency across teams

Linkage to systems monitoring

Impact AnalysisIncident ManagementCrisis Management

Change ControlProject Planning

Incident Recovery“Bottom Up”


Practical Tips


A.For each critical or major service there should be a standard service map, validated regularly by a service owner! “Top down”

B.For each CI it should be possible to “see”impact on services as required “Bottom up”

For example 50 services, 250 Apps, 400 servers

= 50 Top down maps, 650 Bottom up mapsX


CI Imports/Updates CI Exports

Analysis & Authoring

Database (SQL)

Web ImpactAnalysis form

Impact / dependencychecking

AssetGen CMDB

netVizvisualisation

Service Maps

Service Desk CMDB, Spreadsheets, databases, monitoring systems, discovery

tools, asset lists, recovery plans

Connect application

Database (SQL)

Web interface& reporting

User Interface

AssetGen Connect

netVizvisualisation

Physical/logicalMaps

Device Imports/Updates Device Exports

Service ManagementData Centre Management

A Typical Plan

Book experienced guide Know the current stateStrategy & directionScopeOutline planPrototypeDetailed plan Process developmentToolset integration Data repository designData captureService introductionVerification checking


In Summary• Capacity and change management requires

knowledge of both physical and logical infrastructure• Can be justified on cost and speed to deliver change,

reduction in audits, or avoidance of penalties• Is fundamental to ITIL, ISO20000 and other

management frameworks• If you outsource hosting of systems, look at the

supplier management controls – its your responsibility to manage them to your need

• Understand what Square Mile can deliver for you
