CNIC Project AssessmentFindings and Recommendations

Presentation to House Interim Committee on Government Accountability and Information Technology

Legislative Fiscal Office

August 12, 2008

2

Agenda

I. Introduction & Purpose of the AssessmentII. Project BackgroundIII. “Framing” the AssessmentIV. How’d We Do?V. Where Are We?VI. Myths & MisperceptionsVII. Next StepsVIII. Questions

3

I. Introduction & Purpose of Assessment

My Background Background & Purpose of Assessment Disclaimer

4

II. Project Background Project Purpose & Goals Participating Agencies & Participants Project Promises & Success Factors Project Business Case

5

II.a. Original Project Purpose & Goals Significant operating cost reduction.

Maintain or improve service level.– 7x24 operation– Managed service levels– Best in class tools and processes

Address critical risk issues.– Make State’s computers and networks more secure– Deploy better and more reliable technology– Improve ability to recover from disaster

Consolidation allows the state to:– Achieve savings– Correct deficiencies and mitigate risk– Position government for future initiatives– Demonstrate accountability

6

II.a. Original Project Purpose & Goals (cont.)

Consolidated infrastructure lays the foundation for the following:

– Standardization– Additional consolidation– Greater economies of scale– System and data integration– Technology-enabled business processes– More transparent and accessible government

Consolidation shows that state government can:– Work effectively on enterprise strategies– Overcome institutional barriers to change– Resolve complex business and technical issues– Produce the same or better service at lower cost– Be accountable to taxpayers

7

II.b. Participating Agencies & Participants

AgencyAdministrative Services

Consumer & Business Services

Corrections

Education

Employment

Forestry

Housing & Community Services

Human Services

Oregon State Police

Revenue

Transportation

Veterans’ Affairs

Governing Board MemberTheresa McHugh

Cory Streisinger

Max Williams

Vickie Fleming

Deborah Lincoln

Clark Seely

Bob Repine

Gary Weeks

Ron Ruecker

Elizabeth Harchenko

Bruce Warner

Jim Willis

8

II.c. Project Promises & Success Factors

Cost savings within a couple of years. Estimated 16% statewide operating cost reduction.* Reduce SDC staff from 155 to 93 (immediately). Estimated 55 months to recover one-time costs. Savings will continue in perpetuity following ROI. Analysis assumed most conservative scenarios. The business case was compelling.

* Savings estimates are conservative – actual savings could be higher

9

II.d. Original Project Business Case Lies, damned lies, & business cases (FSR’s). Authorship - Completed by a vendor, not the customers it affected. Ignored significant historical best practices & experience info. Flawed process with bad data.

– Baseline data from agencies was low on costs and FTE’s.– Immediate reduction of staff from 155 to 93 not realistic.– All consolidations are not alike.– Didn’t approach the analysis as an outsourcing effort (from scratch).– Too much focus on cost savings.– Didn’t include many other reasons/benefits for consolidating.

Flawed assumptions – everyone was on the cost savings band-wagon. Unrealistic cost-savings expectations. Focused way too much on cost savings too early in the project cycle. Took into account only the “before” and “after” data center costs.

10

Promised “Cost-Savings” Graph

"PROMISED COST-SAVINGS"(Corrections Example)

0

2000

4000

6000

8000

1 2 3 4 5 6 7

YEARS

MO

NTHL

Y SD

C CO

STS

Series1

Series2

11

Actual “Cost-Savings” Graph

"ACTUAL COST-SAVINGS"(Corrections Example)

01000200030004000500060007000

1 2 3 4 5 6 7

YEARS

MO

NTH

LY S

DC

CO

STS

Series1

Series2

12

III. “Framing” the Assessment Instead of framing the discussion solely on the

financial business case and promises made early in the project, we need to look at the bigger picture and understand:

– Consolidations are occurring throughout state governments;– Oregon is facing the same consolidation challenges as other states;– More than consolidation was accomplished/addressed by CNIC;– Importance of IT, in general, and the data center specifically;– The “perfect storm” of issues and challenges facing IT; and – How our consolidation effort measures up relative to other states’

historical experience, costs, service levels, & staff levels.

13

NASCIO Chart of Consolidations – What’s Happening in Other States

14

Reasons for Consolidation Efforts4.2 What factors are driving your state's strategy to consolidate data centers?

States were asked to indicate factors that are driving their state's strategy toconsolidate data centers. Disaster recovery and Replication, redundancy and faulttolerance were the two driving factors most indicated by respondents. Cost savingsand Security and data classification were also listed as significant factors in states'decision to consolidate data cernters. "Other" comments indicated "Existing site isexperience space constraints," and "Optimization and economies of scale." (SeeTable 5. for a complete list of responses).

Factors Driving States’ Strategies to Consolidate Data Centers

Driving Factors Percent Disaster recovery 82.8% Replication, redundancy, and fault tolerance 75.9% Cost savings 65.5% Security and data classification 62.1% Better access to new technologies for all agencies 55.2% Aging state facilities 51.7% Improved information sharing/data integration 51.7% Energy conservation/Environmental concerns 41.4% Size 37.9% Business applications 24.1%

Source: NASCIO’s 2007 survey of state data center consolidation initiatives.

15

Oregon Isn’t Different-Drivers & Results

OREGON’S CONSOLIDATION DRIVING FACTORS & RESULTS

Driving Factors Oregon Driver? (Yes/No)

Value Realized? (Yes/No)

1. Disaster recovery 2. Replication, redundancy, and fault tolerance 3. Cost savings 4. Security and data classification 5. Better access to new technologies for all agencies 6. Aging state facilities 7. Improved information sharing/data integration 8. Energy conservation/environmental concerns 9. Size 10. Business applications 11. Existing site(s) is/are experiencing space constraints 12. Optimization and economies of scale

* cost savings are difficult to measure due to inaccurate baseline data and service growth. Until consolidation efforts are completed, cost savings will not be realized due to the inability of the data center to take advantage of economies of scale.

16

Oregon Isn’t Different - Challenges

TYPICAL CONSOLIDATION CHALLENGES

Challenges Oregon? Yes/No

1. Agency desire to remain independent/autonomous

2. Management resistance to change

3. Workforce resistance to change

4. Unrealistic business case expectations

5. Poorly defined customer business processes & costs

6. Failure to establish baseline costs

7. Higher than anticipated costs

8. Failure to meet service expectations

9. Difficulties in equipment relocation

10. New higher service expectations

11. Statutory and regulatory exemptions

12. Problems in establishing new data center “business”

17

Importance of IT and Data Centers

18

SDC as a Utility Service in “So-That Chain”

PUBLIC

AGENCY BUSINESS SERVICES

SDC

AGENCY IT SERVICES

ELECTRICITY

19

The “Perfect Storm” Facing IT Orgs

SECURITY & DISASTER PLANNING

HUMAN RESOURCESISSUES

COLLABORATION,INTEGRATION, SILOS, &

ENTERPRISE FOCUSISSUES

CUSTOMER SERVICEDEMANDS

FINANCIAL

TECHNOLOGYTRENDS

TECHNOLOGYOBSOLESCENCE(Modernization)

INFORMATION TECHNOLOGY

ORGANIZATIONS

E-BUSINESS DRIVERS

20

More Than Just a $-Savings Effort Outsourcing undertaking (complex). Cost avoidance effort. Technology modernization effort. Process improvement/quality effort. Silo-busting & enterprise-focus effort. Staff re-tooling and training effort. Change in business service model. 1st step in re-architecting IT environment. 1st step in re-engineering IT environment. Security & disaster recovery improvement effort. Asset id’ing & software licensing compliance effort.

21

IV. How’d We do? In general, I feel comfortable in saying:

– CNIC was the right thing to do.– Things didn’t go as well as they could have.– Unrealistic promised cost savings were not achieved.– CNIC was not a failure.– SDC performance is measured using the wrong criteria.– New SDC data center is operating with 10+ agency “data centers”

housed within its walls.– Some agencies are paying higher costs for same level of service.– SDC is getting recognition for its efforts from outside orgs.– Other states are calling for advice on SDC’s efforts.– SDC service costs are lower than average state costs.

22

What went right? Built a quality data center. Successful “lift and drop.” Basic SDC data center “business” infrastructure in place. Basic level of SDC governance is in place. State staff were treated properly. Overall, service at/above what the agencies were providing. Strong, loyal, dedicated data center team with quality focus. Security and disaster planning are better than before. CIO’s are regularly meeting and talking to one another. Began an “enterprise focus” (started “silo-busting”). Significant energy savings. Recognition from “InformationWeek” and ComputerWorld. We hired highly experienced and qualified SDC managers.

23

Example - Network Topology

Existing Proposed

24

What went wrong? Two oversight reports clearly identify the key project related issues. Flawed business case resulted in unrealistic expectations. Insufficient front-end planning. Failed to understand CNIC as an outsourcing effort (which has well-known

best practices for handling effectively). Underestimated what it takes to set up a “data center” business. Didn’t utilize industry consolidation “best practice experiences.” Key vendor (& state) didn’t have sufficient consolidation experienced staff. Didn’t always listen to our “lessons learned” feedback. Didn’t always listen to our consultants, IT managers, & interested parties. Didn’t manage project scope effectively. Passive resistance from agencies (inadequate supplying of critical

information to project and data center staff). Inadequate project controls & governance. Inadequate documentation and turnover support for “lift and drop.”

25

V. Where are we? Phase 0 – Planning & Business Case – completed Phase 1 – Data Center Facility Build – completed Phase 2 – Relocate 10-11 Dept data/ctrs. - completed Phase 3 – Stabilize & Initial Infrastructure - completed - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Phase 4 – Consolidation Planning Phase 5 – Phased Consolidation Approach (driven by

technology refresh funding over 2009-13)

26

Current Situation (SWOT)Strengths: Quality data center in place Excellent service oriented SDC staff Many tough “cultural issues” dealt with Data center reliability and service Data center environment flexibility Reasonable go-forward plan

Opportunities: Chance to realize real cost savings Leverage “economies of scale” Execute a phased consolidation Develop an enterprise perspective Strengthen existing SDC governance Build a “best-in-class” data center

Weaknesses: Inadequate SDC funding & staffing Lack of mutual accountability & SLA’s Lack of commitment to SDC success Myths & misperceptions Unwillingness to think “enterprise” Poor agency documentation Weak chargeback model

Threats: Agency unwillingness to consolidate SDC consolidation resource shortfall Failure to align agency/SDC processes Failed governance support Periodic passive resistance Lack of commitment to SDC success Lack of funding/resource support

27

Current Challenges Ongoing “myths” and “misperceptions.” Data center funding & FTE’s. Moving past business case “promises.” Costs and SDC rates (level and fairness). SDC billing algorithm (i.e. chargeback model). Rumors in the press. SDC manager’s control over data center environment. SDC has significant responsibility without sufficient authority or control. Missing SDC “business infrastructure.” Passive resistance still occurring. Service and expectations growth. Spending continues when existing SDC equipment is often under-utilized. Lack of enterprise focus by agencies. Agencies’ unwillingness to control # of products/tools. Poor or no documentation from agencies.

28

Recent Issues in the Press Cost Savings Service Costs Space Security Disaster Planning Business Continuity Planning

29

VI. Myths & Misperceptions Myth #1 - CNIC Project shouldn’t have been done. Myth #2 – CNIC Project was a failure. Myth #3 – CNIC “business case” was “compelling.” Myth #4 – No CNIC cost savings (ever). Myth #5 – SDC data center was poorly designed (i.e. space, power, air conditioning, etc.). Myth #6 – SDC is providing poor service. Myth #7 – Agencies had clear picture of their costs ($’s & FTE’s). Myth #8 – SDC chargeback rates are too high. Myth #9 – SDC has adequate funding & staff. Myth #10 – SDC security is worse/inadequate. Myth #11 – SDC disaster planning is worse/inadequate. Myth #12 – CNIC consolidation was simple. Myth #13 – Little or no resistance to consolidation. Myth #14 – SDC’s success is in its own hands. Myth #15 – Consolidation is done.

30

Bottom Line Consolidation was the right thing to do. CNIC Project was not a failure. Consolidation is not completed. Can’t re-architect 30 years of silo’ed architecture overnight. There’s no reason to go back to silo’ed data centers. Business case was flawed (unreasonable expectations). Cost

savings only, should not have been the foundation. Front-end project planning was inadequate. Cost savings are difficult to prove (no valid baseline costs),

but real – some have already occurred and others will occur later than desired in the CNIC’s lifecycle.

Many of the savings/benefits of consolidation will not be possible until actual consolidation occurs.

Overall, SDC’s service quality is good.

31

Bottom Line (cont.) Some agency SDC costs are higher than pre-consolidation. Smaller agencies pay disproportionate amount of SDC’s bill. SDC rates are in line with industry standards. “Problems” identified in the press (i.e. space, cooling, electricity,

disaster planning, security, etc.) are, in general, unsupported by the facts.

SDC’s data center is well-designed and ready to provide long-term services.

SDC funding and staffing are inadequate and below industry standards.

SDC’s basic “business function” is in place. We’re moving from a “silo-ed” to an “enterprise” focus. We’re dealing with technology obsolescence issues. SDC has a plan for completing the consolidation over 3-4 years. SDC cannot be successful on its own.

32

VII. Next Steps DAS-SDC Annual Plan “Focus Areas”

– SDC Consolidation– Staffing and Workforce Management– Cost Recovery and Rates Management– Energy and Power Management– Customer & Service Level Management– Disaster Recovery Management– System Security Management– Management Process & Controls

Consolidation is planned to occur over the next 3-4 years driven by the funding available for SDC technological refreshes.

33

Recommendations for SDC Success Let’s finish the job that we started. Survey both SDC and the agencies for what’s needed to finish the job. Develop a joint SDC/agency “go-forward plan.” Provide SDC with additional funding to expedite consolidation. Provide SDC with additional staff to align with industry standards. Expedite consolidation efforts (i.e. support critical POPs). Encourage agencies to prioritize consolidation in planning and budgets. Provide SDC with control of key processes. Improve SDC governance. Implement “pass-thru” billing capability. Establish clear SDC performance/success measures. Assure better teamwork between agencies and SDC. Require accountability from agencies for SDC success. Establish SLA’s that hold both SDC & agencies accountable. Increase agency willingness to use common products/tools. Benchmark Assessment of SDC from Gartner/IBM/Hackett. Benchmark Assessment of three major SDC customers.

34

VIII. Questions?