2006 Jillian Macmurchy

Copyright © 2006, SAS Institute Inc. All rights reserved.

Jillian MacmurchyData Integration Solution ManagerEnterprise Business Intelligence Platform

SAS Enterprise Information Management


Agenda Overview of Information Management

• The impact of poor data management • The need for data management to evolve

Data Integration• The integration challenge• Smarter data integration

The role of Data Quality• The data quality process• Where to start?

SAS Enterprise Data Integration

Case Studies


Agenda Information Management

• The impact of bad data management • The need for data management to evolve

Data Integration• The challenge• Drivers for smarter data integration



Case Studies


Metric mishap caused loss of NASA orbiter - NASA lost a $125 million Mars orbiter because an engineering team used British Imperial units of measure while the agency's team used the more conventional metric system.

In May 1999, during the Bosnian War, the United States inadvertently bombed the Chinese Embassy. The bombing stemmed directly from a data error.

Famous Data Blunders


Business Impact of Badly Managed Data

Data broker ChoicePoint, Inc., payed $15 million in penalties for violating data security procedures and federal laws

Barbra Streisand pulled her investment account from her bank because it misspelled her name as ‘Barbara’

Going out of business… Enron, Andersen, Worldcom


Importance of EIM Re-enforced by Analysts

“At too many companies decision makers lack the information they need to make decisions that will drive sustainable growth, thanks to disparate and disconnected legacy systems coupled with ingrained business processes that owe their survival to inertia alone.” IQ Matters - Deloitte 2006

“Organisation’s should establish enterprise information management as a strategic business discipline that recognises information as a vital asset to be managed and maintained with the same rigor as applied to other significant assets (e.g. finance, real estate and people).”

Ted Friedman – Research V.P. Gartner Group – Nov 2005


The Need to Evolve?

Ongoing process

Survival of the fittest

Constant – evolve or die

Gradual change into a better form

"It is not necessary to change. Survival is not mandatory."

-- W. Edwards Deming


5 Levels of Evolution• Levels build on one another• Levels cannot be skipped• Competition will force

evolution• Proactive better than

reactive

4 Critical Dimensions• People• Process• Culture• Infrastructure

Information Evolution Model


Required Infrastructure Capabilities

Infrastructure







Case Studies


Pressing Business Issues Market Liberalisation & increasing competition

• Speed to market • Decision making

Compliance regulations• In-country (Government agencies, N.Z. Companies Office)

− Public accountability• Worldwide (Basel II, SOX)

Industry Consolidation• Mergers & Acquisitions• Department Mergers

No global view of business, operations, citizens, customers, products

Maintenance of multiple SAP/ERP/Legacy systems is costly


Sources: Delloite IQ Matters & Industry Studies

Despite Awareness, Use of Information is Still Not Fully Efficient and Effective

40%40% of IT budgets of IT budgets projected spend on projected spend on

integration.integration.

30%30% of people’s time is of people’s time is spent searching for spent searching for

relevant information.relevant information.

80% of CFO’s think there is room for 80% of CFO’s think there is room for improvement in timeliness, accuracy improvement in timeliness, accuracy and availability of data for decision and availability of data for decision

making.making.

Only Only 1/31/3 of CFOs believe that the of CFOs believe that the information is easy to use, tailored, cost information is easy to use, tailored, cost

effective or integrated.effective or integrated.

Data quality Data quality degrades at 2% or degrades at 2% or more per monthmore per month

60% +60% + of CEOs need to do a better job of CEOs need to do a better job capturing and understanding information rapidly capturing and understanding information rapidly

in order to make swift business decisions.in order to make swift business decisions.

80% +80% + of decision of decision makers believe there makers believe there

are “multiple are “multiple versions of the versions of the

truth”truth”

Data volumes Data volumes are doubling are doubling

annually annually


Typical Scenario

New project initiated

Does some data analysis

Uses an ETL tool or hand-coding

Has several data quality issues

Implements successfully but late

PeopleSoft DataMart

Transactional Operational Analytical

Sources

Project 1


Enterprise Data

Warehouse

Data warehouse project undertaken

Has shared sources with project 1

Re-analyses data

Uncovers some new data quality issues

Uses an ETL tool or hand-coding

PeopleSoft

Legacy Data

DataMart


DataMart

Sources

Project 2


Enterprise Data

Warehouse

Projects 3, 4, 5, 6…..

Fragmented tools and different approaches

Redundant business rules and multiple versions of the truth

Incomplete information flows

Inaccurate, incomplete, inconsistent data

Inability to keep pace with growing data volumes and velocity

Fragile, complex, hard-coded infrastructureOracle

Othersources

SAP

PeopleSoft

Trading Partners

Siebel

Legacy Data

ElectronicMarketplaces

DataMart


DataMart

Operational Data Store

Constant Changes to BusinessRequirements and Systems Landscape

ConsumerPortals


A Centralised Approach to Data Integration

System Consolidations & Migrations

Supply Chain Optimisation

Business Risk and Compliance

Business Integration

and Business Performance Management

Customer Intelligence and CRM Analyse

Extract

Cleanse

Enrich

Transform

Load

The right data, to the right place, at the right time Lower costs Improve efficiency & repeatability Assist with regulatory compliance

Faster time to market Enhanced customer service Increased productivity Leverage existing assets

End-to-end Metadata Management


Data Integration Maturity In order to mature an organisation must:

• Treat their data as a strategic corporate asset− The organisational structure should reflect this

• Have a clear understanding of the data infrastructure across the enterprise

• Implement an information governance process• Apply a cross-enterprise data management

framework







Case Studies


If your data was water would you drink it?


Reasons for Poor Data Quality Multiple data standards (or none!)

Data buried in free form fields

Redundant data

Multiple data sources

Duplicates

Default values

Invalid range of values

Null values

“60% of respondents claimed bad data and duplicate data as the primary reason for data integration problems”


The Real Reason for Poor Data Quality

Lack of investment in process• Strategy, Skills, Operations• Business Function• No ownership of data

Lack of Investment in people• Internal Staff, Internal Support• External Support, Training, Education, Skills• Staff turnover

Lack of investment in infrastructure• Storage, telecommunications, ETL Tools • Hand-coding• Data Cleansing, BI Applications

Lack of ownership of information quality by the business


Data Quality is Now Achievable More organisations are now beginning to focus

on data quality realising the cost in both time and money

Major initiative’s such as compliance, CRM and Master Data Management require good quality data to be effective

Analysts believe that organisations that focus on data quality programs will gain competitive advantage

“Almost 50% of CRM projects are failing or have failed due to problems managing data quality or reconciling customer

data”…Wayne Eckerson, The Data Warehousing Institute


The SAS Data Quality Process

ProfileUnderstand the quality of source dataQualityReconcile and correct inconsistent data

IntegrateConsolidate and link data across disparate systemsEnrichEnhance the value of dataMonitorAutomatically identify invalid information

Process


Where to Start?

2. Baseline Assessment, Measurement & Improvement:• Establish a baseline for data quality by assessing your

data • Identify objective metrics for measuring the quality of data• Use the results of the assessment to find and correct

problems in the data supply chain• Develop a process for on-going monitoring and auditing to

measure data quality against the KPI’s by domain experts

3. Integrate and automate parts of the data quality process as functions within the data integration process • Investigate how to integrate data enrichment

processes within existing data validation routines• Investigate how to integrate data enrichment

processes with real-time data acquisition processes

1. Business ownership & accountability• Recruit an executive sponsor• Convene a data quality working group• Have the business appoint a data quality steward

for each business unit


Today’s agenda Information Management





Case Studies


Data Integration capabilities…


With supporting services….


Able to Interact with all Systems


Critical Business Initiatives Rely on Integrated Information


Universal Data Integration







Case Studies


CHEP - Case StudyBusiness Case

Data Quality Improvement

Supply Chain Profitability Analysis

Building Marketing Capabilities


Sustainability• Establish clear accountability from Executive down• Establish data quality KPIs for balanced scorecard• Incentivise• Empower / enable self-audit11Data Cleanse• Profile data• Establish quality rules• Automate data cleansing tasks where possible• Measure & report quality of data22Single View of the Customer• “Force” a reconciliation of data stored in multiple systems• Provide visibility of the data entered to the account owner• Maximise access / distribution of information via browser interface33

DATA QUALITY IMPROVEMENTEnabling the DQ Management Strategy


PCMS

ORACLEFinancials

TRANSPORTDbase

CMSSiebel

• Analyse• Extract• Cleanse• Enrich• Transform• Load

Informing data quality improvement in core systems

D & BDATA

DATA QUALITY IMPROVEMENTData Integration, Improvement & Enrichment

End-to-end metadata

SAS Server

Reporting


DATA QUALITY IMPROVEMENTResult

83%

96%


Case Study – Data Migration

WHAT:

HOW:

RESULT:

WHY:

Migrate seamlessly from one data warehousing solution to another (24 million customer records and 7TB)

AA sold from its parent company, Centrica. Needed to build a new data warehouse that would be populated with information that was housed on Centrica's system and had less than 1 year to do it.

Used data integration technologies to extract the relevant data and build the new data warehouse

New Data Warehouse in place within 6 months and significantly reduced cost of operation, ownership over old system



How can SAS Help?

Provide a comprehensive and complete data integration platform

Provide industry and domain expertise built up over many years and projects

Implement using a proven data integration & project methodology

Provide the foundation for actionable BI

Data Quality Assessment engagement to help define a baseline measure


Data Quality Assessment Services lead fixed length engagement to review

a snapshot of your organisations data quality awareness

Analyse sample data

Summarise results and recommendations of data analysis in a report

Help you to develop a Return on Investment (ROI) model based on available information.

Review assessment findings with your team and executive sponsor(s).

Provide knowledge transfer on the methodology and results of the assessment.


Summary of Key Points

An enterprise wide technology based approach to EIM is critical

Data integration, data quality and metadata must be implemented using a common technology platform

Data quality is not a nice to have! Use a proven methodology Implement an information governance process Treat data as a strategic corporate asset

Information is the lifeblood of business; not a by-product of it


Thank You!

Technology

2006 Jillian Macmurchy