Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012T.C. Redman, Page 1 The Ten Habits of Those with the Best Data Thomas C. Redman,

Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T.C. Redman, Page 1

The Ten Habits of Thosewith the Best Data

Thomas C. Redman, Ph.D.

Navesink Consulting Group

at the Toronto DAMA

May 17, 2012

[email protected]

Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 T. C. Redman, Page 2

Introduction and Summary

Those with the best data: Adopt a customer-facing definition of quality. Aim to prevent errors at the points of data creation,

rather than correcting them downstream. Follow “ten habits” that align the entire organization. Enjoy rich rewards for their troubles!

Agenda

What “the best data” looks like Thinking about quality An “non-delegatable” choice The ten habits My (evolving) views on organization Questions anytime


Market Data Vendor

Background: Financial services companies purchase market data from companies such as Reuters, Bloomberg, etc.

Lack of trust causes them to purchase basic data from multiple sources.

Bank request: Far better data, so it could reduce its vendor base and eliminate downstream costs of bad data.

Work conducted: Clear statement of customer needs. Measurement against those needs. Root causes identified and addressed, one at a time. Statistical control. In the course of day-in, day-out work.

Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012 TCR, Page 4

Market Data Example: Results


Each error not made saves an average of $500. Quickly millions!

The day-in, day-out work of data quality management is conducted at the work group level

Access Financial Assurance at AT&T

Background: AT&T expenditures for “access” about $20B/yr. Access Financial Assurance aims to ensure integrity of access

bills, through parallel “billing.”

Key Idea: Get the bill right the first time.

Work conducted: Dissatisfied middle manager, seeking a better way. Top-down deployment. Staff group defined series of deliverables, then audited (regional)

compliance. Supplier and process management. Customer needs, measurement, improvement, control.


Results: Access Financial Assurance

Data accuracy improved 90%. Billing errors reduced 98%. Cycle time (bill period closure) reduced 67%. AT&T costs (of financial assurance) reduced 73%

($100M/year). LEC costs (of access billing) reduced 20%.

Redman-Toronto-ten habits-May2012 TCR, Page 7© Navesink Consulting Group, 2000-2012

There is much hidden “non-value-added work” built in to accommodate bad data.

Enterprise Programme at BT*(British Telecom) Revenue: $33 billion/yr Employees: 95,000 Operates in 170 countries 22 Million customers (4 Million business)

Enterprise Data Quality Improvement Programme (10-year effort) Recognized the inherent complexity of people, process, technology issues

(e.g., data quality problems masquerading as “systems issues.”) Explicit linkage of data (quality improvement) to strategic business

objectives (e.g., business transformation). Over time, magnitude of DQ problem understood and exposed. Governance structure starting at the very top. Consolidated expertise in data quality improvement in IT. Estimated and delivered benefits vetted by Finance.

Redman-Toronto-ten habits-May2012 TCR, Page 8

*This summary largely courtesy of Nigel Turner, who led BT’s programme and is now at Trillium Software. He has vetted this summary.

© Navesink Consulting Group, 2000-2012

BT – ResultsEnterprise Data Quality Improvement Programme, cont’d Dual focus on reducing capital expenditure and the rework that results

from searching for “lost network facilities.” Problem discovery, measurement, audits, new controls (hold the gains)

enhanced by Trillium DQ tool suite. Focused on “big improvement projects” (delivered 75 over ten years). Dual focus on data clean-up and process improvement.


More than you might think, data permeate everything. Bad data are “silent killers.

Business Benefits: > $1B (verified and conservative) Also improved customer satisfaction, better regulatory compliance, reversed brand damage and revenue leakage, and contributed to business transformation: These benefits not quantified.


High Quality in Your Mind’s Eye

Redman’sFavorites

Apple products

Italian tile

C&D Heating and Cooling

Disneyland

Common Characteristics

Relatively few defects.

They are corrected in a prompt, friendly manner.

Easy-to-use

Make it easier to do the things I want to do.

Sleek design!

Trust the company!!


Contrast the I-Phone with a Data Model

Or a financial statement!

E.g.,

"Corporate structure”

"Employment"

"Membership"

etc.

PARTYPARTYRELATIONSHIP

PARTYRELATIONSHIPTYPE

ORGANI-ZATION

PERSON

from

on one side of

to

on the other side of

an example of

embodied in


Data QualityData are of high quality if they are fit for their intended uses

(by customers) in operations, decision-making, and planning (after Juran).

free of defects:- accessible- accurate- timely- complete- consistent with other sources- etc.

possess desired features:- relevant- comprehensive- proper level of detail- easy-to-read- easy-to-interpret- etc.

Data that’s fit for use

Customers are the ultimate arbiters of quality!!


Data Quality - aspirational

“Exactly the right data and information in exactly the right place at the right time and in the right

format to complete an operation, serve a customer, make a decision, or set and execute

strategy.”*

*Redman, Data Driven: Profiting from Your Most Important Business Asset, Harvard Business Press, 2008


Data Quality – day-in, day-out

Meeting the most important needs of the most important customers.

© Navesink Consulting Group, 2000-2012 T. C. Redman, Page 15

Data Quality: The Non-delegatable Choice

Redman-Toronto-ten habits-May2012

Un

man

aged

Pre

vention at s

ource

Find and fix

Eliminate The Sources Of PollutantTo Clean Up The Lake, One Must First


They recognize that, left alone, accountability shifts downstream!!!

Here’s how youdo number 3,

soncos2(x) + sin2(x) = 1


The (nearly-certain) results

Approach

Management

Focus

Typical

Error RateCost of Poor Data Quality

Find and Fix

(First-Gen)The Past

1-5% (at the field level)

20% of revenue

Prevent Future Errors (Sec-Gen)

The FutureTwo orders of

magnitude better

Reduced by two-thirds


Habit 1: Focus on the most important needs of the most important customers

Those with the best data adopt a customer-facing definition of quality.

In doing so, they recognize that: All data are not created equal. Similarly, customers,

problems, and business opportunities are not created equal.

Generally, the most important data are those needed to set and execute the company’s most important business strategies.

And they focus as much of their energies on these customers, strategies, and data.

Said differently, their data quality programs are fully aligned with business strategy.


Data Doc’s Hierarchy of Needs

1. Acquire the data they need

2. Trust that data are correct

3. Understand meaning

4. Understand how data fit with

other data

5. Keep data safe from harmMany people and

organizations exhibit a

“hierarchy of “needs”


Habit 2. Process, process, process

They recognize that they create data via their cross-functional business processes

A B C D

They recognize that most errors occur “in the white space”

They think “BIG-P”

They recognize “the next guy” (serving the customer) as a customer


Data - Defined

A datum consists of three elements:

The thing of interest in the real-world

The particular of interest

(entity, attribute, value)

The value assigned to the attribute for the entity

Example: (Jane Doe, Service Record Date = July 1, 1996) Note that, as defined, data are abstract. “Customers” see them as they are

presented in tables, databases, graphs, etc.


Implications…

Thus even the simplest datum arises from three distinct sources:

The model (entity, attribute)-pair is created within a modeling process, usually by IT or purchased from outside.

The data value is created (at enormous rates) by the business process.

The presentation may be created by database tools, application programs, PowerPoint presenters, etc in an application development process.

All three processes must be managed end-to-end for high-quality data to result.


They use the Customer-Supplier Model to establish requirements and feedback loops

Suppliers Customers“Your Process”

inputs outputs

requirementsrequirements

feedback feedback

BIG-P process


Habit 3: They employ supplier management for external sources of data

Suppliers Customers“Your Process”

inputs outputs

requirementsrequirements

feedback feedback

They expect high-quality data from outside. And invest (time) with their suppliers to get them


Habit 4: They measure quality at the source in business terms

They define metrics with clear business implications.

Private Bank’s Customer Data:

Percent of statements with

an error

Telecom’s Access Charges:

Risk = Overbilling + Underbilling

Many organizations:

Fraction “perfect” records

(interpreted as “work” done correctly)

Time-Series Record-Level Accuracy

0.20.30.40.50.60.70.80.9

1 3 5 7 9 11 13 15 17 19 21 23 25

week

frac

tio

n r

eco

rds

com

ple

tely

co

rrec

t

They measure continuously

They get good at interpreting results

They integrate top-line DQ metrics with other business results


Habit 5: They employ controls at all levels to halt simple errors and establish a basis for moving forward

Time-Series, Record-Level Accuracy

0.20.3

0.40.50.60.7

0.80.9

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37

week

reco

rds

com

ple

tely

co

rrec

t

UCL

LCL

They employ simple edits to stop errors in their tracks:

Ex: (Title = Mrs., Sex = M) cannot be correct

They employ statistical control to identify process issues early and to look forward:


Habit 6: They have a knack for continuous improvement

Time Series, Record-Level Accuracy

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 4 7 10 13 16 19 22 25 28 31 34 37 40 42 46 48

week

reco

rds

com

ple

tely

co

rrec

t

They have a way of not just starting, but completing improvement projects, both to:

• eliminate root causes of error

• acquire new data


Habit 7: Set and achieve aggressive targets

Time-Series, Record-Level Accuracy

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1 4 7 10 13 16 19 22 25 28 31 34 37 40 42 46 48 52 55

week

reco

rds

com

ple

tely

co

rrec

t

They focus not just on the level, but also on the rate of improvement

They set targets like:

• half the error rate every year

• add two significant new features every year

They decide to position themselves near the front with respect to quality in their industries

In many respects, for them planning for quality is no different than planning for revenue growth, new product development, etc.


Habit 8: Formalize management accountabilities for data

I’ve told that CIO about these data problemsa million times! Why can’t

they get them right?

They recognize that responsibility for data lies with “the business,” not IT.

Some codify responsibilities in policy.

My favorite (adopted for data):

“Don’t take junk data from the guy upstream. And don’t pass junk data on to the next guy!”


Habit 9: A broad, senior group leads the effort

They know that that quality programs go as far and fast as the senior person leading the effort demands.

So a broad, committed, senior team leads the effort.

“They thought they could make the right speeches, establish broad goals, and leave everything else to subordinates... They didn’t realize that fixing quality meant fixing whole companies, a task that can’t be delegated.”

Dr. Juran, 1993

Experience so far is that “data” is even tougher than the factory floor.


They: Distinguish “I” from “IT.” They recognize that they

can’t automate their way out of a quality issue. Start small. Create early wins. Actively manage change. Avoid unwinnable battles, especially early on. Build political capital. Over time, they build data quality into:

The organization People’s psyche To new systems

Habit 10: Recognize that the “hard issues are soft” and actively manage change

Who Does the Work?*8. Formalize management

accountabilities for data

Senior Leadership:

Middle Management (Command):

10. Advance a culture that

values data and data quality

9. Broad, informed,

demanding leadership

2. Manage processes that

create data (so they do so correctly)

3. Manage “suppliers” (both

inside the Army and out) of data

7. Set and meet

aggressive targets for

improvement: top-to-bottomWork is highly

interconnected

1. Focus on the most important

needs (of customers)

6. Improvement: Find and

eliminate root causes of error

4. Measure quality levels

against customer needs

5. Deploy controls, at all

levels, to remain error-free*

Everyone who touches data = Four Basic “Steps”

*Ten habits of those with the best data from Redman, Data Driven: Profiting from Your Most Important Business Asset, Harvard Business Press, 2008.

Taken together, the tasks define

an overall “Management

System for Data Quality”


The ten habits reinforce one another*9. Broad senior

leadership

8. Data Policy

3. Supplier Mgmt

1. CustomerNeeds

10.ManageData Culture

5. Control4. Measurement 6. Improvement 7. QualityPlanning

DefinesAccountabilities

via

MustAdvance

DeployedTo

Supports

Supports

APlatform

For

LeadsTo

Identify"gaps"using

UnderliesEverything

Responsiblefor meeting

Responsiblefor meeting

ToBetterMeet

DeployedTo

MonitorConformance

Using

Underlies

Everything

SetTargets

For

2. Process Management

*This figure adapted from Redman, Data Quality: The Field Guide, Digital Press, Boston, 2001Redman-Toronto-ten habits-May2012 T. C. Redman, Page 33© Navesink Consulting Group, 2000-2012


The Ten Habits apply to all data, in all industries and government

Market, product, and people (customer and employee) data. Intelligence, scientific and logistics data. Health care data.

Data created internally or gathered from external sources.

Meta-data, master data, enterprise data. Data to be stored on paper, in operational systems, in

warehouses, enterprise systems. Client statements, 10-Ks, prospectuses. Data only seen by computers and data that convince

people to trust industries and companies (or not).

Fundamental Organization Unit for Data Quality


*Quality Improvement facilitator is a permanent role, supporting a series of project teams, which disband when their projects complete

Leadership

Tech SupportManager

Supplier Management

Process Management

Requirements Team

Control Team

Measurement Team

Customer Team

Improvement Teams*

Control Team

Measurement Team

Current “Best” Overall Organization Structure for Data Quality*

Data Council• Leadership• Data Policy• Define process and supplier structure• Advance Data Culture

Process B(metadata)

Supplier D

*This figure adapted from Redman, Data Quality: The Field Guide, Digital Press, Boston, 2001

Chief Data Office

• Day-in, day-out leadership• Secretary to Council• Metadata process owners• Training• Deep expertise• Supplier Program Office

Chief Information TECH Office• Technical Infrastructure• Security/Privacy impl.• Build DQ features into new systems• (One-time) data cleanups

Process AManage and

improve data, following agreed

process mgmt methods

Supplier CManage and

improve data, following agreed supplier mgmt

process)

… Other improvement projects …

“PROCESS VIEW:” These structures overlaid on current organization chart

Improvement project team

Follow agreed methodto complete assigned

project

Redman-Toronto-ten habits-May2012 T. C. Redman, Page 36© Navesink Consulting Group, 2000-2012

Primary responsibility for DQ

Primary responsibility for DQ

Federated Org Structure for Data QualityFederated Org Structure for Data Quality

Org HeadOrg Head

-

• Audit

-

• Audit

AuditAudit

•Policy Deployment•DQ Transparency•Common Methods

•Policy Deployment•DQ Transparency•Common Methods

Chief Data OfficeChief Data OfficeDepartment LeadershipDepartment Leadership

Senior Data BoardSenior Data Board

DQ Policy

Dpt DQ Team

Dpt DQ Team

Redman-Toronto-ten habits-May2012 TCR, Page 37© NCG and DBP, 2012

CreatorsCreators CustomersCustomers

Department Organization for Data Quality Department Organization for Data Quality

Note: DQ facilitator leads efforts to understand customer needs, conduct improvement projects, etc. Ideally, reports into functional management, with a dotted line to the data team, as pictured

Department Head Org’s Data Board

IT: Systems, databases, metadata repository, tools

MetricsTeam

TrainingTeam

ControlTeam

Metadata/Stds team

BusinessCase Team

FacilitationTeam

Head of Data Program

Data Creation

rqmtsrqmts

feedbackfeedback

input output

Support Tech

DataSupplier

DataCustomer

coordFunctional Activities

DQ facilitator

Dept DataTeam

ServicesTeam

Dept Data Committee

SupplierMgmt Team



Final Remarks

Those with the best data: Adopt a customer-facing definition of quality. Aim to prevent errors at the points of data

creation, rather than correcting them downstream.

Follow “ten habits” that align the entire organization.

Enjoy rich rewards for their troubles!


What Did He Say?

Questions?

Thomas C. Redman, Ph.D.

+1 [email protected]

Documents

Redman-Toronto-ten habits-May2012 © Navesink Consulting Group, 2000-2012T.C. Redman, Page 1 The Ten Habits of Those with the Best Data Thomas C. Redman,