How to Measure Data QualityData Quality Management System (DQMS)

Self Assessment

Product Inspection & KPIs

Continuous Improvement

“Quality is a measurable, manageable business issue”

John Guaspari

© 2012 GS1

How to Measure Data QualityKey Elements of DQF

• Data Quality Management System• Self-assessment Model• Product Inspection Model• Scorecard & KPIs• Monitoring & Continuous Improvement Guidelines

© 2012 GS1

Data Quality Management System (DQMS)

•Plan your Data Quality strategy

Define goals and objectives

© 2012 GS1

How to Measure Data QualityData Quality Management System (DQMS)

• A strategic approach that enables enterprise-level governance and management of data quality to generate superior information assets.

• Based upon best practices and capabilities that support increasing data quality. Organized by: • Functional Areas• Action Types

• Flexible design and implementation depending on the unique requirements of an organisation

• Scalable model that accommodates any size organization large or small

© 2012 GS1

How to Measure Data Quality DQF DQMS - Capabilities

“Capability”• Ability to perform• Capacity, expertise,

knowledge, or talent required to perform a task

A business capability is the expression of the ability, materials, and expertise an organization needs in order to perform core functions.

Data

Quality

People

ProcessTechnology

© 2012 GS1

How to Measure Data Quality DQF DQMS – Functional Areas

Capabilities have been organised by Functional Area:• Organisational

– What DQ enterprise level structure is required?– What DQ roles and responsibilities must be defined?

• Policies and standards – What DQ guidelines are needed to provide governance and

reference?• Business processes

– What critical processes drive day-to-day operations?• Systems capabilities

– What technology is necessary to support the business?

© 2012 GS1

How to Measure Data Quality DQF DQMS – Activity Types

Within each functional area, there are four key elements or activity types:• Plan:

• Strategy & Goals

• Document• Policies, Process SOPs, Org, RACI

• Execute• Communication, Training, Change Mgmt

• Monitor/Control• Audits, Reporting• Life Cycle, Continuous Improvement

© 2012 GS1

How to Measure Data Quality Example – DQF DQMS Capability Matrix

Organisational Capabilities

Policies & Standards Business Process System Capabilities

Plan

Executive sponsorship - Mission & vision; Accountable leadership; Staff roles & skill sets; Data owners & stakeholders; Data governance office.

Mission & vision; Goals & objectives; Guiding principles; Success measures; Action plans; Policy & standards management

Initial data entry & setup; Ongoing data maintenance; Processes involved in the information's life-cycle

Unified data repository; Design & architecture; Workflow, user interface; Data validations; Security, access controls; Revision/change history; External publication; Internal publication

Document

Governance organisational structure; Roles & responsibilities; Personal objectives; Reporting alignment

Mission, goals, principles and success measures; Governance model, decision process; Data definitions & standards; Security & use policy; Audit procedures; Documentation standards; Risk Management; Customer feedback policy

Operating procedures; Process flow diagrams; Job aids, work instructions; Performance metrics

System requirements; Operating procedures; Performance metrics

ExecuteEducation & awareness; Internal communication; Training

Education & awareness; Documentation management; Policies & standards management; Data issue management Training; Customer feedback resolution

Education & awareness; Performance management; Process issue management; Change management

*See note on section 2.4.3

MonitorOrganisational capability review; Review of personal objectives

Policy & standards review

Workflow controls; System validations; Performance reporting on service levels; Performance reporting on data quality; External & internal feedback; Process compliance audits; Product measurements; Review & reporting audit results; Monitor impact of erroneous data

Performance reporting on service levels

© 2012 GS1

How to Measure Data Quality Example – DQF DQMS Documentation

Functional Area

Activity Type

Capability Name

What – definition

Why - rationale

Recommendations - hints for implementation

Practical Examples

Self assessment - Questions linked to capability

© 2012 GS110

How to Measure Data Quality DQF DQMS – Essential Steps

1. Gain top management commitment

2. Appoint responsible managers

3. Start data quality awareness programmes

4. Provide training

5. Create Data Quality Management Processes

6. Develop data quality management system documentation

7. Document controls

8. Implement and execute DQ program

9. Perform Internal data quality audit

10. Conduct management review

11. Conduct conformity assessment (Optional)

12. Perform continual improvement

© 2012 GS111

Self Assessment

•Assessment of capabilities and priorities

Identify areas of

opportunity

© 2012 GS112

How to Measure Data Quality Why Self-Assess?

• Identify areas of improvement as part of a continuous DQ program

• Engage in trading partner collaboration for process improvement• Although the final assessment results are discussed among

trading partners, the execution of the self-assessments themselves is always performed by one organisation without interference or involvement from external parties.

• Measure compliance with global standards or best practices

• Benchmark DQ practices within organisation or external to the industry

© 2012 GS1

How to Measure Data Quality Self-Assessment Tools

The following self-assessment tools are currently available as part of the DQF:• Questionnaire

• The questionnaire contains 73 questions that relate to an organisation’s data quality management capabilities and their deployment level within the organisation. This is the core component of the self-assessment process.

• Scoring Model • Indication of how many of the recommended best practices for a

DQMS are in place within a given organisation.

• Master Data KPIs • Performance indicators based on the monitoring and inspection

of key GDSN attributes.

© 2012 GS1

Questionnaire

KPIs,

Scoring model

How to Measure Data Quality Self-Assessment Tools

© 2012 GS115

How to Measure Data Quality Self-Assessment – Scope

In defining the scope consider the following:• How will the results of this assessment support the organisation’s

goals?• Factors impacting the assessment:

• Product categories• Product life-cycle (e.g. new introductions vs. line items)• Brands• Specific DQ process and it’s complexity• Type of change in the data/Attributes to be assessed• Manufacturing facilities/locations• Timing• Performance goals• Greatest areas of market interest

• Scope must be clearly defined, documented, and well communicated• Scope definition will determine the size and complexity of the effort –

start small

© 2012 GS1

How to Measure Data Quality Self-Assessment – Considerations

A Self-Assessment…• Is a snapshot in time• Should include a Capabilities Assessment and

Process Review at a minimum• Is not be the ultimate goal; it should lead to further

improvements• Needs to be carefully conducted to ensure the results

are reliable • the DQ Framework and Implementation Guide provide a

detailed process flow for the execution of a self-assessment (see chapter 3 in both)*

* Additional implementation training is available on the DQF

© 2012 GS117

How to Measure Data Quality Self-Assessment – Essential Steps

1. Decide to self-assess

2. Define the scope of the self-assessment

3. Appoint an assessment leader

4. Form an assessment team

5. Educate the organisation about the self-assessment

6. Baseline deployed capabilities

7. Apply the self-assessment questionnaire

8. Consolidate and analyse results

9. Communicate results to the organisation

10.Communicate results to trading partners (optional)

© 2012 GS118

Product Inspection & KPI Model

• Assessment of capabilities and priorities

Identify areas of

opportunity

© 2012 GS119

How to Measure Data Quality Product Inspection – What is it?

Use of a standardised methodology for the physical inspection of a product to: • Verify, objectively, electronic data in comparison with a

physical product or global standard• Ensure the results of the inspection are consistent and

reliable• Monitor KPIs developed to give a more granular

indication of the quality of an organisation’s data output• Track or Audit performance of an organisation and its

data quality management system

© 2012 GS1

How to Measure Data Quality Product Inspection – Scope

• Goals and objectives of inspection• Document accuracy of the data sample• Snapshot of DQMS process

• Methodology• Sample size• Information sources to be compared• Attributes to be reviewed

• Product type • Category

• Location of production• Target Market

© 2012 GS121

How to Measure Data Quality Product Inspection – Essential Steps

1. Select inspection body

2. Prepare for inspection• Define scope of inspection• Identify sample• Gather documents for inspectors

– KPI Model & Scorecard– GS1 Package Measurement Rules and related standards

• Calibrate and Secure measuring equipment

3. Perform inspection

4. Report results

5. Launch appeals procedure• Document complaints

6. Apply corrective measures

© 2012 GS1

KPI Model & Scorecard

• Assessment of capabilities and priorities

Identify areas of

opportunity

“Not everything that can be counted counts." Albert Einstein

© 2012 GS1

How to Measure Data Quality KPI Model – What is it?

A list of Key Performance Indicators (KPIs)• Key:

• Measures a critical business processes • Supports DQ strategy

• Performance Indicator: an objective measure• Internally focused and may be proprietary• Short-term• Long-term• Based on DQ Dimensions

– DQ dimensions list is unique to enterprise– Accuracy - the degree in which the (electronic) product information

stored in a repository is consistent with the physically observable characteristics of the trade item.

© 2012 GS1

How to Measure Data Quality DQF KPI Model

• DQF KPI model was defined to:• Provide trading partners with a neutral, common set of KPIs for data

accuracy• Cover the most commonly synchronised attributes across all regions• Offer a basic structure to validate the effectiveness of data quality

management systems deployed within an organisation

• DQF KPI model includes the following:• Overall item accuracy• Generic attribute accuracy• Dimension and weight accuracy• Hierarchy accuracy• Active/Orderable

• Target % is a business decision

© 2012 GS1

Data quality KPI scorecard

© 2012 GS126

How to Measure Data Quality DQF KPI Model – When to use it

The DQF KPI model may be used for any of the following scenarios:

• As a means to report the results of product audits:– the KPI model is a good way to offer structured results of product

audits as it proposes a logical way to group related attributes.• As a benchmark:

– the KPI model may be used as a reference to compare the accuracy of the data performance of two different entities.

• To track progress on improvements: – the KPI model can be also used to compare the progression of data

accuracy within the organisation by striving always to improve the results obtained every time the KPIs are measured.

© 2012 GS127

How to Measure Data Quality DQF KPI Model – How to use it

1. Customise KPI model• Select KPI categories relevant to organisation• Add or remove attributes to/from the categories• Add specific performance targets

2. Conduct product inspection

3. Analyse using the KPI Scorecard

4. Report results

© 2012 GS1

Implementation plan

Implement improvement

s

© 2012 GS1

How to Measure Data Quality Product Inspection – Results

• Communicate results• Prioritize actions

• Identify the data issues causing the most negative impact on the organisation

• Identify and adjust the processes and capabilities required to correct the negative impact

• Develop process and capacities to prevent reoccurrence

• Define realistic timeframes for execution• Dependent on the extent and complexity of the changes

• Communicate Action Plan

© 2012 GS1

How to Measure Data Quality DQF Phased Implementation

Organisational Capabilities

Policies & Standards Business ProcessSystem

Capabilities

Plan

Executive sponsorship - Mission & vision; Accountable leadership; Staff roles & skill sets; Data owners & stakeholders; Data governance office.

Mission & vision; Goals & objectives; Guiding principles; Success measures; Action plans; Policy & standards management

Initial data entry & setup; Ongoing data maintenance; Processes involved in the information's life-cycle

Unified data repository; Design & architecture; Workflow, user interface; Data validations; Security, access controls; Revision/change history; External publication; Internal publication

Document

Governance organisational structure; Roles & responsibilities; Personal objectives; Reporting alignment

Mission, goals, principles and success measures; Governance model, decision process; Data definitions & standards; Security & use policy; Audit procedures; Documentation standards; Risk Management; Customer feedback policy

Operating procedures; Process flow diagrams; Job aids, work instructions; Performance metrics

System requirements; Operating procedures; Performance metrics

ExecuteEducation & awareness; Internal communication; Training

Education & awareness; Documentation management; Policies & standards management; Data issue management Training; Customer feedback resolution

Education & awareness; Performance management; Process issue management; Change management

*See note on section 2.4.3

MonitorOrganisational capability review; Review of personal objectives

Policy & standards review

Workflow controls; System validations; Performance reporting on service levels; Performance reporting on data quality; External & internal feedback; Process compliance audits; Product measurements; Review & reporting audit results; Monitor impact of erroneous data

Performance reporting on service levels

© 2012 GS1

Continuous Improvement

Monitor changes and plan further improvemen

ts

© 2012 GS1

How to Measure Data Quality Monitoring & Continuous Improvement

Ensuring DQ is a continuous, dynamic, day-to-day activity. • Must be Integrated with the product life-cycle

• Create, Syndicate, Maintain, Archive, Purge

• does not end when a product is published to trading partners • data must be maintained and updated • Periodical audits (both of processes and data) are necessary to monitor

progress.• Most importantly, it needs to ensure that there is good control and

monitoring on performance in order to ensure processes remain in optimal condition.

• Best Practice - “GDSN Trade Item Implementation Guide” Chapter 11: Item Futurisation

• and sustain the corrective action• That is why ongoing monitoring is the key to creating sustainability and

ensuring a continuous level of quality.

© 2012 GS1

How to Measure Data Quality Summary – Elevator Pitch

The DQF is adaptable and scalable and may be applied whole or in part – as required by the organisations goals & objectives

DQF as part of a continuous cycle:• Planning & Strategy• Opportunity identification• Corrective Action & Implementation• Continuous Monitoring

DQF Components• Elements for Data Quality Management System• Model for a self-assessment• Model for a product inspection • Model KPIs & Scorecard• Recommendations for Monitoring & Continuous Improvement

© 2012 GS1

The Data Quality Framework PACKAGE is publically available

• All you need to use the Framework in one package

• Includes:• The Data Quality Framework v3.0• Implementation Guides (user’s manual!)• Automated scorecard for self-assessment• Automated scorecard for KPIs• Data Quality Introductory Presentation• Read me

http://www.gs1.org/gdsn/dqf/data_quality_framework