Master Data ManagementFrom Assessment, Design up to operation

Ali BELCAID – Managing Consultant

Master Data Management : An OverviewInformation is a Priority

Quality and actionable information is fundamental to deliver many business strategies.

MDM

Enterprise OperationsManagement & Capabilities

Enterprise InformationManagement & Capabilities

Solutions• ERP• CRM• Supply Chain

Solutions• BI/DW• BPM• Portals

MDM is the glue that blends operational and informationmanagement solutions

Analytical MDMOperational MDM

Master Data Management : An OverviewMDM Requires Both IT and Business

MDM is a component that promotes process efficiency, simplicity, and data quality, improving the value IT brings to business.

Global Master Data Management

Business IT

EnterpriseWideConsistency

CostEffectiveness

ReliableAnalytics andReporting

Centralized, Efficient Data Storage

Improved System Integration

Minimal Data Conversions

• Avoid data redundancies• Assure data consistency

• Centralize data distribution (one source)

• Provide unique identifier• Create global hierarchies

and attributes

Impact of MDM Initiative on Business and IT

Master Data Management : An OverviewMDM Implementation Styles

Implementation Style Description

External Databases (Service Provider)

Third party suppliers and managers of domain specific master data

Examples: database marketing, government service bureaus

Persistent(Database)

Master information file/database, system of record (SOR) Operational data store, active data warehouse Relational DBMS + extract-transform-load (ETL) + data

quality (DQ)

Registry(Virtual)

Metadata layer + distributed query (e.g., EII) Enterprise application integration (e.g., EAI), distributed

system Portal

Composite(Hybrid)

Ability to fine-tune performance and availability by altering amount of master data persisted

XML, web services, service-oriented architecture (SOA)

Master Data Management An Overview “Persistent” Master Data Repository

(Illustrative Scenario)

This Customer Data Integration (CDI) solution architecture illustrates how process and technology work together through a centralized “persistent” master data repository

Customer Master Data RepositoryBusiness AnalystsData Stewards

Workflow Business Rules Mapping Rules

Automated Entry Updates

SAP

Siebel

IMSCustomer

PBMS

Extr

act T

rans

form

Loa

d(E

TL)

Ente

rpris

e Ap

plic

ation

Inte

grati

on(E

AI)

OperationalData Store(ODS)

DIM

DIM

DIM

DIM

DIM

DIM

FACT

Data Mart

Data Mart

Data Mart

CareReporting

CustomerReporting

FinancialReporting

Customer Care

CRM

CampaignManagement

ContractNegotiations

FinancialConsolidation

Monthly EndClose

Enterprise Warehouse Data Marts

Aggregate

DATA INFORMATION

Map

ping

Up

date

System Owners

Initiate Entry/Update

Initiate Entry/Update

Operational Systems Master Data Management Business Process

EvaluateRequest

ApproveRequest

Catalogue/Index

Approach to MDM ImplementationBusiness Assessment & Technology Selection

Quick scan

Current State Details

Requirements

Gap Analysis

Organization & Governance

Process & Methodology

Technology Selection

Implementation Roadmap

Current State Future State Develop Roadmap

Deliverables

Project Initiation Current State : Data Mgt. Current Sate : Organization Current State : Architecture

Gap Analysis Future State : Data Mgt. Future Sate : Organization Future State : Architecture

Prioritization Roadmap

4 to 6 weeks varies with scope

Activities

Approach to MDM ImplementationBusiness Assessment & Technology Selection

Topic What to Do ? What to Deliver ?

Quick Scan

• Maturity Assessment• Business Direction, objectives, …• Engagement Management (project

management, change management, quality management and risk management )

• Scope of the Project : Which MDM should be implemented ?

Business Requirement Analysis • workshops with key client stakeholders to

identify business issues with master data (Data quality, Duplicity, Incoherence, …)

• MDM Finding & AssessmentTechnical Requirement Analysis

• workshops with key client stakeholders to identify technology issues that could delay the delivery of accurate and reliable master data to consumers (multi-systems, duplicity, non-synchronization, …)

Gap Analysis • based on business and technical findings and requirements

Future State Recommendations • that consists of business, technology and data architecture

Roadmap Definition • for attaining the future state • Roadmap Definition & Planning

Approach to MDM ImplementationBusiness Assessment & Technology Selection

Maturity Assessment

Approach to MDM ImplementationMDM Implementation Framework

Business Requirement

Technology Assessment & Software Selection

Design

Build

Integrate

Operate

Roadmap & Foundation Activities

Iteration 1

Iteratio

n 2

Iteratio

n n

Continues Implementation Phases

Begin next Iteration

This part is done once(Part of the Assessment Phase)

Define & Validate the data governance & operating model Implement the data governance & Operating model

Kick

Off

of th

e M

DM

Initi

ative

Approach to MDM ImplementationRoadmap & Foundation Activities

The Roadmap provides the detailed requirements and solution definition that applies to the continuous implementation. It has the following objectives:

Refine strategic business requirements to a detailed level for iterative design

Establish standards and develop solutions to common problems Define the development and delivery environments Detailed planning for this cycle of the implementation

The Roadmap can be summarized as providing the Plan, the Solution Requirements and the Solution Definition for the continuous implementation part.

Foundation Activities focus on aspects of each of the streams of development. These activities are :

Meta Data Management Data Modelling Data Migration Data Integration Data Reengineering Data Profiling Data Solution Architecture

Master Data Modelling

Master Data Migration

Master Data Integration

Master Data Re-engineering

Master Data Profiling

Meta Data Management

Master Data Architecture

Major deliverables and points to be addressed when setting up the roadmap & foundation activities :

Detailed Project Roadmap Testing and Deployment plans Detailed Information Modeling Detailed Migration Plan (historical Data) Recommended process and system changes for improved

Data Governance Identification of root causes leading to Data Governance

issues Data Governance Metrics Quantitative Data Investigation Improved Data Quality Create/Revise Solution Architecture Ensure the availability of Software Development

Environment

Approach to MDM ImplementationMDM Work streams

MDM Program Management

Change/Issue Management

Operations Management

Training and Support

Meta Data Management

Master Data Modelling

Master Data Migration

Master Data Integration

Master Data Re-engineering

Master Data Profiling

Master Data Architecture

OperateIntegrateBuildDesign

Iteration

Approach to MDM ImplementationMeta Data Management

Significant metadata artifacts are produced related to data definition, business rules, transformation logic and data quality. This information should be stored in a metadata repository; getting this repository in place from the early stages of the MDM project.

Model Management is the capability to manage structures and processes used to describe the metadata in a system.

Metadata Integration capability provides a basic ability to build metadata flows into and out of a managed metadata environment.

Identity Matching as a foundation capability ensures consistent and accurate reuse of metadata. a system must have the ability to identify metadata uniquely so that the metadata may be reused, validated and versioned within the managed metadata environment.

Validation capabilities ensure the quality and consistency of metadata flowing through the managed metadata environment

Versioning of metadata provides the ability for looking back into history to gain a more comprehensive understanding of the current state

Configuration Management is a fundamental process for developing metadata. It is the role that process and governance plays in the development and operations of a managed metadata environment.

Model Query provides the fundamental ability for publication of metadata. Its capabilities form the foundation of providing Metadata Reporting Packages

Metadata Access Control is a capability for providing a control layer over metadata models. Metadata can often be sensitive information that should have restrictive controls to prevent unauthorized access

Approach to MDM ImplementationMaster Data Modelling

The data modeling process is used as an intermediary data store to bring data together from multiple systems in a hub fashion. This data store provides a common, integrated model where data may undergo significant re-engineering.

Implement Physical Master Data Model

Design Logical Master Data Model

Input: Conceptual Data Model Data Specification Standards Data Modeling Standards Data Security Standards Detailed Business Requirements for

each Iteration

Output: Logical Data Model

Input: Logical Data Model Solution Architecture Data Specification Standards Data Modeling Standards Data Security Standards Detailed Business Requirements for each iteration

Output: Physical Data Model Database Definition Language (DDL) Scripts Sizing Estimates

Approach to MDM ImplementationMaster Data Integration

Input: Business requirements Designed Process Flow Source & Target interfaces load dependencies and integration with

metadata processes Source & Target Data models

Output: ETL Logical Design

Input: ETL Logical Design Solution Architecture Data Specification Standards Data Modeling Standards Data Security Standards

Output: ETL flows and Jobs

ETL flows &jobs Testing

ETL Physical Design

ETL Logical Design

Input: Test scenarios Data Sampling

Output: Tested flows and jobs

Dependencies: Metadata Management Data Profiling Data Re-engineering Data Modeling Data Migration

Data Integration is one of the Foundation Capabilities of MDM Development. It provides a mechanism for bringing together information from a number of distributed systems by interfacing into sources, providing a capability to transform data between the systems, enforcing business rules and being able to load data into a different types of target areas.

Approach to MDM ImplementationMaster Data Re-engineering

Data Re-Engineering is a term used to describe a number of related functions for standardizing data to a common format, correcting data quality issues, removing duplicate information/building linkages between records that did not exist previously, or enriching data with supplementary information.

Data standardization brings data into a common format for migrating into target environment. It addresses problems related to:

Redundant domain values Formatting problems Non-atomic data from complex fields Embedded meaning in data

Data Correction typically addresses problems related to:

Missing data Value issues due to range problems Value issues related non-unique fields Temporal or state issues Name and address data that can be referenced against

existing reference sets

In the Data Matching and Consolidation task, data is associated with other records to identify matching sets. Matching records can then either be consolidated to remove duplications or linked to another to form new associations.

Data Enrichment provide an organisation’s internal data with data from external sources like :

Personal data such as date-of-birth and gender codes Geographical data Postal Data, such as Delivery Point Identifiers (DPID) Demographic information Economic data World event information

Approach to MDM ImplementationMaster Data Profiling

Data Profiling focuses on conducting an assessment of actual data and data structures. It helps provide the following:

Identifies data quality issues - measurements are taken against a number of dimensions, to help identify issues at the individual attribute level, at the table-level and between tables.

Captures metadata. Identifies business rules – The next step is to perform the data mapping. Data profiling will assist in gaining an understanding of

the data held in the system and in identifying business rules for handling the data. This will feed into the future data mapping exercise.

Assesses the source system data to satisfy the business requirements. The focus is on gaining a very detailed understanding of the source data that will feed the MDM target system, to ensure that the quality level is sufficient to meet the requirements.

Major Deliverables

Data Quality Assessment Report (per Source System) Data Quality Metrics updated to Metadata Repository Mapping Rules and Business Rules updated to Metadata Repository

Finalize Data Quality Report(Signoff of Data Quality Report)

Perform Multi Tables Profiling

(Analyze redundancy and referential integrity issues)

Perform Table Profiling

(Analyze Data across rows in

single table)

Perform Column Profiling

(Analysis of single or complex field)

Approach to MDM ImplementationMaster Data Profiling

1.Column Profiling

2.Table Profiling

3.Multi-Table Profiling

4. Quality Report

Input: Completion of Table Profiling Information Requirements for multi-table level data

analysis Relevant data extracts Output: Completion of Multi-Table Profiling Redundancy Analysis will identify:

Potential relationships with fields in other tables Redundant data between tables Potential referential integrity issues eg.

Identification of orphans records Completion of the relevant sections of the Data Quality

Assessment Report Updates to metadata repository

Input:

Information Requirements for column-level data analysis

Relevant data extracts

Output: Completion of Column Profiling Understanding all the fields and document their

descriptions in the profiling tool Completion of the relevant sections of the Data Quality

Assessment Report Updates to metadata repository

Input:

Completion of Column Profiling Information Requirements for table-level data analysis Relevant data extracts

Output:

Completion of Table Profiling Understand all the fields and document their

descriptions in the profiling tool Primary keys for each table Completion of the relevant sections of the Data Quality

Assessment Report Updates to metadata repository

Input: Completion of Column Profiling Completion of Table Profiling Completion of Multi-Table Profiling

Output: Completion of the Data Quality Assessment Report

Data Producers (ERP, CRM, Legacy, …)

Approach to MDM ImplementationMaster Data Migration

An MDM program will typically involve a migration of historical data across systems, into or through a centralized hub. This is where many of the data quality issues are resolved in a progressive fashion before operationalizing some of these rule-sets for the ongoing implementation.

Migration Staging

• Attribute Scan• Tables Scan• Assessment• Reporting

Transformations

Dat

a In

tegr

ation

Prod Target

Test Target

Data Profiling Data Integration

Data Re-engineering

Metadata Management

Integrated Data Store

• Common Data Model• Detailed Data• Apply Re-engineering rules

Master Data Modelling

1

2

3

4

5

6

7

Approach to MDM ImplementationMaster Data Migration

The key activities in the MDM migration process include:

1. Extraction of data from producers (ERP, CRM, Legacy systems, …) into a staging area.

2. The data in the staging area will be profiled to measure down columns, across rows and between tables. This information will be used to determine which business rules and transformations need to be invoked early in the process.

3. Metadata such as data mapping rules will begin to be established at this time. Data Standards will be agreed to and invoked at this stage in preparation for data movement. All source attributes will be mapped into the target attributes within the metadata management environment.

4. All agreed to transformations and standardizations required to move the data into the staging area for testing and production are implemented. The data is moved into the Integrated Data Store.

5. Data Profiling is done again and measured against the agreed upon move success criteria for all steps up to this point. Additional data standardizations are performed in to assist in the data matching and generally measure data quality against agreed upon criteria. After the standardizations the rules for which records can not or should not be moved are applied. It expected that this step will require considerable analysis.

6. This step involves the actual move of the data into either the testing environment

7. Data is loaded into the production system where some further data quality cleanup may be required. Production Verification Testing is conducted, which should also include functional testing of features that are environment specific. After testing is complete, the system is activated as a live production system.

Approach to MDM ImplementationMaster Data Architecture

The Master Data Architecture defines in detail the Solution Architecture for the MDM environment. The Solution Architecture provides the overall technology solution for a specific increment and ties together the overall approach.

Define Data Quality Processes- Data model-Profiling- Re-engineering

Deliverables:• Data Quality Design architecture• Data Quality Implementation

Software Documents• Data Quality Technical architecture

document

Define ETL conceptual Design- List of sources- List of targets- Major Transformations- Estimate volumes- Timing

Deliverables:• ETL Design architecture• ETL Implementation Software

Documents• ETL Technical architecture document

Define Metadata Management conceptual Design- Business definition of the data - Physical data models - Data Re-Engineering metadata -Data Quality metadata formulas used to derive data

Deliverables:• Metadata Design architecture• Metadata Implementation Software

Documents• Metadata Technical architecture

document

Define Security conceptual Design- Security Standards-Security Requirements

Deliverables:• Security Implementation Document

Define Infrastructure Management conceptual Design-Backup & Recovery-Archiving-Controlling & Monitoring- Environments (dev, test, prod) setup

Deliverables:• Configuration Management Document

Define SDLC conceptual Design- Testing Strategy - SDLC Procedures - Testing Plans for Applications & Infrastructure-Deployment Plan

Deliverables:• SDLC procedures document• Testing Plan• Deployment Plan

Master Data Solution Architecture

MDM Software Implementation -Software Implementation Planning- Parameterization/Configuration- Software Testing and deployment

Deliverables:• Software Installation and Configuration

Document

Approach to MDM ImplementationPrototyping the Architecture

Prototyping the architecture helps to :

• test some of the major technology risk areas for the proposed MDM Solution Architecture• gain a better understanding of how the solution will work before moving into a more formalized

design process. • Prototyping the proposed solution should provide an end-to-end approach that includes each of

the major components of the architecture.

Key Lessons

Joint business and IT team

Make the case for change

Data as a common good

Think big but start small

Measure and communicate success

Processes first, technology last

Business ownership of data

Roles and responsibilities

Data cleanliness and migration

Communicate, communicate, communicate !

Approach to MDM ImplementationMDM - Key Lessons Learned

In an MDM implementation, there are some key lessons learned that should be considered when initiating an MDM program.

Knowledge, is quite simply question of sharing.

http://intelligenteenterprise.blogspot.com/http://www.linkedin.com/in/albel