Upload
others
View
6
Download
0
Embed Size (px)
Citation preview
© 2008 IBM Corporation
MDM Architecture and Implementation Best Practices
Sadagopan Krishnamachari
2
Agenda
Fragmented Data Problems …
Master Data & Master Data Management
IBM Infosphere Master Data Management Server
MDM Server Implementation Pattern
MDM Architecture
MDM Data Stewardship Application
MDM Search / Matching Algorithm
3
Islands of InformationReduce customer satisfaction, decrease revenue, hinder relationships
DocumentManagement
Michael Johnson
Mortgage.tif ERP
JP Morgan, USA
Cust ID : JP003
CRMCall Center
Mike Johnson
JP Morgan Chase
Last Interaction: 4/11/03 (product not
received)
Retail Pointof Sale
JP Morgan & Chase
Contact : Michael A Johnson, CIO
270 West St, NY
Portal
Michael Johnson
User ID: Mjohnso
! Personalized access
! Gold Customer
! Sub: Newsletter 1
CRMMarketing
Michael Johnson
! Opt-Out flag
! No Promotion flag
Application Forms
PaymentsService and
Support
Online Registration
In-StoreInteractions
DirectMail Response
DataWarehouse
Michael P Johnson1400 54rd Avenue
NY NY212 995-3345
3rd PartyInformation
Internet Commerce
Michael Johnson
User id :mjohnson
JP Morgan
Contract:: JP987
Online Purchases
7
4
Coffee Beans GTIN 20012294219421
CANCode : 21204
USACode : 21192
BR, CR, MEXCode : 21186
CH, AUT, DE, UK, FR, BEL, NL, IT : Code : 21184
DE, FIN, SWE, NOR, ESP, POR, Code : 21190
BUL, YUG, CR, RO, SLOVCode : 19616
ISRCode : 21204
MidEaCode : 21204
World TradeCode : 19619, 19616
AUSCode : 21190
HK, TAI, SIN, MAL, S.KORCode : 21188
JAP, THAI, INDO, PHICode : 21189
CZ, LIT, EST, SLOV, RUCode : 2002494
Inconsistent Master Information is a Major HurdleImpacts Revenue, Cost, Agility and Compliance
ARGCode : 21184
Gaining control over product information results: Errors in data – 30% of data in retailers systems is wrong Lost productiv ity – 25 minutes manual cleansing per SKU, per year Slow time to market – 4 weeks to introduce new products Inv oice deductions – 43% of invoices result in deductions Failed scans – up to 70,000 per week (1 large US Retailer) Lost sales – up to 3.5% per year
Source: A.T. Kearney, GMA, AMR
Industry Driv ers: RFID, Waste Electrical and Electronic Equipment Recycling, Product Information Exchange Standards, Return of Hazardous Substances, Global Data Synchronization, Sarbanes Oxley, etc. (Yankee Group, 2005)
6
5
Master Data
Facts describing the core business entities: customers, suppliers, partners, products, materials, bill of materials, chart of accounts, location and employees•The high value information an organization uses repeatedly across many business processes•Generally used across multiple LOB
Master Data Management
Master Data Management (MDM) is the set of disciplines, technologies and solutions to create and maintain consistent, complete, contextual and accurate business data for all stakeholders across and beyond the enterprise
6
MDM Repository / Hub is analogous to a Version Control System, albeit for your Master Data
7
MDM Server Implementation Environment
MDM Domains
Business Services
Pre-buil t Customi zable
Data Stewardship
Party Account Product Custom
Integratio n
Content Data Analy tics
InfoSphere MDM ServerInfoSphere MDM Server
MDM Domains
Business Services
Pre-buil t Customi zable
Data Stewardship
Party Account Product Custom
Integratio n
Content Data Analy tics
InfoSphere MDM ServerInfoSphere MDM Server
RDBMS
Security,Mail, etc
Real-time/Near-Real-Time Connectivity Services (ESB, EAI, Web Services, MQ, etc.)
Enterprise Data Integration (InfoServer)
Billing/Provisioning
Call CenterW eb Phone Sales Vendor & Other Business Partners
Management
Service
Master Data Batch Load
External Data Providers (e.g. D&B, ACXIOM,
Experian)
EnterpriseData Warehouse/ Data Mart
Corporate & Others
Content Management
Process Server
Understand Cleanse Transform Deliv er
Order
Customer Product Account Others
Customer Product Account Others
Customer Product Account Others
Customer Product Account Others
New Systems (e.g. SOA –based)
(e.g. Siebel) (e.g COGNOS)Real-time
IBM InfoSphere MDM Server
8
Understand Cleanse Transform DeliverUnified Deployment
ETL Tooling
Unified Metadata Management
The Complete Picture – Multi Form MDM Server
Multiform MDM manages data
domains critical to business processes
Multiform MDM leverages merged,
cleansed and standardized
data via the ETL
IBM Master Data Management
Banking Insurance Government Healthcare Retail TelcoFocused on critical
information intensive business problems
Indu
stry
Mod
els
& A
sset
s
Multiform Master Data Management
Collaborate Operationalize Analyze
Party (Customer, citizen, prospect, organization, supplier, distributor, etc.)
Product (good, service, product bundle, catalogue, product component, etc.)
Account (Agreement, financial account, reward program, etc.)
9
A dminWeb A pp
DataStewardship
Web A pp
WebServicesA dapter
BatchP rocessor
EventManager
ESB / MQ / EA I Broker
Dashboard/Portal U Is
C lientA pplications
MDM Consumers
Service Controller P arser
C onstructor
Request Framework
Bus iness Transaction Manager
Bus iness Proxies
MDM Core
UtilityComponents
Bus iness Logic C omponents
ExtensionC ontroller
Java Classes
RuleSets
RulesEngine
Pre/PostTxn
Pre/PostAction
Extension Framework
Common Components
Standardization
Audit Data
Metadata
ErrorMessages
Data-LevelEntitlements
MetaData
Event Manager
P erformanceT racker
ErrorMessaging
Logging
T ransaction AuditInformation Log
ExternalV alidation
ExternalBus iness Rules
Rules ofV isibility
C onfigurationManager
RulesEngine
Logs
ARMAgent
ConfigurationSettings
ValidationRules
XMLC ompositeT ransaction
Handler
Java Classes
RuleSets
MessagingA dapter
MDM Product Component Common Component 3rd Party Product Client ComponentMDM 8.0Jan, 2008 Significant Pluggable Component
NotificationRequest Handler
C ontroller Components
BusinessServices
AdminServices
PartyServices
ContractServices
HistoryServices
FastTrack
Security
Events
Validators
EvergreenP rocessor
Suspect ProcessingComponents
Notifications
Adapters
Web Services Asynchronous Synchronous Client-Defined Interface
Data
W C C C o r e
OperationalTables
History Data
HistoryTables
Rule Data
CodeTablesData
ExtensionTables
Behavior Extensions
Data Extensions
JMS Topic
ExtensionToolkit
10
MDM Workbench – A Model based Data Model
11
Data Stewardship Application – Duplicate Suspect Processing
12
Step 1: Optimizes data for statistical comparisons– Normalizes & compacts data, creates derived data layer
source data remains intact– Phonetic equivalences, tokenization, nicknames, etc.
Step 2: Finds all the potential matches– Casts a wide net – all matches on current or historical attributes,
prevents misses– Partial matches, reversals, anonymous values, etc.
Step 3: Scores accurately via probabilistic statistics– Compares attributes one-by-one and produces a weighted score
(likelihood ratio) for each pair of records– Frequency weights specific to your business– Edit distance, proximity of match
Step 4: Custom threshold settings– Single or dual threshold models– Link, don’t link, don’t know – “learns”
from manual input
Manual reviewManual reviewLowestpossiblescore
Lowestpossiblescore
Highestpossiblescore
Highestpossiblescore
Don’tlinkDon’tlink LinkLink
Lowestthreshold
Upperthreshold
Should be linked
Should not be linked
Name122222223334
Data Derived Hash Buckets
Robert RBT 121213444
Potter PTR 34839020
ZIP345666665435
SSN342133333555
7 9
8.63
Matching and Searching Algorithm
13
14
Extras
15
Useful Links IBM Information On Demand website
(http://www-306.ibm.com/software/data/information-on-demand)
IBM Information Management website (http://www.ibm.com/software/data)
Extreme Leverage website
(http://w3-103.ibm.com/software/xl/portal/viewcontent?type=doc&srcID=XT&docID=B329727F31168I32) Customer Success Stories
(http://www-306.ibm.com/software/success/cssdb.nsf/topstoriesFM?OpenForm&Site=db2software) Information Management Demos
(http://demos.dfw.ibm.com/solutions/infomgmt/) Information as a Service Demo
(http://media.dvdpowertools.com/ibm/infomanagement/interface.php) Videos
(http://www-306.ibm.com/software/info/television/index.jsp)