Upload
dinhlien
View
222
Download
4
Embed Size (px)
Citation preview
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.2
The following is intended to outline our general product direction. It is intended for information purposes only, and may not be
incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and
timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.3
Enterprise Data Quality
Martin Boyd – Senior Director Product Strategy,
Enterprise Data Quality
MDM Partner Summit – May 2014
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.4
Agenda
EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.5
Enterprise Data Quality Growth
• Revenues up 10x in 11 quarters (rolling 12 months basis)
• Completely replaced ODQ (Trillium and Informatica OEMs)
• 14 new EDQ/EDQP deals in NAS last quarter alone
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.6
“Ultimately, poor data quality is like dirt on the windshield. You may be able to drive for a long time with slowly degrading vision, but at some point, you either have to stop and clear the windshield or risk everything.”
Ken Orr, The Cutter Consortium
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.7
• How do you know?
• What is the business impact?
• What should you do about it?
Data Quality – Is Your Data “Fit for Purpose”?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.8
Data Quality Maturity Progression
CHAOS!
DQ Tools
DQ Process/ System
DQ + MDM– Single Domain
DQ + MDM– Multiple Domains
No tools, no process – the Wild West!
Project based – benefits are short lived
DQ ‘Masters’ data in legacy applications with repeatable process
Hubs persist and sync data – leveraged impact
Greater coverage, greater impact
Gover
nanc
e
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.9
Gartner's BI/EPM Maturity Model
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.10
Agenda
EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.11
Enterprise Data QualityIntegrated Solution for All Data Quality Problems
• Process metrics• Quality metrics
• Case Management• Remediation
• Party (individuals, households) match
• Entity match
• Semantic (category) match
• Statistical match
• Match review• Merge/survivorship
• Global parse• Category parse• Extract
• Transform• Address verification
& geocoding
• Substitute • Enrich• Classify
• Statistics• Patterns• Phrases
• Duplicates• Completeness• Max/min values
Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Com
mon
Acc
ess/
UI
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.12
Enterprise Data Quality
Broadest DQ offering• Best of breed capabilities for both Party Data and Product
Data• Profiling, standardization, matching, case management,
governance
Most usable DQ offering• Completely integrated offering – designed to work together• Designed for business and technical users• Transparent operation and results – no black boxes
Pervasive operation for enterprise quality governance • Within legacy systems and MDM Hubs• As part of migration/system load• On data entry/capture• As part of data movement/transfer
Profile
Standardize
Match
Govern
Quickly understand data content
Drive conformance to standards
Identify & merge duplicates
Monitor effectiveness & resolve problems
Com
mon
Acc
ess/
UI
Enterprise DQ Platform
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.13
Typical Customer/Party Data IssuesVariation or
Error Example Variation or Error Example
Sequence errors • Mark Douglas or Douglas Mark Transcription mistakes
• Hannah, Hamah
Involuntary corrections • Browne – Brown Missing or extra
tokens• George W Smith, George Smith, Smith
Concatenated names • Mary Anne, Maryanne Foreign sourced
data• Khader AL Ghamdi, Khadir A.
AlGamdeyNicknames and aliases • Chris – Christine, Christopher, Tina Unpredictable
use of initials• John Alan Smith, J A Smith
Noise • Full stops, dashes, slashes, titles, apostrophes
Transposed characters
• Johnson, Jhonson
Abbreviations • Wlm/William, Mfg/Manufacturing Localization • Stanislav Milosovich – Stan Milo
Truncations • Credit Suisse First Bost Inaccurate dates• 12/10/1915, 21/10/1951, 10121951,
00001951
Prefix/suffix errors • MacDonald/McDonald/Donald Transliteration
differences• Gang, Kang, Kwang
Spelling & typing errors
• P0rter, Beht Phonetic errors • Graeme – Graham
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.14
Typical Product/Item Data Issues
10hp motor 115V Yoke mount
mtr, ac(115) 10 horsepower 115volts
MOT-10,115V, 48YZ,YOKE
This 10hp yoke mounted motor is rated for 115V with a 5 year warranty
10 Caballos, Motor, 115 Voltios
TEAO HP = 10.0 1725RPM 115V 48YZ YOKE MTR
Motor, TEAO, 1725 RPM, 48YZ, 15 Voltios, Montaje de Yugo, hp = 10
Item MotorClassification 26101600Power 10 horsepowerVoltage 115Mounting Yoke
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.15
Urban Myth: Customer and Product Data can use the Same Approach
Party Data (customer, supplier, employee, citizen etc.)
Product Data (part, SKU, item, asset etc.)
Over 30,000 different product categoriesVariations by country/locale
Requires: • Pre-built processes• Pattern-based matching
Name & Address Data• Relatively fixed syntax• Mis-spellings and name equivalents• Verify against postal files
Requires:• Contextual understanding• Ability to learn new domains quickly
Product Data• No fixed syntax – few standards• Infinite variability – format, content, syntax• Different rules for each product category
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.16
Main EDQ Console, Focused on the User
ProjectBrowser
Main Canvas
Results Browser
Tool Palette
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.17
Immediate drill-down to examine real data
Field-level completeness, uniqueness, validity etc.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.18
Immediate drill-down to examine real data
Drill-down to see actual data values and determine
required rules, standards etc.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.19
Immediate drill-down to examine real data
Drill-down to see actual data values and determine
required rules, standards etc.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.20
Build-out Full DQ Process
Process can be ‘simple’ or ‘complex’
Process thumbnail assists in navigation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.21
Key Feature: Pre-built processors • Comprehensive DQ Functionality with a Single User Interface and Repository
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.22
Key Feature: Language, Region and Country-Specific Capabilities
Country and region-specific processors Fully unicode compliant
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.23
Agenda
EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.24
Pricing Objectives Objective: Maximize revenue opportunity from both Tech and Apps with minimal jump-
ball issues– Tech: Sell EDQ into data integration scenarios– Apps: Sell EDQ into application enablement scenarios
Apps Changes (as of March 13th, 2014)– Merge EDQ Parsing & Standardization + EDQ Match & Merge– Merge EDQ Product Data Parsing & Standardization + EDQ Product Data Match & Merge and make EDQ
platform a pre-req– Provide Weblogic & Oracle DB SE as RULs for EDQ
Tech changes (as of March 13th, 2014)– New SKU for Audit and Dashboard– New SKU for Real-time Processing (call EDQ as web service)– Remove requirement for ODI license– Cannot be called from Oracle applications (MDM, EBS, etc.)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.25
EDQ Pricing Changes on Apps PL
Customer/Party data
EDQ Product Data Match and Merge
EDQ Match and Merge
EDQ Product Data Parse and Standardize
EDQ Parse and Standardize
Product/Item data
• Includes Connectors to Customer Hub (UCM) and Product Hub
• Structure is 1:1 replacement for Informatica OEM
$150k
$125k
$150k
$125k
Oracle Confidential 25
EDQ Product Data Extension$150K
• EDQ Product Data Extension • Combines EDQ Product Data Match and Merge and
EDQ product Data Parse and Standardize• Has pre-req of EDQ Parse and Standardize
EDQ Profile & Audit$150K
EDQ Profile & Audit$150K
ED
Q A
ddre
ss V
erifi
catio
n $6
3.3K
ED
Q A
ddre
ss V
erifi
catio
n $6
3.3K
EDQ Standardize and Match $275K
New/Changed SKU
Apps PricelistGo-forward
Apps PricelistPrevious
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.26
Tech & Apps Comparison
EDQ Standardization and Match$275k
EDQ Profile & Audit$150k EDQ Profiling for Data Integration
$100K
EDQ Batch Processing for Data Integration
$100K
Apps Pricelist
• Includes Application Connectors• CRM/UCM/Product Hub/Endeca etc.
• Includes option for EDQ Product Data Extension for deep product data capabilities
• Batch & Real-time SKUs are restricted:• Cannot be called by Oracle App (sold off Apps PL)• Cannot return real-time call to Oracle App user• Do not include App Connectors
• EDQ Real-time Processing SKU requires PM approval (CA)
EDQ Real-time Processing for Data
Integration (CA)
$100K
EDQ Audit & Dashboard for Data Integration $50K
Changed SKU
New SKU
EDQ Product Data Extension - $150k
ED
Q A
ddre
ss V
erifi
catio
n $6
3.3K
ED
Q A
ddre
ss V
erifi
catio
n fo
r D
ata
Inte
grat
ion
$63
.3K
Tech Pricelist
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27
Is this deal a good fit?
Tech: Sell EDQ into data integration scenarios
Tech should sell if:– EDQ is standalone, or being called
from a DI system (ODI, OGG, SOA, competitor ETL, etc.)
Hand over to Apps if:– Calling EDQ from an Oracle application
such as Siebel, MDM, EBS, etc.
Apps: Sell EDQ into application enablement scenarios
Apps should sell if:– EDQ is standalone or enabling an
Oracle application (something sold on Application pricelist – EBS, CRM, MDM, etc.)
Hand over to Tech if:– It’s all about ODI and data integration
with no apps impact
Note: A Tech license can be migrated to Apps using a net-net-min 10% method – so this will NOT be free and will involve admin hassles for your customer, but should not be a big issue if there is license expansion involved
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.28
Pre-Requisites & RULs – Apps
• Arrows show inter-module pre-reqs• Everything builds from Profiling
Other details:• EDQ includes a restricted use
license of Weblogic Basic and Oracle Database EE
• EDQ and EDQ Product Data Extension can be installed on the same server, but this is not certified and careful though should be given to potential resource conflicts. Should be evaluated case by case.
• Servers can be segmented using OVM (EDQ works under VMWare, but Oracle does not respect it for licensing purposes)
• Oracle charges for all processors (live, QA, Development, etc.)
• EDQ runs on EXA*
EDQ Standardization and Match$275k
EDQ Profile & Audit$150k
EDQ Product Data Extension - $150k
ED
Q A
ddre
ss V
erifi
catio
n $6
3.3K
Requires
Requires
Requires
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.29
Agenda
EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.30
• How do you know?
• What is the business impact?
• What should you do about it?
Data Quality – Is Your Data “Fit for Purpose”?
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.31
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.32
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Poor data quality is the most common source of project delay
and failure – but it is easily avoided with systematic data
preparation
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.33
Why Focus on System Migration/Consolidation?
• More than 80% of data migration projects run over time and/or over budget. Cost overruns average 30%. Time overruns average 41%.Bloor Group
• 80% of organizations… will underestimate the costs related to the data acquisition tasks by an average of 50 percent”Gartner
Because poor data biggest single risk to the project
33
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.34
Case Management for Governance
Usage• Cases/alerts are assigned a work queues and a priority• Data specialists sign in and review/resolve issues• Management reports allow monitoring of work queues
and productivity• Helpful for
o One-time cleanse/migrationo Ongoing governance program
Features• Hierarchical Case/alert functionality• Configurable Workflows• Automatic prioritization of cases/alerts• Timers• Email Notification Support• Comprehensive audit trail• Immediate ad-hoc reporting
Review and resolve exceptions from the DQ process
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.35
Data Prep for System Migration/Implementation
EDQ Process
Apps and hubs
Governance and Case Management to ‘Perfect’ Data
• DQ Insight (Dashboard)• Reporting• Trend Analysis• Case Management• Workflow• Remediation
Legacy Data ‘Fit for Purpose’ Data
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.36
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Even after initial data preparation, ongoing data governance is
required
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.37
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)Business Intelligence is perhaps the
most obvious example of garbage-in, garbage-out
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.38
Impact of Data Quality on BI
“Only 30% of BI/DW implementations fully succeed. The top two reasons for failure? Budget constraints and data quality.”
Gartner
“More than 50 percent of data warehouse projects will have limited acceptance, or will be outright failures, as a result of a
lack of attention to data quality issues” Gartner
“It does not really matter how good your management sponsorship or your business-driven motivation is. If you do not have the data, or the data does not have sufficient quality, any
BI implementation will fail.” Kimball
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.39
EDQ for DW/BI1. ODI or GoldenGate move data into DW2. ODI maps data into Staging tables3. EDQ ‘cleanses’ data4. ODI moves from staging into production tables
Data in DW is ‘Fit for Purpose’BI/AnalyticsSource of truth
ProductionTables
Data Sources
BI/analytics
Single source of truth for reference
StagingtablesODI
EDQParsing, Cleansing,
Standardization, Matching
Replicasourcetables
ODI
Golden Gate
ODI
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.40
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Sometimes best practices are mandated by law and have large
penalties for non-compliance (FATCA, OFAC, etc.)
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.41
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Even where hub implementation is a core objective, DQ often comes first
• To prepare the way• Lower implementation risk • Deliver tactical ROI
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.42
Enterprise Grade MDMModern. Purpose-Built. Multi-Domain. SaaS.
Cus
tom
er
Sup
plie
r
Ass
et
Pro
duct
Loca
tion
Em
ploy
ee
Fina
ncia
l
Oracle MDM
EPM/BI
CRM
Legacy…
CRM
Other Sources
ERP
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.43
Common Data Quality Use Cases
System Consolidation/Migration• Enforce new system standards
on legacy data
Compliance• Drive consistent data and processes to
meet regulatory requirements (watchlist screening, anti-money laundering, tax compliance, etc.)
Enterprise DQ Services • Common DQ services to clean-up and
govern application data across multiple applications
Business Intelligence Enablement• Enforce BI standards on disparate
data
MDM Enablement • Verify, standardize, match and
and merge data from disparate sources
Application Enablement• Clean-up and govern application
data for a single application (CRM, HR, PLM, Retail search, etc.)
Enforcing common DQ standards across the enterprise is a simple,
tactical fix to systemic problems and a good foundation for enterprise
governance
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.44
EDQ Web ServicesEnforce common DQ standards across the enterprise
Common Services
Applications App 1 App 2 App 3
Library of enterprise standard DQ services
Any EDQ process may be called as a real-time web service
Call any process from any application to
1. Enforce common standards
2. Minimize architectural changes
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.46
Governance Vision in ActionLibrary of enterprise standard DQ services
Data Steward “Cockpit”
• DQ and Process Visibility• Rules Management• Case Management
Case Management
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.47
Agenda
EDQ Opportunity for Partners EDQ Product Overview EDQ Pricing Changes Typical EDQ Use Cases Roadmap
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.48
Key Investment Drivers Cloud
– Cloud DQ services– DQ capabilities built into Fusion Apps– Cloud DQ governance enablement
Governance II– Improved DQ issue lifecycle support
Integration of DQ and DQ Services– Oracle Middleware – Oracle Applications– External Applications
N.B. Several mandatory requirements for cloud adoption, including full HA/Clustering, Elastic provisioning support etc.
Copyright © 2013, Oracle and/or its affiliates. All rights reserved.49
Q&A
Martin Boyd – Senior Director Product Strategy, Enterprise Data [email protected]