Embed Size (px)
DESCRIPTION
Third set of 5x5 Speed Presenting Updates: 1) Selling Data Vault - Elwyn Lloyd Jones 2) DV as Leverage for Data Migration - Antoine Stelma 3) QUIPU in Banking - Juan-José van der Linden 4) Volvo Data Vault Automation - Frederik Naessens 5) Are we ready for a DV Conversion Standard - Frederik Naessens, Stijn Roelens, Kristof Vanduren
Citation preview
Selling Data Vault
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Elwyn Lloyd Jones5 June 2014
Sevenoaks Systems [email protected]
Types of Argument
1. ad Verecundiam / ad HominemNSA, US Army, JP Morgan, ABN AMRO, KPN, Bank of England, Volvo, Mercedes, Volkswagen, Bill Inmon
2. ad Populum800 organizations
3. ad BaculumAudit-able system of records
4. Reducto ad Misericordiam Throw away and start from scratch Source integration => Enterprise crash
5. Up and UnderWhile client watches the BI - we’ll watch the DWH
6. Scallability (cost) – busting Moore’s LawNo columnar Netezza, Exadata, etc.
Data Vault as leverage for Data Migration
Exploring the benefits
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Antoine StelmaJune 5 2014Presentation
Centennium [email protected]@antoinestelma
Challenges
Client: • wants new ERP• 10 old source systems• own plant with intensive processes• MD/MT data is spread• unify & simplify architecture• business process changed
Quest
A data warehouse that:• helps speeding up and unifying data migration• contains current data and processes• is a solid base for a new ERP • unifies master & meta data
Two scenario’s
Source Source
Source SourceE D W
Source Source
Source Source
E D W
ERP
Module 1
Module 2 Module 3
Module 4
Module 5
Migrated data ETL
Source/MD/MT
Classic approach
Suggested approach
ERP
Module 1
Module 2 Module 3
Module 4
Module 5
Migrated data
ETL
Organization
How?
• scope of data for migration• type DV • collect master & metadata in H+S• unify & data cleaning • relations/links/etc. • alignment between old source and new ERP
Lessons (to be) Learned
#lesson1: An EDW can help you with data migration by using DV#lesson 2: EDW does not have to wait for full ERP implementation#lesson 3: It’s all about Data! # be creative
in Banking
Enabling market risk data analysis
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Juan-José van der LindenJune 5, 2014
[email protected]@OS_Quipu
Bank Example
- Prudential Regulation Authority (PRA) - Prevent new financial crisis- Regular collection and storage data of firms’
trading book positions- Simulate risk scenarios
High-level requirements
Deliver an effective stress testing capability:
– Report stress results with material accuracy and on a timely basis
Requires an industrial-strength infrastructure for data collection, risk analysis and reporting
Challenges:Need for a flexible data repository:
– Effective data management (model changes)
– Facilitate historical data analysis – Facilitate data provenance
management
Architecture
Metadata
Data
Legend
Oracle data modeler
SQL Server
SSIS/SSAS/SSRS
Quipu
Tableau
Source data files
Matadata flow
Data flow
Based on
Stg DV 3NFETL
Src
SrcBM
xls Stg DV 3NFSSISSSIS
SSAS
SSIS
SSRS
Tableau
Proof of Concept and Project
Three-week proof of concept:- 1 week business data model support- 1 week data warehouse development - 1 week demonstrating changes and developing
Tableau reporting
Project support:- Model validation- Quipu training- Architecture support and validation
Ongoing:- Quipu product support as part of software
license
Takeaways
# 1 – Invest in getting the input 3NF business model.# 2 – Automation requires good data models.# 3 – Model and code generation drastically speeds up projects.# 4 – Generation tooling needs to support model changes. # 5 – Business value is made visible at reporting level.# 6 – DV modeling is an enabler, not the goal.
Volvo DV Automation
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Frederik NaessensJune 5 2014In cooperation with Stijn Roelens & Kristof [email protected]
Volvo: criteria Automation Tool selection
Agile way of working– Handle migration between DEV-QA-PROD– Handle parallel development
Automation– Perform automatic data migration when data
model is modified– Automatically generate ELT code
Respect industry standards– Be netezza compliant– Generate a 100% Data Vault compliant data
model
Fluent interaction between business & it– Provide or interact with a business data model
3D – Configure extended properties
3D – Use predefined but flexible Model Conversion rules
3D – Configure extended properties
Are we ready for a Data Vault Standard?
Dutch Data Vault MastersA Gathering of Creative Minds
Presenter: Date:
Note:
Company:eMail:
Twitter:
Frederik NaessensJune 5 2014In cooperation with Stijn Roelens & Kristof [email protected]
Wherescape 3D
What we do: • Configure Source/Business model using 3NF
modeling conventions• Add additional metadata at Table /
Column / Link / … level• Based on a set of preconfigured rules,
generate:– Data vault model– Staging model– Staging Out views
Marketplace
Isn’t this what different software packages or consultancy firms are currently providing in the marketplace?
See:• Presentation of Lulzim Bilali on DWA
congress• QUIPU• BIReady• Generators built on top of data modeling
tools (Powerdesigner, ERWIN,… )
DV Model Conversion standard
Is now not the moment to define a DV model conversion standard?Why?• Different tools can align to a single standard
and create synergies• This can be a catalysator to convince
organisations or people which are new to DV concepts
• Brings the focus to the really important concepts of Data Vault:
• Business keys• UOW• …
