25
Data Vault Modeling DW2.0 & Unstructured Data Big Data Agile DW Ensemble Modeling Agile BI with the Data Vault DW © 2013 Genesee Academy, LLC USA +1 303 526 0340 Sweden 072 736 8700 [email protected] www.GeneseeAcademy.com Knowledge is Power. Hans Hultgren gohansgo

Data Vault Agility Bi Podium November 2013

Embed Size (px)

DESCRIPTION

Achieving Agility with Data Vault Data Modeling for the Data Warehouse EDW. BI Podium Conference EU Netherlands 2013.

Citation preview

  • 1. Data Vault Modeling DW2.0 & Unstructured Data Big Data Ensemble Modeling Agile DWAgile BI with the Data Vault DW Knowledge is Power. 2013 Genesee Academy, LLC USA +1 303 526 0340 Sweden 072 736 8700 [email protected] www.GeneseeAcademy.comgohansgoHans Hultgren

2. Agile BI with the Data Vault DW How is it that Data Vault supports Agile BI so well? What about the other new approaches? In this session we break down the characteristics of the data modeling approaches that support data warehouse agility. Using a Modeling Pattern Characteristics analysis, Hans will in this session compare leading Ensemble modeling methods and their unique capabilities for supporting features of your EDW program. These include Data Vault, Anchor Modeling, Focal Point, and Hyper Agility modeling. What are the strengths and what are the weaknesses for each approach? How are they deployed and what do each mean for our overall DWBI architecture?Topics covered in this session also include Unified Decomposition, the definition and purpose of Raw versus BDW layers, the role of the Information Model related to our EDW, and a discussion on where and how to apply automation tooling and techniques. 2013 Genesee Academy, LLC2 3. Agile BI with the Data Vault DW Knowledge is Power. | Premise #1 You wont achieve agility with a modeling pattern alone. You wont achieve agility without a modeling pattern that supports it. 2013 Genesee Academy, LLC3 4. Agile BI with the Data Vault DW A Saga of Data Warehousing: Once upon a time data warehousing was becoming more popular and everyone was eager to build their own. But whenever they tried they failed. They called upon their best to fix this but they just couldnt solve the problem. They discovered that meeting the needs of the data warehouse meant that the tables got too big and too hard to work with. They just could not handle changes over time. If the smallest thing changed it always meant they had to change the entire table. When just a single attribute was updated they had to insert a record for all of the attributes. All seemed lost. But around the world there were rebels who questioned the conventional wisdom. And their voices were finally heard: Why not separate the things that change from the things that dont change? 2013 Genesee Academy, LLC4 5. Modeling Pattern Awareness Separating the things that change from the things that dont change. break things out into component parts for flexibility and to capture things that are interpreted in different ways or changing independently of each other Unified Decomposition 2013 Genesee Academy, LLC5 6. Ensemble Modeling The constellation of component parts acts as a whole an Ensemble. All the parts of a thing taken together, so that each part is considered only in relation to the whole. With Ensemble Modeling the Core Business Concepts that we define and model are represented as a whole an ensemble including all of the component parts. 2013 Genesee Academy, LLC6 7. What Forms are there? Data MartERP EDWAcctgData MartSales3NFAnchorEnsembleFocal PointData Vault2GData MartDimensionalTemporal, 6NF, Hyper Agillity+Matter, DV2.0+ 2013 Genesee Academy, LLC7 8. The Data Vault Ensemble The Data Vault Ensemble conforms to a single key embodied in the Hub construct. The component parts for the Data Vault Ensemble include: Hub The Natural Business Key Link The Natural Business Relationships Satellite All Context, Descriptive Data and History 2013 Genesee Academy, LLC8 9. Comparing the Big 3 Modeling Forms Both 3NF and Dimensional use highly encapsulated concepts. All forms are Attributed and stay rather close to the Business Concept. 3NF can sometimes move towards Abstracted Concepts. None of these forms are focused on Fully Abstracted or Generic Context methods. 2013 Genesee Academy, LLC 10. Data Vault means thinking differently CustomerCustomer 2013 Genesee Academy, LLC10 11. Agile BI with the Data Vault DW Knowledge is Power. | Premise #2 Modeling Awareness (understanding the pattern you are applying) is the key to successful modeling. Not knowing when you are applying a modeling exception is dangerous. 2013 Genesee Academy, LLC11 12. Applying the Data Vault Ensemble Mixing color types of data is not Data Vaulting but rather unvaulting A blended pattern has different dynamics? Thinking Differently! 2013 Genesee Academy, LLC12 13. Applying the Data Vault Ensemble Stay with the Ensemble Modeling Pattern. Continue practicing Unified Decomposition. Continue Vaulting. Be aware when you change patterns. Option 1Option 2or 2013 Genesee Academy, LLC13 14. Agile BI with the Data Vault DW Knowledge is Power. | Premise #3 Identify your own specific Modeling Pattern (including hybrid or blended patterns) to meet the needs of your EDW. Document your pattern and apply it consistently to gain the full benefits (including agility, repeatability and automation). 2013 Genesee Academy, LLC14 15. Sample: Sales Data Vault Model 2013 Genesee Academy, LLC15 16. Agile BI with the Data Vault DW Knowledge is Power. | Premise #4 There is no Integration without Semantic Integration. Integrating Data without understanding the meaning of that data is meaningless. 2013 Genesee Academy, LLC16 17. Agile BI with the Data Vault DW Information Modeling Logical Models, Conceptual Models, Industry Models, Semantic Models, Taxonomies and Ontologies are all forms of Information Modeling. These seek to target the central meaning of the core business concepts we work with and also how they relate to each other. These should be consistent with the concepts coming from your BICC, SOA, MDM, MDD, GCI, and other organizational information modeling, data governance or business glossary initiatives.Information Model 2013 Genesee Academy, LLC17 18. Staging 2013 Genesee Academy, LLC LoadTransformCalculate ConvertCleanseProfile ValidateExtractRawTransformCalculate ConvertCleanseProfile ValidateIntegrateLoadD/T StampIntegrateExtractArchitecture: Raw and BDVInformation ModelBDW Data MartData Mart Data MartEDW 18 19. Agile BI with the Data Vault DW Knowledge is Power. | Premise #5 Automation must address the Central Meaning of our data in order for it to be useful. Automating the modeling and loading of a source system can never result in multi-source data integration. 2013 Genesee Academy, LLC19 20. Automation Matrix Consider and compare the full set of capabilities for your automation solutions. 21. Process for modeling/deploying DV EDW Business Driven Modeling. Since an EDW integrates data from several sources, departments, divisions, functions, and countries over time, the integration target needs to be based on the organizations central view and not on a handful of source systems that happen to be in scope at the time.t + 10 yrsNow / Todayt - 10 yrs 2013 Genesee Academy, LLCInformation ModelEDW 22. The Enterprise Data WarehouseInformation ModelData 2013 Genesee Academy, LLCStageData WarehouseMartsInfo 23. Recap Agile modeling for the DW comes from Unified Decomposition Data Vault modeling leads the Ensemble Modeling family Knowledge is Power: Agility is more than modeling | Modeling must support it Know your modeling pattern | Know your architecture Customize your pattern, but | Apply it consistently Integration requires meaning | Create an Information Model Understand what you automate | Automate beyond sources 2013 Genesee Academy, LLC23 24. About Data Vault EnsembleEstimated 800 Data Vault based Data Warehouses around the world 2013 Genesee Academy, LLC24 25. Links and Information CDVDM Training & Certification www.GeneseeAcademy.com [email protected] DataVaultBook.blogspot.com HansHultgren.WordPress.com HansHultgren DataVaultAcademyOnline video-lesson trainingDataVaultAcademy.com 2013 Genesee Academy, LLC25