Oracle Enterprise Metadata Management 12c: The Solution to ... · T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256

T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256 7970 (India)

E : [email protected] W : www.rittmanmead.com

Oracle Enterprise Metadata Management 12c: The Solution to Manage all your Metadata and Not Die in the Attempt



Edelweiss Kammermann

•Computer engineer, BI and BPM specialist with over 18 years of consulting and project management experience in Oracle technology.

•Co-founder and Vice President of Uruguayan Oracle User Group (UYOUG) •Business Intelligence articles published on OTN site •Frequent speaker at international conferences as Collaborate, OTN Tour LA, UKOUG Annual Conference, OOW and Rittman Mead BI Forum

•Oracle ACE •Training Materials Specialist in Rittman Mead

email: [email protected] Twitter: @EdelweissK

mailto:[email protected]



Rittman Mead

•Oracle BI and DW Gold partner with offices in USA (Atlanta), Europa, Australia and India

•Winner of five UKOUG Partner of the Year awards 2014 including BI

•World leading specialist partner for technical excellence, solutions delivery and innovation in Oracle BI

•100+ consultants worldwide included 1 Oracle ACE Director and 4 Oracle ACEs

•Skills in broad range of Oracle technologies including OBIEE,OBIA, BI Cloud, ODI, Essbase, GoldenGate, Endeca Exadata, Exalytics and BigData

•Blog : http://www.rittmanmead.com/blog/ •Twitter : @rittmanmead

http://www.rittmanmead.com/blog/



Agenda

•Problem: Which Data am I Using in this Application? •Solution: Oracle Metadata Management • Installation and Configuration Steps • Importing Metadata into OEMM 12c •Relating Metadata in OEMM 12c •Trace Data Lineage • Impact Analysis



Problem: Which Data am I Using in This Application?

•This is the typical question that not only Business Users but also technical people as SysAdmin, Data Scientist, Data Steward and Developers have.

•Because of the proliferation of different applications and data sources, is very difficult to track which data is used by which applications. ‣The problem is bigger when more systems are involved. ‣In some cases is almost impossible to perform an impact analysis to be sure the impact of changing an object in a data source.



Solution: Oracle Enterprise Metadata Management

•OEMM is a complete metadata management platform that can reverse engineer (harvest) and catalog metadata from any source: relational, Big Data, ETL, BI, data modelling, etc.

•OMM allows to perform: ‣Interactive searching ‣Data lineage ‣Impact analysis

•The metadata from different sources and applications can be related (stitched) ‣Metadata can be from different providers (Oracle or/and third-party)

•Manages versioning and comparison of metadata models. •Shows the complete path of data from source to report or vice versa.



Oracle Metadata Management

•Oracle Metadata Management solution offers two products: ‣OEMM (Oracle Enterprise Metadata Management) ‣Use metadata providers from Oracle and third-party technologies.

‣OMM for OBI (Oracle Metadata Management for Oracle Business Intelligence) ‣Use metadata for databases, OBIEE, ODI and DAC.



OEMM 12c Installation and Configuration Prerequisites

•OMM is supported only for Windows Operative System ‣Windows 2008 Server, Windows 2012 Server, Windows 7, Windows 8, or Windows 8.1. ‣Microsoft .NET Framework 3.5 or higher has to be installed

•Oracle Database is the only database server supported for the OMM repository ‣Oracle Database from 10gR2 to 12 64-bit ‣Character set MUST be AL32UTF8 (UTF8).

•The installer comes with the Apache Tomcat as Web Application Server ‣Oracle JRE 6 as Java Run Environment

•For a small test environment: ‣2 GHZ or higher quad core processor ‣4 GB RAM (8 GB if 64bit OS using 64bits Web Application Server) ‣10 GB of disk space (all storage is primarily in the database server)



OEMM 12c Installation and Configuration Steps

1. Download the software from OTN or e-delivery. 2. Create Database Schema as Repository 3. Install the OMM Software 4. Start OMM as a Service 5. Initialise OMM 6. Start OMM



OEMM 12c Installation Step 1: Download the Software

•Go to the OTN site to download the OEMM software. ‣http://www.oracle.com/technetwork/middleware/oemm/downloads/index.html

http://www.oracle.com/technetwork/middleware/oemm/downloads/index.html



OEMM 12c Installation Step 2: Create the OMM Repository

•An Oracle Database Schema need to be created to hold the OMM Repository. •Can be created on the users tablespace or create dedicated user and temporary tablespaces for this schema.

•This schema should have the following grants: ‣Create Session, Create Procedure, Create Sequence, Create Table, Create Trigger, Create Type and Create View

•And also needs privilege to execute the following packages: ‣CTX_DDL package from CTXSYS ‣DBMS_LOCK from SYS.



OEMM 12c Installation Step 3: Install the OMM Software

•The OMM Installation is very straightforward. Just follow the indications of each screen.



OEMM 12c Installation Step 4: Start OMM as a Service

•After the computer restarted, start OMM as a Service ‣Using the Start Menu option or directly Start it through Windows Services



OEMM 12c Installation Step 5: Initialise OMM

•Go to the URL: http://localhost:11580/MM •Enter the connection details for the database schema that you created for the OMM repository and Test the Connection

•Press the Save button to run the initialisation process which consist in OEMM creating the objects in the database schema to manage the repository.



OEMM 12c Installation Step 6: Start OMM

•Close the browser tab and open again the OEMM URL (http://localhost:11580/MM) •When the login page appears, enter Administrator/Administrator as User and Password

http://localhost:11580/MM



OEMM 12c Importing Metadata

•Just to clarify, Metadata is the description of the data structures not the data itself.

• In order to analyse where the data is coming from and which applications are using it, we need to import the metadata into OEMM from the different sources and applications.

•To import metadata from a specific provider we need to create a Model in OEMM

•A Model in OEMM has all the connection details for a specific source or metadata provider ‣For example: a database schema, an ODI repository

•A Model is the container for the metadata of an specific provider ‣So one model can connect to one specific source or application.



OEMM 12c Importing Metadata: Creating Folders

•For organisational purposes it is recommended to create folders to contain and arrange the models ‣Folders can be nested

•To create a folder, right-click on the Repository entry under the Repository panel

•Select New > Folder in the pop-up menu, enter a name and press the Create button.



OEMM 12c Importing Metadata: Creating Models

•Right click on the folder or the repository entry, and select New > Model.

• In the Create Model window, enter a name for the new model and select the import bridge to use in this import process.

•The import process is managed by the Meta Integration® Model Bridge (MIMB) software. ‣There are import bridges for a wide range of technologies like different databases,BI solutions, big data, etc.




•Then in the Import Setup tab, we need to enter the connection details for this model.

•We can test the connection before starting the importing process

•Press the Create button to create the model.

•Answer Yes to the “Import New Version Now”, to start the harvesting (importing) process.




•After the succeeded import message, open the model to see objects structure.

•You need to repeat this process to create a model for for each data source or application that you want to relate.



Harvesting Examples: Oracle Database Model



Harvesting Examples: Oracle Data Integrator Model



Harvesting Examples: Oracle Business Intelligence Model



Harvesting Examples: Oracle Warehouse Builder Model



OEMM 12c Relating Metadata

•Once we have defined our models we need to relate them and to validate their relationship. •The automated process of relate these models through the validation is named stitching. • In order to do that we must create a Configuration. •A Configuration in OEMM is a collection of models and another objects that are related in someway.



OEMM 12c Relating Metadata

•According to the documentation a configuration is as any of these options: •Repository workspace: a collection of Repository Objects to be analysed together as a technical scope.

•Enterprise architecture: a collection of data store Models (ODS, data staging areas, data warehouses, data marts, etc.) and data process Models (ETL/DI, and BI) connected together through data flow stitching.

•Design workflow: a collection of conceptual, logical and physical Models connected (semantically stitched) together through semantic mappings modelling the design process.



OEMM 12c Importing Metadata: Creating Configurations

•To create a Configuration, right-click on a selected folder or the repository entry and choose New> Configuration.

•Enter a name for the configuration and press the Create button.




•Once the configuration is opened, drag the models that you want to be stitched inside this configuration.




•You maybe see that some of the dragged models have a warning icon after you include them in the configuration ‣This is because we need to connect that model with the appropriate source of data.

•Select the model in the configuration and press Edit Connection. Choose the correspondent store for each connection and press OK.




•After the assignment of the sources for all the models, press the Validate button, to start stitching or relate them.

• In most of the cases, OEMM can assign the correspondent default schema for each of the connections in the model. ‣Otherwise you need to do it manually.




•You can see all the validated models in the Model Manager tab of the Configuration




•You can see the relationship between the models that belong to a configuration in a graphical view in the Architecture Diagram tab of the Configuration.

•Press the Edit button and then Layout to rearrange the models in the diagram.



OEMM 12c Tracing Data Lineage

•The Trace Data Lineage process is to find the path from the source to the target object that you selected.

•There are two possibles scopes for Tracing Data Lineage: ‣Model: which trace the object in the same model ‣Configuration: to trace the object including its related objects in the configuration.

•The format of the trace data lineage could be Graph or Text.




•To perform a trace data lineage, open the required model.

•Right-click on traceable objects like columns, and select Trace Data Lineage




• In the Advanced Lineage Options, select the Scope and if you want to display it as a list or Graph.

•Press the Trace button.




•The result graph will show the object and the sources from which is derived from.




• In this example if we want to see the ETL process for one particular column, right click the required connection and select Trace ETL Details.




• In this example if we want to see the ETL process for one particular column, right click the required connection and select Trace ETL Details.



OEMM 12c Tracing Data Impact

•The process of Data Impact is to find out which data items in target database and applications are affected if we change one source object, like column or table.

•Open the required model and right-click on the required object in the model and select



OEMM 12c Tracing Data Impact

• In the resulted graph you will show the targets that used this object as a source.

•You can also highlight a specific column in the table. ‣Very useful when you have configurations with a large amount of related objects.



Additional Information

•Rittman Mead blog posts ‣http://www.rittmanmead.com/2015/01/why-and-how-to-use-oracle-metadata-management-12c-part-1-getting-started/ ‣http://www.rittmanmead.com/2015/02/why-and-how-to-use-oracle-metadata-management-12c-part-2-importing-and-relating-metadata/

•Linkedin post OEMM and Big Data by Issam Hijazi ‣https://www.linkedin.com/pulse/20141203160343-48890009-big-data-oracle-enterprise-metadata-management

•OTN Site for OMM ‣http://www.oracle.com/technetwork/middleware/oemm/index.html

http://www.rittmanmead.com/2015/01/why-and-how-to-use-oracle-metadata-management-12c-part-1-getting-started/

http://www.rittmanmead.com/2015/02/why-and-how-to-use-oracle-metadata-management-12c-part-2-importing-and-relating-metadata/

https://www.linkedin.com/pulse/20141203160343-48890009-big-data-oracle-enterprise-metadata-management

http://www.oracle.com/technetwork/middleware/oemm/index.html



Questions?



Oracle Enterprise Metadata Management 12c: The Solution to Manage all your Metadata and Not Die in the Attempt

Documents

Oracle Enterprise Metadata Management 12c: The Solution to ... · T : +44 (0) 1273 911 268 (UK) or (888) 631-1410 (USA) or +61 3 9596 7186 (Australia & New Zealand) or +91 997 256