Scripting and Automating ODI 11g using Groovy and the ODI SDK

T : +44 (0) 8446 697 995 E : enquiries@rittmanmead.com W: www.rittmanmead.com

Scripting and Automating ODI 11g using Groovy and the ODI SDKMichael Rainey, Principal Consultant, Rittman MeadRittman Mead BI Forum 2013 Master Class, May 2013

•Introduction to the ODI SDK•Introduction to Groovy scripting language•Demonstrations‣Developer’s Quickstart‣Metadata Setup‣Initial Load: Interfaces, Scenarios, and Load Plans‣ETL Migration to ODI

About this presentation...

Intro to the ODI SDK

•Java based API‣Develop using Java, Jython, or Groovy

•Perform nearly every action that can be done in ODI Studio‣Generate metadata‣Create / Modify Interfaces, Packages, Load Plans, etc‣Manage execution sessions

•Resources‣Data Integration bloghttps://blogs.oracle.com/dataintegration/tags/sdk‣Rittman Mead bloghttp://www.rittmanmead.com/tag/odi-11g-sdk/‣Oracle Data Integrator Java API Referencehttp://docs.oracle.com/cd/E28280_01/apirefs.1111/e17060/index.html

Intro to Groovy

•Agile, dynamic object oriented programming language for the Java Virtual Machine

•Like Java...but simplified‣Most Java syntax is valid in Groovy‣Additional features inspired by Python, Ruby, and Smalltalk‣Seamless integration with existing Java classes and libraries

•SwingBuilder ‣Create Swing GUIs using Groovy “builders”‣Easy to develop and maintain‣Hierarchy of components

•http://groovy.codehaus.org/

Groovy and ODI Integration

•Oracle Data Integrator is delivered with a Groovy editor‣Create and save .groovy script files‣Integrates with the ODI instance and security manager

•Transaction management simplified‣Create transaction‣Persist‣Commit transaction‣On exception...Rollback transaction

Develop Groovy in ODI Procedures

•ODI Procedure and Knowledge Module steps‣Groovy as a Technology

•Less integrated than .groovy scripts‣Must define

-Connection to repository-ODI username & password-ODI Instance for security and transaction management

Use Case - Developer’s Quickstart

•Case:‣Many find it difficult to use Oracle Data Integrator “right out of the box”‣Other ETL tools provide a quick and easy way to begin development

•Solution:‣Determine which tasks need to be automated and develop Groovy script

-Create Data Server, Physical Schema, and Logical Schema-Map Logical Schema/Physical Schema using the default Global context-Create new Model-Build and execute Package to reverse engineer tables into ODI Datastores‣Initial code developed by Oracle’s David Allan “ODI 11g – Expert Accelerator for Model Creation”https://blogs.oracle.com/dataintegration/entry/odi_11g_expert_accelerator_for

Developer’s Quickstart - Capture User Input

Demonstration

Developer’s Quickstart - ODI SDK and Groovy

Use Case - Metadata Setup

•Case:‣Setup of Staging and Foundation metadata ‣Add data warehouse audit columns

•Solution:‣Reverse engineer Source tables‣Copy tables to Staging and Foundation Models‣Add data warehouse audit columns to Foundation Datastores via Groovy script

Demonstration

Metadata Setup - ODI SDK and Groovy

Use Case - Initial Load Setup: Interfaces, Scenarios, and Load Plans

•Case:‣Setup objects for Source to Foundation initial load

•Solution:‣Groovy scripts

-Create Source to Foundation Interfaces-Generate a Scenario for each Interface -Build a Load Plan to execute all Scenarios

Initial Load Setup - InteractiveInterfaceHelperWithActions

DemonstrationInitial Load Setup - Interfaces, Scenarios, & Load Plans - ODI SDK and Groovy

Use Case - Migration to Oracle Data Integrator

•Case:‣Automate the migration of mappings from an “old” ETL tool to Oracle Data Integrator‣Oracle Warehouse Builder is the ETL tool in this example

•Solution:‣Place the “old” ETL tool metadata in a database schema‣Build Groovy scripts against the metadata to create ODI objects

-Topology-Project-Metadata- Interfaces

Migration to ODI - OWB Metadata

Demonstration

Migration to Oracle Data Integrator - ODI SDK and Groovy

•More information can be found at http://www.rittmanmead.com•Contact us at info@rittmanmead.com or michael.rainey@rittmanmead.com•Follow-us on Twitter (@rittmanmead & @mRainey) or Facebook (facebook.com/rittmanmead)

In summary...

•ODI SDK can perform nearly every action as the ODI Studio•Groovy script is the language of choice for developing against the ODI SDK‣Java-based, but simple to develop‣Developed in Groovy script files or Procedure steps

•Automation of large, batch tasks in ODI can save loads of time

Scripting and Automating ODI 11g using Groovy and the ODI SDKMichael Rainey, Principal Consultant, Rittman MeadRittman Mead BI Forum 2013 Master Class, May 2013

Scripting and Automating ODI 11g using Groovy and the ODI SDK

Technology

Programming Groovy 2media.pragprog.com/titles/vslg2/toc.pdf10.3 Using Groovy Classes from Groovy ? 10.4 Intermixing Groovy and Java with Joint Compilation ? 10.5 Creating and Passing

Groovy Tutorial

Why and How Migrate Informatica to ODI | Infa to ODI Migration | Infa to ODI Conversion

4 Billet Alminum Handle Pole - RESUCO57 odi odi PWC Parts & Accessories 2009 F70APS F70ECB Billet Bar End Cap Snap Cap Alminum Bar End Cap Replacement 1 3 2 4 odi odi odi odi odi odi

Groovy base

Groovy grails

Finding the Right Data Integration Toolset for Cloud and ...€¦ · solution and its ownership. While options such as Oracle Data Integrator (ODI), Groovy, or the REST API are all

Groovy Basics

GROOVY COTBED BED GROOVY LIT GROOVY · GROOVY GROOVY BED LIT. SAFETY INSTRUCTIONS Your child’s safety is your responsibility.. and use only spare parts approved by the manufacturer

Groovy Scripting - Apache Software Foundationpeople.apache.org/.../presentations/jsug-2016/groovy-scripting.pdf · • Groovy is easy for Java developers • Groovy is nice for non-trivial

Programming Groovy 2 - 3.droppdf.com3.droppdf.com/files/uHbLI/programming-groovy-2.pdf · 10.1 The Melting Pot of Java and Groovy 159 10.2 Running Groovy 160 10.3 Using Groovy Classes

Groovy And Grails - Chariot Solutionschariotsolutions.com/.../archive/449/groovy-grails-keynote.pdf · Groovy And Grails An Overview. Groovy. What Is Groovy?

Paulking groovy

Groovy ADF

Groovy scripts with Groovy

Siegfried Goeschl Groovy Scripting - home.apache.orgpeople.apache.org/.../2016/linuxwochen/groovy-scripting.pdf · • Groovy is easy for Java developers • Groovy is nice for non-trivial

Groovy Documentationdocs.groovy-lang.org/docs/groovy-2.1.3/pdf/wiki-snapshot.pdf · Groovy Introduction Groovy... • is an agile and dynamic language for the Java Virtual Machine

Agile Development with Groovy and Grails - s3.amazonaws.com · Beginning Groovy and Grails 1. Introduction to Groovy 2. Groovy Basics 3. More Advanced Groovy 4. Introduction to Grails

Adventures in Groovy – Part 20: Groovy On-Premise vs

Groovy Primer - chrisdail.com€¦ · Groovy Primer (March 2010) Groovy … Groovy Java •Is a superset of Java –Makes interop easy •Uses Java’s Type system •Everything is