Embed Size (px)
Citation preview
Using Oracle Data Integrator with
Essbase, Planning and the Rest of
the Oracle EPM Products
Edward Roske
BLOG: LookSmarter.blogspot.com
WEBSITE: www.interrel.com
TWITTER: ERoske
2
About interRel
2008 & 2009 Oracle Titan Award winner - EPM Solution of the
year
2008 Oracle EPM Excellence Award
2009 Oracle EPM/BI Innovation Award
One of the fastest growing companies in the world
(Inc. Magazine, ’08 & ‘09)
Two of the three Hyperion Oracle ACE Directors in the world
Founding Hyperion Platinum Partner; now Oracle Certified Partner
Focused exclusively on Oracle Hyperion EPM software
Consulting
Training
Infrastructure and Installation
Support
Software sales
4
5 Hyperion Books Available:
Essbase (7): Complete Guide
Essbase System 9: Complete Guide
Essbase System 9: End User Guide
Smart View 11: End User Guide
Essbase 11: Admin Guide
eBooks available on Amazon Kindle
Just out!
Hyperion Planning for End Users
Coming Soon
Hyperion Planning for Admins (days now…)
Hyperion Financial Management (Q1 2010)
To order, check out www.lulu.com
Copyright © 2007, Hyperion. All rights reserved.5
Disclaimer
These slides represent the work and opinions of the
presenter and do not constitute official positions of Oracle
or any other organization.
This material has not been peer reviewed and is presented
here with the permission of the presenter.
This material should not be reproduced without the written
permission of interRel Consulting.
Agenda
What is ODI?
A Real World Example – ODI and Planning
ODI Demo
A glimpse into the concepts
Tips
Getting Started
Questions and Answers
What is ODI?
Oracle Data Integrator
Oracle Data Integrator (ODI) Overview
Integration tool that replaces HAL and DIM
Used to move and transform information across systems
Processes are based on metadata of existing systems
Only metadata is stored in ODI, not data
Sometimes referred to as Sunopsis
How is ODI used with EPM applications?
Interface metadata and data between source data systems
Build dimensions
Load data
Transfer data across files and relational databases and the
EPM applications
All supported with Hyperion Planning, Financial
Management and Essbase
Not supported with EPMA
Oracle ODI to EPM Functionality
Planning Essbase HFM
Metadata
discovery &
model creation
P P P
Load dataP P P
Load MetadataP P P
Extract dataO P P
Extract
metadata O P P
Other Refresh to
Essbase
Calc, post and
pre MaxL
scripts
Consolidate
Marketing Message
Three main points:
Extract-Load & Transform (E-LT) architecture
Removes ETL server from infrastructure
USP = Native target database performs
transformations
Declarative Design
Developer define the rules, ODI:
Writes the data flows
Manages the data flow complexity
Writes the behind-the-scenes code
Knowledge Modules
Similar to HAL adapters
Talks (and almost always writes) to almost everything
MaxL – Run Allocations,
Aggregate Data, etc
A Real World Integration with ODI
DW
Oracle
GL
SAP
Planning
HFM
Essbase
Fcst Data
Consolidated
Results
Sales by
Customer
DB
Accounts /
Entities
1
Accounts /
Entities
2
Accounts /
Entities
3
Customer Dimension
Sales by Customer Data
4
5
6
7
8
ODI Demo
A Glimpse
ODI Components and Concepts
ODI Components
Master
Repository
(Infrastructure)
Work
Repository
(Interfaces)
Topology Manager
Configure Architecture /
Infrastructure of source
& target technologies
Designer
Design & execute metadata
models and interfaces to move
data across systems
Operator
Monitor Sessions
Review errors
Agent
Coordinates
communication &
scheduling
Security Manager
Maintain ODI Security
Metadata Navigator
Web access to
mettadata repository
Source Systems Target SystemsODI
(ELT) Extract–Load-Transform
ODI Components
Components
GUI Purpose Users
Repository • Configuration Info
• Metadata
• OLTP RDBMS
• Sunopsis Infrastructure
Administrators
Developers
Operators
Designer • Reverse Engineer
• Develop Projects
• Release Scenarios
Developers
Metadata Administrators
Operator • Operate Production
• Monitor Sessions
Production Operators
Topology
Manager
• Physical and Logical
Architecture of the
infrastructure (servers,
schemas and agents)
Infrastructure Administrators
Security
Manager
• User privileges Security Administrators
\\
Deployment
Work
Repository
(s)(Interfaces)
Topology
Security
VersioningModels
Projects
Execution
Typically have 1 Master Repository that talks to multiple
Work Repository (Dev, Test & Prod).
Can set the production work repository to execution only
Master
Repository
(Infrastructure)
Topology Manager
Representation of the Information System
Connect to resources for integration purposes
Contains:
Physical Architecture
Data Server
Physical Schemas and Physical Agents
Logical Architecture (logical schemas & logical agents)
Contexts – Links logical and physical architectures
Repositories – Master and attached work repositories
Languages - different types of languages available
A Sample Topology
DW Essbase Source
FilesPlanning
Windows Machine X
(Development Environment)
PlanSamp.
Consol
FinOp.Plan
1
Windows Machine Y
(Production Environment)
Windows Machine Z
(Production Environment)
DW PlanningEssbase Source
FilesRep.App1
Rep.App2
Rep.App1
Essbase
Rep.App2
Essbase
Rep.App1
Rep.App2
PlanSamp.
Consol
FinOp.Plan
1
C:\ODI\FileC:\ODI\File
dbo.databa
sedbo.databa
se
PlanSamp.
Consol
Planning
FinOp.Plan
1
Planning
C:\ODI\File
Filedbo.database
MSSQL
Physical Agents
Logical Agents
Logical Architecture
Physical Architecture
Logical Schema
Physical SchemaData Server
Languages
Languages used to generate code for the specific
technologies
Designer
Designer includes:
Models – Metadata for your source and target systems
Projects - Integrations that move data from your source
and target systems
Interfaces
Packages
Procedures
Variables
Knowledge Modules
The Designer module stores this information in a work
repository, references the topology and the security
information defined in the master repository.
Designing an Interface
Before you start building an interface you:
Import Knowledge Modules into the project
Create your Metadata Models
Then in the interface you:
Define your source and target
Define the mapping and translations (business rules)
Define the flow requirements
Execute
To automate execution, move on to:
Packages
Procedures
Variables
Sequencing
Knowledge Modules
Knowledge Modules (KMs) are components with built in
knowledge (functions and logic) that connect technologies
to ODI.
Similar to HAL Adapters, but many more connection
options.
KMs connect to the technology, extract data from it,
transform the data, check it, integrate it, etc.
You can find a complete list of KMs at
http://www.oracle.com/technology/products/oracle-data-
integrator/10.1.3/htdocs/documentation/oracledi_km_refer
ence.pdf
Knowledge Modules
Interfaces
LKM (Loading Knowledge Modules) are used to extract
data from the source database tables and other systems
(files, middleware, mainframe, etc.).
IKM (Integration Knowledge Modules) are used to
integrate (load) data to the target system.
CKM (Check Knowledge Modules) are used to check that
constraints on the sources and targets are not violated.
.
Knowledge Modules
Models
RKM (Reverse Knowledge Modules) are used to perform a
customized reverse-engineering of data models for a
specific technology.
JKM (Journalizing Knowledge Modules) are used to create
a journal of data modifications (insert, update and delete)
of the source databases to keep track of the changes.
Knowledge Modules
Others
SKM (Service Knowledge Modules) are used to generate
the code required for creating dataservices.
An Example: Hyperion Planning Model
Dimensions
Planning
Metadata
An Example: Hyperion Essbase Model
Dimensions
Essbase
Metadata
Interfaces
An ODI object which loads one target data store with data
from one or more sources, based on business rules
implemented as mappings
Inside the Interface: Mappings
More on Mappings
Expression
Editor
Execute
On
UpdateTarget
Column
Multiple Sources, Joins and Filters
Joins (similar to Joins in RDMS)
Links multiple data sources together
Uses SQL Clauses
Multiple sources in a single interface must be connected
with a join.
Create joins by dragging and dropping columns together
Alternatively, you can use the expression editor to add
them manually
Advanced joins
Connect models from different technologies or model /
schemas. (Heterogeneous Join)
Connect 2 or more datastores
Join Definition
Specify
Join typeUse ISO-92 Syntax,
Join Order, Auto-
calculate Order
SQL
Syntax
Filter
Reduce amount of data
processed by the interface
by applying a filter.
Funnel icon represents
filter applied on the source
Use the Expression Editor
or manually type in the
SQL.
Test filter, right-click and
select Data.
Expression Editor
Flow Tab
SUNOPSIS_MEMORY_ENGINE
SUNOPSIS_MEMORY_ENGINE sits between source and
target
Temporary staging area that is created during interfaces to
facilitate mapping and translation
Not recommended for large data sets for performance
reasons
Can be difficult to debug
We recommend configuring a separate staging area
Operator: Checking Status
Automation
Packages - Group of steps to complete a workflow.
Can include:
Interfaces
OS Commands
Procedures
Workflow – can include error trapping and email status
notifications
Scenarios – can schedule as a scenario or call a scenario
from a batch file
Variables - can have variables defined to pass parameters
(i.e. file name, current month, etc.)
Create Error Log Files in the Interface
Enable logging in
the interface in the
files to obtain more
details or send
information to a
text file for use in
debugging.
Errors
Errors are not always ODI errors, they are typically being
sent from other technologies (Oracle EPM in our case)
You will begin to see commonality of errors but they are
not always descriptive or useful.
Plan extra time for debugging interfaces
Real Life Tips
Tips – General Information
Good sources of information:
interRel 3 day ODI and EPM training class (available
this fall)
OTN Forum:
http://forums.oracle.com/forums/forum.jspa?forumID=3
74
http://john-goodwin.blogspot.com/
Get the 10.1.3.5 release
No patching required for Essbase, Planning, and HFM
Available at:
http://www.oracle.com/technology/software/products/od
i/index.html
Design and Development Tips
Whiteboard the Topology before configuring in ODI
Standardize naming conventions for topology components
Incorporate as many error logs and error trapping
processes as possible.
Consider using a staging area versus Sunopsis Memory
Engine. This will allow performance optimization and error
resolution.
If the source is a relational database, perform as many
translations in a view as possible.
Design and Development Tips
Recommend against using comma as a file delimiter for
source files. Use | or !.
Versus loading Essbase Data with ODI, write a MaxL to
load data to Essbase and add it as a step in the ODI
package.
Plan extra development time, learning curve is steep and it
is more complex than HAL.
Project Tasks: Getting Started
Plan the scope, identify source and target
Establish success criteria
Identify the resources
Define the Topology
Define your Models
Create Interfaces
Unit Testing
Automate with Packages
Schedule with Scenarios
Integration Testing
Deploy
How long will take?
No short answer, depends on the complexity of the
interface. If this is the first time, allow ample time to debug
issues.
Minimize risk by:
Working with infrastructure team for connectivity issues
Keeping data transformations simple or nonexistent in the
initial development (prototype)
Then consider the opportunities to automate translations or
business rules
Plan out your topology on a whiteboard first! It is critical to
understand what links to what to avoid silly mistakes.
Use Relational Sources where possible.
OTHER
Security Manager
Separate from Shared Services
Manage security for all ODI modules
Create Users & Profiles
Assign access
ODI Objects (data servers, projects, etc)
Methods (Edit, Create, Delete)
Servers
Metadata Navigator
J2EE application that provides web access to the
repositories
View Flow maps, trace the source of all data and drill down
to field level
Possible to trigger and monitor processing jobs from a web
browser using this tool
By default, installed on Jakarta Tomcat Application Server
Oracle Data Profiling and Quality
Integrator extends data quality features of Oracle Data
Integrator to provide more advanced data governance
capabilities.
Data Cleansing
Duplicates
Parsing
Data Enrichment
User driven rules
Integrated with ODI
ODI and EPM Training
Coming Soon from interRel Training!
Introduction to ODI for EPM Administrators
This class will be useful for administrators or developers that
will design, develop and / or deploy data integrations that
update Essbase, Planning or HFM with ODI.
Includes several hours of hands on workshops !!!
Thank You.
Edward Roske
BLOG: LookSmarter.blogspot.com
WEBSITE: www.interrel.com
TWITTER: ERoske
