Upload
sundafunda
View
220
Download
0
Embed Size (px)
Citation preview
8/12/2019 P.3 PTS ODI11g ODI Architecture
1/36
Copyright 2010, Oracle. All rights reserved.
8/12/2019 P.3 PTS ODI11g ODI Architecture
2/36
Oracle Data Integrator
Architecture
PTS Workshop: ODI 11g
8/12/2019 P.3 PTS ODI11g ODI Architecture
3/36
Copyright 2010, Oracle. All rights reserved.
Insert Picture Here>
Agenda
ODI Architecture Overview
ODI Components
Agents
Console
Repositories Data Flow
Q&A
8/12/2019 P.3 PTS ODI11g ODI Architecture
4/36
Copyright 2010, Oracle. All rights reserved.
ODI Architecture
8/12/2019 P.3 PTS ODI11g ODI Architecture
5/36
Copyright 2010, Oracle. All rights reserved.
ODI Architecture
Desktop
Repositories
ODI Studio
Operator
Designer
Topology
Security
ODI MasterRepository ODI Work
Repository
Sources and Targets
Legacy Applications
ERP/CRM/PLM/SCM
Files / XML DBMS DW / BI / EPM
JVM
JavaApplication
ODI SDK
WebLogic11g/ ApplicationServer
Data Sources Connection Pool
Web Service Container
Public WS DataServices
FMW ConsoleODI Plug-in
Servlet Container
ODI Console
JavaEE
Application
ODI SDK
Runtime WS
Java EEAgent
JVMRuntime WS
StandaloneAgent
Open Web Services fordecoupled invocation of
any ODI jobs
Open WebServices fordecoupled
invocation of anyODI jobs
SDK APIs for Javageneration of any ODI
system metadata
Metadata is an openrelational modelwith built-in Flex
Fields
OpenTools for adding OSfunctionality to any ETL
job
8/12/2019 P.3 PTS ODI11g ODI Architecture
6/36
Copyright 2010, Oracle. All rights reserved.
ODI Components
8/12/2019 P.3 PTS ODI11g ODI Architecture
7/36
Copyright 2010, Oracle. All rights reserved.
Graphical Modules
JavaMost Platforms
DB - Most ISO-92
Security Manager
Manage user
privileges
Operator
Operate production
Monitor sessions
Topology Manager
Define the infrastructure
of the IS
Designer
Reverse-Engineer
Develop Projects
Release Scenarios
Repository
ODI Studio
8/12/2019 P.3 PTS ODI11g ODI Architecture
8/36
Copyright 2010, Oracle. All rights reserved.
SOA / Java EE Run-Time Components
Any ISO-92 RDBMS
Execute Jobs
Java EE Agent
Handles schedules
Orchestrate sessions
Java - Any Platform
Operator
Operate productionMonitor sessions
Designer
Reverse-EngineerDevelop Projects
Release Scenarios
WebLogic Java EE server
Information System
Return Code
Submit Jobs
Read sessionsWrite reports
Monitor sessionsView Reports
Repository
Public Web Services
Manage sessions
Access Data
ODI Studio
Any Client App
Consume DataControl EL-T Operations
Invoke and managesessions
Data Services
Access Data and
Changed Data
Access Data
8/12/2019 P.3 PTS ODI11g ODI Architecture
9/36
Copyright 2010, Oracle. All rights reserved.
Standalone Run-Time Components
Any ISO-92 RDBMS
Execute Jobs
Standalone Agent
Handles schedules
Orchestrate sessions
Java - Any Platform
Operator
Operate production
Monitor sessions
Designer
Reverse-Engineer
Develop ProjectsRelease Scenarios
Java - Any Platform
Information System
Return Code
Submit Jobs
Read sessionsWrite reports
Monitor sessionsView Reports
Lightweight
Distributed Architecture
Repository
8/12/2019 P.3 PTS ODI11g ODI Architecture
10/36
Copyright 2010, Oracle. All rights reserved.
Enterprise Manager
and ODI ConsoleAny Web Browser
Control and monitor ODI
production
Browse metadata lineage
Any ISO-92 RDBMS
SubmitExecutions
Execute Jobs
Java EE Agent
Handles schedules
Orchestrate sessions
Information System
Return Code
Repository
WebLogic Java EE server
ODI Console
Web access to the
repository
Enterprise ManagerFusion Middleware Console
Other components
SOA Suite, WLS
Servers, Web apps.
8/12/2019 P.3 PTS ODI11g ODI Architecture
11/36
Copyright 2010, Oracle. All rights reserved.
ODI ConsoleWeb access to the
repository
Enterprise Manager FMC
Other
componentsSOA Suite, WLS
Servers, Web apps.
Security
ManagerManage user privileges
OperatorOperate production
Monitor sessions
Topology
ManagerDefine the IS
infrastructure
DesignerReverse-Engineer
Develop Projects
Release Scenarios
ODI Studio
Components: a Global View
Any ISO-92 RDBMS
Java EE Agent
Handles schedules
Orchestrate sessions
Java - Any Platform
Java - Any Platform
Any Web BrowserBrowse metadata
lineage
Operate production
Information System
Repository Access
HTTP Connection
Execution Query
Repository
Any Client AppConsume DataControl EL-T Operations
Public Web Services
Manage sessions
Access Data
Invoke andmanage sessions
Data Services
Access Data and
Changed Data
Access Data
8/12/2019 P.3 PTS ODI11g ODI Architecture
12/36
Copyright 2010, Oracle. All rights reserved.
Designer Navigator
In ODI Designer, youcan create, configure,and execute various
ODI objects.
Workspace to defineODI objects and designEL-T transformations
Toolbar
Objecttree
OtherDesignereditors
DesignerProjects
editor
8/12/2019 P.3 PTS ODI11g ODI Architecture
13/36
Copyright 2010, Oracle. All rights reserved.
Operator Navigator
In OperatorNavigator, you canmonitor execution
of ODI objects.Toolbar
Workspace todefine ODI objects
and design EL-Ttransformations
The monitoringsession steps
enable you to
perform debugging.
8/12/2019 P.3 PTS ODI11g ODI Architecture
14/36
Copyright 2010, Oracle. All rights reserved.
Topology Navigator
In ODI Topology Manager,
you set the architecture ofyour information system, thetechnologies, servers,
schemas, and repositories.
Technologies
tree
Toolbar
Workspace to
define ODITopology objects
8/12/2019 P.3 PTS ODI11g ODI Architecture
15/36
Copyright 2010, Oracle. All rights reserved.
Security Navigator
In Security Navigator,you manage objects,profiles, users, and
hosts.Objecttree
Workspace to
define objects,profiles, users, and
hosts
8/12/2019 P.3 PTS ODI11g ODI Architecture
16/36
Copyright 2010, Oracle. All rights reserved.
Agents
8/12/2019 P.3 PTS ODI11g ODI Architecture
17/36
Copyright 2010, Oracle. All rights reserved.
What Is an Agent?
An agent is a run-time component of ODI that orchestrates theintegration process.
At design time, developers generate scenarios from the business
rules that they have designed. The code of these scenarios is
then retrieved from the repository by the agent at run-time.
This agent then connects to the data servers and orchestratesthe code execution on these servers.
8/12/2019 P.3 PTS ODI11g ODI Architecture
18/36
Copyright 2010, Oracle. All rights reserved.
ODI Agents
Agents are lightweight Java processesthat orchestrate the execution of objects
at run time.
Agents can:
Execute objects on demand, or
Execute according to predefinedschedules
8/12/2019 P.3 PTS ODI11g ODI Architecture
19/36
Copyright 2010, Oracle. All rights reserved.
Two Types of AgentsJEE and Standalone
JDeveloper
ODI Studio
Designer Operator
Security Topology
WebLogic 11g(or other JEE)
Data Sources Connection Pool
Repository Data Sources Sources Data Sources Target Data SourcesRepository Data Sources Sources Data Sources Target Data Sources
Repository Data Sources Sources Data Sources Target Data Sources
Servlet Container
ODI Agent
ODI Console
Web Service Container
ODI Public
Web Service
ODI DataServices
Any Application Container
Any Application
ODI SDK APIs
MBeans Server Registry
Any MBeans App
ODI MBeansfor ODI Agent
Sources and Targets
ERP
---
------
---
---
---
CRM
- - -
Legacy
---
------
---
---
------
------
---
---
---
Best-of-Breed ApplicationsPLM
RDBMS ODI Repositories
ODI MasterRepository
ODI WorkRepositoryODI Work
RepositoryODI Work
Repository
Standalone Java
Any Java App
ODI SDK APIs
Any Web App
KnowledgeModules
ODI Agent
Standaloneagent
JEEagent
8/12/2019 P.3 PTS ODI11g ODI Architecture
20/36
8/12/2019 P.3 PTS ODI11g ODI Architecture
21/36
Copyright 2010, Oracle. All rights reserved.
Example of Standalone Agent
Lightweight
Distributed Architecture
Any ISO-92 RDBMS
ExecuteJobs
Standalone Agent
Orchestratessessions
Java - Any Platform
Operator
Operateproduction
Monitorsessions
Java - Any Platform Return Code
SubmitJobs
Readsessions
Write reports
Monitor sessions
View Reports
Designer
Reverseengineer
Developprojects
Releasescenarios
Information System
Repository
8/12/2019 P.3 PTS ODI11g ODI Architecture
22/36
Copyright 2010, Oracle. All rights reserved.
Console
8/12/2019 P.3 PTS ODI11g ODI Architecture
23/36
Copyright 2010, Oracle. All rights reserved.
ODI ConsoleDevelopment
ODIConsole
ODIConsole
Agent
8/12/2019 P.3 PTS ODI11g ODI Architecture
24/36
Copyright 2010, Oracle. All rights reserved.
Enterprise Manager Console
Development
Enterprise ManagerConsoleODI Plug-infor EM
Console
Agent
8/12/2019 P.3 PTS ODI11g ODI Architecture
25/36
Copyright 2010, Oracle. All rights reserved.
ODI Repositories
8/12/2019 P.3 PTS ODI11g ODI Architecture
26/36
Copyright 2010, Oracle. All rights reserved.
ODI Repositories
There are two types of repositories in ODI: Master repository
Work repository
Development repository
Execution repository
Work repositories are always attached to a Master repository.
8/12/2019 P.3 PTS ODI11g ODI Architecture
27/36
Copyright 2010, Oracle. All rights reserved.
Master Repository
Master and Work Repositories
Models
Projects
Execution
Work Repository(Development)
Execution
Execution Repository(Production)
Security
Topology
Versioning
Two type of Repositories: Master and Work
Work Repositories are always attached to a Master Repository
8/12/2019 P.3 PTS ODI11g ODI Architecture
28/36
Copyright 2010, Oracle. All rights reserved.
Example of a Repository Set-Up
Models
Projects
Execution
Work Repository
(Development)
Models
Projects
Execution
Work Repository
(Test & QA)
Execution
Execution Repository(Production)
Security
Topology
Versioning
Master
Repository
Create and archive
versions of models,
projects and
scenarios Import released
versions of models,
projects andscenarios for testing
Import released and
tested versions of
scenarios for
production
DevelopmentTest Production Cycle
8/12/2019 P.3 PTS ODI11g ODI Architecture
29/36
Copyright 2010, Oracle. All rights reserved.
Models
Projects
Execution
Work Repository
(Development)
Models
Projects
Execution
Work Repository
(Test & QA)
Execution
Execution Repository
(Production)
Security
Topology
Versioning
Master
Repository
Create and archive
versions of models,
projects and
scenarios (from the
GUI)Import released
versions of models,
projects andscenarios for testing
(From the GUI)
XML export / Import
(manual)
DevelopmentTest Production Cycle
Security
Topology
Master
Repository
Repository Set-Up: Separate Networks
XML export / Import
(manual)
Firewall
8/12/2019 P.3 PTS ODI11g ODI Architecture
30/36
Copyright 2010, Oracle. All rights reserved.
ODI Data Flow
8/12/2019 P.3 PTS ODI11g ODI Architecture
31/36
Copyright 2010, Oracle. All rights reserved.
Oracle Data IntegratorArchitecture
Repositories
User Interfaces
Developer
Developer
DeveloperExecution
Log
Code
Metadata
Rules
Agent
Data FlowConductor
Servers and Applications
CRM
Legacy
ER
P
Data
Warehouse
Files /
XMLESB
Return
Codes
Execution
No Server: only an orchestrator (The Agent) that will send theappropriate SQL statements to the different databases.
Developers each have the Java GUI on their machineand share a central repository to store the metadataand the transformation processes (no local storage).
8/12/2019 P.3 PTS ODI11g ODI Architecture
32/36
Copyright 2010, Oracle. All rights reserved.
Oracle Data IntegratorArchitecture
Repositories
User Interfaces
Developer
Developer
Developer
Execution
Log
Code
Metadata
Rules
Agent
Data FlowConductor
Servers and Applications
CRM
Legacy
ER
P
Data
Warehouse
Files /XML
ESB
Return
Codes
Execution
Processes are orchestrated (and possibly scheduled)by an ODI Java agent (no graphical component to
allow for the largest possible set of platform support).
8/12/2019 P.3 PTS ODI11g ODI Architecture
33/36
Copyright 2010, Oracle. All rights reserved.
Oracle Data IntegratorArchitecture
Repositories
User Interfaces
Developer
Developer
Developer
Execution
Log
Code
Metadata
Rules
Agent
Data FlowConductor
Servers and Applications
CRM
Legacy
ER
P
Data
Warehouse
Files /XML
ESB
Return
Codes
Execution
Multiple agents can co-exist in any given environment.Typical locations for the agents are on the targetsystem and/or on the system that hosts the ODI
repository.
O i 6 t t P d ti
8/12/2019 P.3 PTS ODI11g ODI Architecture
34/36
Copyright 2010, Oracle. All rights reserved.
ODI Design-Time Environment ODI Runtime Environment
User Interfaces
Overview: 6 steps to Production
1. Retrieve/Enrich metadata
2. Design transformations
3. Orchestrate data flows
4. Generate/Deploy data flows
5. Monitor executions
6. Analyze impact / data lineage
Development
Development Servers and Applications
Agent
Data FlowConductor
CRM
Legacy
ERP
DataWarehouse
Files / XML
User Interfaces
Administrators
Designers
ESB
Design-timeRepositoriesDesign-time
Repositories
Production
Production Servers and Applications
Agent
Data FlowConductor
CRM
Legacy
ERP
DataWarehouse
Files / XML
Operator
ODI Console
ESB
RuntimeRepository
8/12/2019 P.3 PTS ODI11g ODI Architecture
35/36
Copyright 2010, Oracle. All rights reserved.
8/12/2019 P.3 PTS ODI11g ODI Architecture
36/36
Copyright 2010 Oracle All rights reserved