Milomir Vojvodic Oracle DIS

Oracle GoldenGate and Oracle Data IntegratorMilomir Vojvodic

Oracle Data Integration Solutions

Oracle Applications

Custom Applications

MDM Applications

Business Intelligence

Activity Monitoring

SOA Platforms

Comprehensive Data Integration SolutionSOA AbstractionProcess Manager Service Bus Data Services

Data Federation

Oracle Data Integrator E-LT/ETL

Oracle GoldenGate

Oracle Data Quality Profiling Cleansing Parsing

Real-Time Data Integration

Storage

Data Warehouse/ Data Mart

OLTP System

OLAP Cube

Flat Files

Web 2.0

Web and Event Services, SOA

DIS products vs. out of the box

Oracle OOTB

DIS Oracle strategic productsOracle GoldenGate

Difference?- Heterogeneous - Scalability, performance, reliability - Best Productivity - Makes HA/DR investments work - Best MAA solution for 11g - Provides reporting benefits

Streams

Data Guard

Active Data Guard

OWB

Oracle Data Integrator

- Increased ROI - Broader support/heterogenity - Best productivity & scalability

Maintenance Mode

Oracle Excellent Product ViabilityWeight -ingIBM Oracle

Informatica

iWay

SAP

SAS

Micro -soft

Bulk data movement

55%

5.0

4.5

5.0

4.1

4.6

4.3

4.3

Federated views Message-oriented movement Data replication and synchronization Total Rating Weighted Rating

15%

3.9

3.0

3.9

3.7

4.2

3.4

2.2

10%

4.5

3.7

3.3

4.1

3.2

2.0

3.7

20%

4.6 18 4.705

5.0 16.2 4.295

4.5 16.7 4.565

2.9 14.8 3.8

4.0 16 4.28

3.0 12.7 3.675

2.9 13.1 3.645

Source: Gartner Critical Capabilities for Data Integration Tools: Common Data Delivery Styles, Dec 2010 (showing top 7 of 11 vendors in chart above)

Pre-Built SolutionsSOA SuiteBAM & CEPODI Embedded Pre-built Content

ID MgmtBPEL & OSBODI Integrated

Business IntelligenceBI AppsODI Embedded Pre-built Content

Other Apps (+more)Agile PLMODI Embedded Pre-built Content

Identity AnalyticsODI Embedded Pre-built Content

BIEE SuiteODI Integrated

RetekODI Embedded Pre-built Content

Enterprise Performance Management and FinancialsHFMODI Embedded

PlanningODI Embedded Pre-built Content

Essbase

MDM/DRM

Spend AnalyzerODI Embedded Pre-built Content

Governance & RiskODI Embedded Pre-built Content

Siebel CRMODI Integrated GoldenGate Integrated New

ODI Integrated

ODI Integrated

Oracle Fusion ERP ApplicationsFusion Pillar ReplicationOGG Embedded Pre-built Content

Fusion AIANew ODI Embedded Pre-built Content

Fusion HCMODI Embedded Pre-built Content

Fusion MDMODI Embedded Pre-built Content

Fusion CRMODI Embedded Pre-built Content

Fusion FinancialsODI Embedded Pre-built Content

Fusion MigrationODI Embedded Pre-built Content

New

Oracle Golden GateMilomir Vojvodic

What is GG and how it works?Capture: Committed changes are captured (and can be filtered) as they occur by reading the transaction logs.

Trail files: Stages and queues data for routing. Pump: Distribute data for routing to multiple targets. Route: Data is compressed, encrypted for routing to targets. Delivery: Applies data with transaction integrity, transforming the data as required.

Pump Capture Source Trail Pump LAN / WAN / Internet (TCP/IP) Target Trail Delivery

Source Database(s)7

Target Trail

Source Trail

Delivery

Capture

Target Database(s)

Bi-directional

Expanded HeterogeneityDatabasesOracle GoldenGate Capture: Oracle NEW DB2 for v 9.7 NEW Microsoft SQL Server Sybase ASE Teradata Enscribe SQL/MP SQL/MX NEW MySQL JMS message queues NEW

O/S and PlatformsLnux for 2008 Sun Solaris Windows 2000, 2003, XP HP NonStop HP-UX HP TRU64 HP OpenVMS IBM AIX IBM z Series zLinux

Oracle GoldenGate Delivery: All listed above, plus: NEW TimesTen, IBM i Series Netezza, Greenplum, and any ODBC compatible databases ETL products

8

If we can identify top 3 differentiators

Performance

Sub-second latency for data movement Low-impact capture, routing, transformation, and delivery of transactional data

Flexible & Extendable

Open, modular architecture Supports heterogeneous sources and targets Agile and adaptable to modern data integration architectures Maintains transactional integrity Resilient against interruptions and failures Fast Recovery after process interruptions.

Reliable

Oracle GoldenGateZero Downtime Upgrade & Migration New Application/ OS/ Database/Hardware Message Bus Query Offloading, Disaster Recovery Replica/ Standby

Oracle GoldenGateDatabase Distributed Systems

Data Synchronization across the Enterprise

Real-time BI, Operational Reporting ODS/Data Warehouse Legacy Message Bus Event Driven Architecture, SOA

Popular use cases of GG

App A1

1 Data Failure Protection 3 Real Time DW/BI 4 Offload DB Deployment

db A1

2 Zero DT Migrationapp A app A

solution X

db X

db A

db A

db A

5 Active Active DB Deployment 6 Regional DBs 7 Data level solutions/apps integration

db A

BI

DW regional part ofdb A


1 Data Failure Protection

app A

db A

db A

Data Failure Protection

Business ImpactsAvoid revenue & employee productivity loss Prevent decrease in customer loyalty and brand equity Decrease the manpower needed for system maintenance and recovery Limit credits and penalties due to not meeting SLAs for availability

Where it is happeningHA/MAA Infrastructure for Cloud Databases Minimizes risks during data center consolidation or migration to better platforms government regulations In Telecommunications OSS/BSS systems without interrupting operations In Financial Services/Banking Ensure continuous operations for critical applications such as payment processing, fraud detection, ATM, core-banking, trading systems Ecommerce/Retail Continuous operations for customer portal and POS systems

Active Data Guard and Golden Gate

Physical replication with Active Data Guard Best for Disaster Recovery for Oracle Applications Best for Disaster Recovery for Oracle Database 11g

Logical, heterogeneous replication with GoldenGate Only solution for real-time replication from Teradata Best for regular maintenance in heterogeneous environment

Active-Active bi-directional or multi-master replication with GoldenGate Best for custom and specialized applications


App A1

db A1

1 Data Failure Protection 2 Zero DT Migrationapp A

db A

Zero DT Migration

Business ImpactsAvoid revenue & employee productivity loss Prevent decrease in customer loyalty and brand equity Decrease the manpower needed for system maintenance and recovery Limit credits and penalties due to not meeting SLAs for availability SAP maintaince costs Oracle support costsModernize hardware and move to the most cost effective one

Where it is happeningHA/MAA Infrastructure for Cloud Databases Minimizes risks during data center consolidation or migration to better platforms government regulations In Telecommunications OSS/BSS systems without interrupting operations In Financial Services/Banking Ensure continuous operations for critical applications such as payment processing, fraud detection, ATM, core-banking, trading systems Ecommerce/Retail Continuous operations for customer portal and POS systems Move to the clouds Customers that want to consolidate databases and data centers to reduce costs Oracle DB customers that need ZDT Migration from 8i or 9i SAP customers Siebel CRM and JDE customers

Oracle GoldenGate 11g for Oracle Applications

Oracle GoldenGate for Application Upgrades & Migrations Siebel CRM (V6 V8 upgrading to V8.x) All platforms (Oracle, DB2(LUW & z/OS) & MSSQL) Also available to support Siebel Remote and incremental application repository zero downtime releases. JDE (all versions) Low downtime (not zero) upgrade solution All platforms except iSeries are supported. (iSeries is on the roadmap) SAP (Oracle to Oracle Online Migrations aka Triple-O)


App A1

1 Data Failure Protection 3 Real Time DW/BI

db A1

2 Zero DT Migrationapp A

db A

db A

BI

DW

Real Time DW/BI

Business ImpactsAgility to prevent costs/looses and achive revenue Performance of the production system (productivity) Decreasing Manpower costs for batch loading of DW

Where it is happeningCustomers want faster Apps (less invasive reporting), esspecially Oracle Apps BI customers that are looking to Use realtime or near real-time data from heterogeneous sources to heterogeneous targets in a mixed platform environment. BI customers that require transaction integrity for accurate and granular analysis ETL users that want to reduce overhead on source systems via log-based change data capture Also other BI customers that need stability of the BI environment BI customers where it t it takes too much time to load/refresh the Data Warehouse


App A1

1 Data Failure Protection 3 Real Time DW/BI 4 Offload DB Deploymentdb A

db A1


db A

db A

BI

DW

Oracle GoldenGate 11g for Oracle Applications

Oracle GoldenGate is Certified for Operational Reporting for: Oracle E-Business Suite v12.x and higher Oracle JD Edwards v9.x and higher Oracle PeopleSoft Enterprise v8.51 and higher Oracle Siebel CRM v8.x and higher


App A1

1 Data Failure Protection 3 Real Time DW/BI 4 Offload DB Deploymentdb A db A

db A1


db A

5 Active Active DB Deployment

db A

BI

DW

Active Active DB Deployments

Business Impacts Increased availabilty and performance but at the same time protect from data failure Better use of existing server assets

Where it is happening Telco


App A1

1 Data Failure Protection 3 Real Time DW/BI 4 Offload DB Deploymentdb A db A

db A1


db A

5 Active Active DB Deployment 6 Regional DBs

db A

BI



App A1


db A1


solution X

db X

db A

db A

db A


db A

BI


Oracle Data IntegratorMilomir Vojvodic

What is ODI and how it works?ODI Data ServicesData Access Service Transformation Service Batch Data Service

ODI Declarative Design ODIDeclarative Design

ODI Agent Data Flow Conductor

1Define

2Automatically Generate Dataflow

WhatYou Want

Define How: Built-in Templates

ODI Knowledge ModulesReverse Engineer Metadata Reverse Journalize Read from CDC Source Load From Sources to Staging Check Constraints before Load Integrate Transform and Move to Targets Service Expose Data and Transformati on Services W W W S S S

ODI E-LT

Staging Tables Load CDC Sources Journal ize Check Integrate Services

Target Tables Error Tables

Transform ExtractOracle Web Services DB2 Web Services

Transform Load

Sample out-of-the-box Knowledge Modules SQL Server Triggers Oracle DBLink JMS Queues Check MS Excel TPump/ Multiload Oracle Merge

SAP/R3

Log Miner

Siebel

DB2 Journals

DB2 Exp/Imp

Oracle SQL*Load er

Check Sybase

Type II SCD

Siebel EIM Schema

Benefits

27

27

If we can identify top 3 differentiators

Faster

Fastest E-LT bulk/batch performance Real-Time integration (sub-sec trickle) with CDC, replication, and SOA infrastructure Faster project setup, design and delivery Simpler to setup, configure, manage, & monitor Map using declarative SQL interfaces Deploy with fewer hardware devices Extend with knowledge module & code templates Less HW & Energy Costs with E-LT Architecture Less Time Wasted on Unnecessary ETL Mappings, Scripting, and Complex Training Less Integration Overhead Integrating with Applications, SOA, and Management Software

Simpler

Saves Money(Lower TCO, Higher ROI)

Capital Cost ReductionReduce or Eliminate Unneeded Hardware and Human Capital

ODI-based Oracle BI Apps Number of Setup Steps Number of Servers Number of connections 7 1 3

INFA-based Oracle BI Apps Number of Setup Steps Number of Servers 10 3

Number of connections

7


App A1


db A1


solution X

db X

db A

db A

db A


db A

BI


Which of them require data transformation and bulk data movement?App A1

db A1


3 Real Time DW/BI

solution X

db X

db A

BI

7 Data level solutions/apps integration

DW

Popular use cases of ODI

X, Y, A, B

solution X

solution Y

app B

app A

db XYAB

db XFiles Excel XML

db Y

db B

db A

1 Loading DW

2 Data level solutions/apps integration 3 Data migrations&conversions in modernization/consolidation projects

BI

DW

Not only DB

E-LTany

and E-T-L

OLTP to EDW/Mart

Any to OLTP DB

any

any

App/API to EDW/Mart

Any to File(s)

Any to App/API

any

File/WS/JMS to EDW/Mart

Any to OLAP


solution X

solution Y

app B

app A

db X

db Y

db B

db A

1 Loading DW

Files Excel XML

BI

DW

Loading DW

Business Impacts Comparing to Manual Coding : Productivity (number of langauges to use, prebuilt code (no need to write SQL)) Maintaince (hard to change smt, every script contains special rules in several machines)and Peformance and availabilty and Costs (E-LT) Comparing to other ETL : Costs (ELT) and Productivity (still manual coding and SQL tuning)

Where it is happeningCustomers with recent M&A activities (looking to reduce capital costs) Customer has dedicated cluster of ETL servers running at/near capacity and is looking to add capacity Large teams of ETL development staff are fully engaged with maintenance activity and not able to take on new projects Siebel, PeopleSoft, JDEdwards applications customers since they are more likely to have heterogeneous environments and to value heterogeneous database support. Oracle BI Applications, Oracle EPM, Oracle BI EE Plus


solution X

solution Y

app B

app A

db XFiles Excel XML

db Y

db B

db A

1 Loading DW

2 Data level solutions/apps integrationBI

DW

Data Level Solutions/Apps Integration

Business ImpactsOne architecture for all integration tasks No more performance issues No more maintenance complexity (easy change a transformation rule) get a project done in a couple of weeks than a couple of months

Where it is happeningCustomers with heterogeneous sources or targets in a mixed platform environment and with advanced knowledge of database technologies and understanding of database power For the data like Customer, Product, Supplier, Financials to create customData Hubs and Master Data Mgmt Mergers and acquisitions SOA customers Customers want to standradize on Java

Data Hubs in Key IndustriesFinancial Services Single view of high quality customer data drives accurate customer insight and improved marketing effectiveness Supports compliance and reporting KYC requirements

Retail

Harmonizes customer data from multiple channels to improve sales and marketing effectiveness Enhances opportunity and customer lifetime value identification Improves customer insight for revenue optimization and targeted customer retention Effective compliance and risk mitigation for next generation services Expands understanding of network assets and customer delivery points Improves management of regulatory compliance and reporting requirements Delivers a comprehensive view of patient for care and billing Manages patient, epidemiology, diagnosis and treatment data quality across systems and organizations Single view of citizen for better internal information sharing, service delivery, licensing, provision of child care, and fraud detection Reduce costs through system rationalisation

Telco

Energy & Utilities Utilities

Healthcare

Government

38

Understanding Performance ChoicesLess than 10MBDepends on whether an intermediary XML format is useful for other processing (use ESB), or if joining File data to tabular RDB data is required (use ODI)

XML ESB ESB ESB XML ESB depends

File ESB ESB depends File depends ODI

DB ESB depends ODI DB ODI ODI

XML File DBBetween 10-50MB

Depends on ho much cross-referencing among the data values and rows is required during transformation the more there is, the faster ODI will perform relative to ESB

XML File

DBGreater than 50MB

ODIXML depends ODI ODI

ODIFile ODI ODI ODI

ODIDB ODI ODI ODI

If the source and target are both XML, and there is no cross-referencing of data among rows, then a streaming-type or parallel-engine-type approach might scale

XML File DB

*caveat always benchmark if you are unsure and require best possible results


X, Y, A, B

solution X

solution Y

app B

app A

db XYAB

db XFiles Excel XML

db Y

db B

db A

1 Loading DW

2 Data level solutions/apps integration 3 Data migrations&conversions in modernization/consolidation projects

BI

DW

Data migrations&conversions in modernization/consolidation projectsBusiness ImpactsGet double-duty from database investment by using it for transformations Reduced migration time and expense

Where it is happeningVirtualization Cloud Exadata (ODI for mixed workload use cases and handle both loading and querying workloads simultaneously.) Application Migrations require heavy data transformation mainframe application to open systems (major schema changes) new generation of DW and BI solutions

ODI Outperforms Informatica for Less

ODIE 7.5 TB /hr $0 ETL H/W L T

7tb/hr

Run ODI Directly on Exadata Complex Data Transformations Linear ETL Scalability Fully Leverages DBFS/Infiniband, Smart Storage, and Advanced Compression

4:1 advantage*

$5m 3yr savings**

1 Exadata X2-8

Informatica1.8 TB /hr

$5m ETL H/W

E

T

1.8 tb/hr

L

* TPC-H data sets with transformations ** Production hardware savings(not including Dev + Test environments, management costs or software savings)

HP Superdome 64 + XP24000 w/ Flash

Copyright 2011, Oracle Corporation and/or its affiliates

DIS Packaging, References and CompetitiveMilomir Vojvodic

Data Integration & Availability Pillar Products

$17,500(Src & Trgt DB CPUs)

Oracle GoldenGateFor Oracle (includes Active Data Guard) For Non-Oracle Database (Includes Teradata)

Use Case: HA / Real-time DI Options: Management Pack ($3,500 per CPU) Application Adapters ($20,000 per CPU) Veridata ($30,000 per mid-tier CPU) Use Cases: HA / Real-time DI Change Capture for Data Warehouses Use Case: Bulk Data Transformation Options: Application Adapters ($2300 per CPU) Use Cases: ODI-EE with Clustering & ODI-EE for SOA Use Cases

$100,000(Src & Trgt DB CPUs)

Oracle GoldenGate for MainframeIncludes HP Non-stop and DB2 Mainframe

$23,000(Target DB CPUs)

ODI Enterprise EditionOracle Data Integrator Oracle Warehouse Builder Enterprise ETL

$70,000(Target DB CPUs)

ODI SuiteOracle Data Integrator Enterprise Edition Oracle Data Integrator Management Pack WebLogic (App Server) Suite (restricted) Oracle Coherence (restricted) Oracle Service Bus (restricted) Oracle BPEL Process Manager (restricted)

$10,000(per CPU)

Active Data GuardRequires Oracle Database Enterprise Edition

Use Cases: Disaster recovery & data protection for Oracle DB

Oracle Data Integration SolutionsHigh Availability & ReplicationPlatform Specific (logical HA) Block-Level Solutions (physical HA)

Sybase Replication Server

EMC Symantec / Veritas Quest Shareplex Hitachi Data Systems Microsoft SQL Replication IBM PPRC Not suitable for HP StorageWorks heterogeneous, enterprise deployments no transaction management, no transformation, homogenous Data IntegrationETL Solutions EAI Solutions

IBM Infosphere (Ascential) Informatica SAP / Business Objects Ab Initio Invasive to source, Not real time, Requires Mid-Tier, Expensive & Monolithic

Tibco Progress Software IBM Websphere MQ Lower throughput, more overhead, not direct to database

45

Oracle ETL strategyMilomir Vojvodic

Evolving Data Integration Roadmap

Roadmap 2000 to 2007 2006

Roadmap 2007 to 2008 2007

OWB-EETL

ODI OWB-EETLOWB (free DB features)

OWB

OWB (free DB features)

Roadmap 2009 to 2010

Roadmap from Jan 2010 to Current

KMs

New ETL

ODIKMs



OWB Support PoliciesCustomer Downloadable OWB Support Policies: http://www.oracle.com/us/support/library/lifetime-support-technology-069183.pdf Release GA Date Premier Support Ends Extended Support Ends Sustaining Support Ends

OWB 9.2

Jul 2003

Oct 2007

Not Available

Oct 2010

OWB 10.1OWB 10.2 OWB 11.1

Apr 2004Jul 2005 Aug 2007

Jul 2007Jul 2010 Aug 2012

Not AvailableJul 2013 Aug 2015

Jul 2010Indefinite Indefinite

OWB 11.2

Sep 2009

Current Dates: Jan 2015Future Dates: Same as DB12

Current Dates: Jan 2018Future Dates: Same as DB12

Indefinite

For more Information about Support Levels: http://www.oracle.com/us/support/lifetime-support/index.html

OWB License MigrationsExisting Product License OWB ETL features (included in Database licenses) OWB Enterprise ETL option by CPU Migrating to ODI EE license No migration credit applies. Ratio 2:1 Notes Pay full ODI EE price. Using list-to-list migration.

OWB Enterprise ETL option by NUP

Ratio 100:1

Using list-to-list. If customers have less than 100 NUP, use a percentage credit. i.e. 25 NUP would = 25% credit toward purchase of 1 processor. Customer pays the remaining 75% of the list price, minus standard and approved discounts.All 9i features became free with OWB 10.1. Note that OWB SAP Connector migrations are covered in the eSource Migration Listing

OWB included in internet Developer Suite (9i) or OWB 2.x versions

No migration credit applies.

OWB DQ and DP

No migration credit applies.

In some cases customers may buy more Licenses (if they already own some of the same type) For additional License Migration queries, contact [email protected] Additional discounts or price promotions may apply, contact [email protected]

Oracle ETL Product BundlesFeaturesBasic features for Oracle data loading

DB (free ETL)Y Oracle Gateways Y Oracle OLAP only Oracle DBMS only -

ODI-EEY Y Y Y Y Y Y Y Y Y Y

ODI SuiteY Y Y Y Y Y Y Y Y Y Y

Oracle DB Targets Non-Oracle Sources Slowly Changing Dimensions (type 1) OLAP Targets (Oracle + Essbase) Built-in Scheduler

Standard features for enterprise ETL use cases

Non-Oracle DB Targets Changed Data Capture (including OGG integration) Web Services (API & Src/Trgt) & XML Targets Data Lineage / Impact Analysis Declarative Design Environment Slowly Changing Dimensions (2 & 3)

Load Plans, Parallel Execution and RestartabilityJEE Deployment , Clustering and High Availability Java based SDK for full Programmatic Control Unified Management and Administration with OEM Pre-built Integration with OBI-EE, Hyperion, SOA/AIA, etc.Advanced features for enterprise information management

-

YY Y Y Y -

YY Y Y Y Y Y Y Y

Human Workflow (BPEL PM) Process Orchestration & Service Bus Data Relationship Management / MDM WebLogic Suite (Coherence Data Grid, B2B, Biz Rules)

OWB Enterprise ETL

Accessing sources and targetsNative support for heterogeneous databases, including the definition of new platform types, JDBC connectivity, knowledge modules, and code templates Change Data Capture mappings in ETL Publishing and consuming web services, including integrating with process orchestration technologies such as Oracle BPEL Process Manager Importing Cobol copybooks using the built-in wizard Targeting XML files

Designing and executing ETL jobsRight-time data warehousing utilizing queues and real-time mappings Reusing and embedding mapping code with pluggable mappings Specifying the load order of multiple targets in a single mapping Partitioning and parallelizing updates to tables by utilizing the data chunking feature Scheduling ETL jobs using the built-in Schedule interface

Documents

Milomir Vojvodic Oracle DIS