26

Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Embed Size (px)

Citation preview

Page 1: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4
Page 2: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Data Integrator 12c

Joachim Jaensch Principal Sales Consultant

New Features

Page 3: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3

Page 4: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

What‘s New In Oracle Data Integrator 12c?

Oracle OpenWorld 2014 4

• Declarative Flow-Based User Interface

• Reusable Mappings

• Multi-Threaded Target Load

• Step-by-Step Debugger

• Runtime Performance Enhancement

• Oracle GoldenGate Integration Improvements

• Standalone Agent Management with WebLogic Management Framework

• Integration with Oracle Platform Security Services (OPSS) Enterprise Roles

• JSON and XML Improvements

• Oracle Warehouse Builder (OWB) Integration

• OWB to ODI Migration Tool

• OEM – Management Pack for Oracle Data Integrator

• New Mapping Components: PIVOT, UNPIVOT

Page 5: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 5

Reusable Mappings

• Capture common logic (sources, targets, transformations)

• Reuse logic in mappings

Page 6: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Multi-Threaded Target Load

Source Target Database

ODI Agent

Parallel Load

Threads

JDBC

Data Server settings: • Fast Load through parallel JDBC

connections

• Configurable per Data Server

Page 7: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 7

In-Session Parallelism

• Goal

• Allow the execution of tasks and steps in

parallel rather than sequentially

• How

• Introduction of a new Runtime concept:

Execution Units

• In 12.1.2 loading of C$ tables can be

executed in parallel

ODI 11g

ODI 12.1.2

SrcSet1

SrcSet2

SrcSet3

Integration

SrcSet1

SrcSet2

SrcSet3

Integration

Total Execution Time

Total Execution Time

Page 8: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 8

Low Overhead Session Execution

• Goal • Reduce status and session management overhead

• How • Cache and reuse a Session Blueprint instead of accessing the ODI repositories

• Avoid writing logs in the repository when it is not necessary

• Benefits • Better performance when executing scenarios multiple times, when restarting a session,

when logging sessions activity

• Less fragmentation of repository tables

Page 9: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Concurrent Execution Control

• You can now limit concurrent executions in a scenario or load plan and force a concurrent execution to either wait or raise an execution error

Page 10: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Step by Step Debugger

• Give control to the user on the Execution Flow of a Session

• Pause, Resume, Run to next Step, etc.

• Define breakpoints on steps in packages

• Toggle breakpoints in Sessions

• Query data through agent

(gives access to uncommitted data)

• Start a session with the option to wait for user inputs on Error

Page 11: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

One-Step Model and Topology Creation

• One dialog to create all metadata objects:

• Data Server

• Physical and Logical Schema

• Context

• Model

• Simplifies and accelerates ODI development

Page 12: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 12

Integration Between GoldenGate and ODI 12c

• Support Real-Time and Bulk from the same Interface

• Deploy GoldenGate Processes via ODI

• Enable GoldenGate Journalizing on Interfaces without

restructuring.

• Manage GoldenGate Capture/Delivery instances in ODI

Topology

• Set common GoldenGate parameters graphically

Bulk Load Scenario Real Time Scenario

Extract Replicat

Page 13: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Security Enhancements

• Product-wide standardization on AES encryption

• FIPS compliant (US)

• AES key 128bit by default, algorithm and key length can be configured: 128, 192 & 256.

• ODI import and export uses secure encryption for sensitive data

Page 14: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Improved JSON and XML Support

PK_CUST first last

1 John Smith

{ "first": "John", "last": "Smith", "phone": [ { "type": "home", "number": "212 555-1234" }, { "type": "work", "number": "646 555-4567" } ] }

PK_PHONE FK_CUST type number

1 1 home 212-555-1234

2 1 work 646-555-4567

Datastore PHONE

Datastore CUST

JSON • Complex File Driver reads and writes JSON

• Conversion to tabular Datastore format

• XML support for advanced constructs: • any, anyType, anyAttribute

• Custom XML column mappings

Page 15: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 15

• New ‘OWB Runtime Repository’ technology available in Topology

• OWB processes can be invoked using the OdiStartOwbJob tool in Packages

• OWB audit information is available in Operator, ODI Console and Enterprise Manager

Invoking OWB 11.2 Processes from ODI 12c

Page 16: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Oracle Warehouse Builder Migration

• Support for incremental migration • Objects already created by a previous migration will now be reused when MIGRATION_STRATEGY

option is set to NODUP

• Support for migrating to ODI 12.1.2 and 12.1.3 from OWB 11.2.0.3 and 11.2.0.4

• Bug fixes from the previous release

Page 17: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Management Pack for Oracle Data Integrator (1)

• New in the 12.1.0.6 FMW Plugins release

• IBM DB2 support (repository and source/target)

• Microsoft SQL Server support (repository and source/target)

• Redesigned Load Plan Execution/Sessions page

• More information at http://my.oracle.com/site/pd/fmw/products/DI/SalesPlays/mgmtodi/index.html

• Additional License needed

Page 18: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 18

• Discovery of ODI components

• Agents: Standalone and JEE

• Repositories

• Source and Target databases

• Monitoring

• Availability Monitoring

• ODI homepage to quickly assess infrastructure status

• Performance Monitoring

• ODI components performance

Management Pack for Oracle Data Integrator (2)

Page 19: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

ODI Big Data – HBase and Hive Support

• HBase now supported as a Big Data source or target

• New Knowledge Modules for Hbase • RKM HBase

• LKM HBase to Hive

• IKM Hive to HBase Incremental Update

• Support for Hive Append

• Existing KMs enhanced to support append

Page 20: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

ODI Big Data – SQOOP Load

• Fast Load between databases and Hadoop

• Database can be source or target

• HDFS, Hive, or HBase supported

• Parallelized load using a data partitioning scheme

Oracle Data Integrator

SQOOP HDFS

SQOOP HDFS

SQOOP HDFS

Parallel JDBC Queries

Source Database

Page 21: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | Oracle OpenWorld 2014 21

Terminology

• Mappings was renamed to Interfaces

Page 22: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Useful Links

• ODI 12c New Features:

http://www.oracle.com/us/products/middleware/data-integration/odi-12c-new-features-wp-2226353.pdf

• PM Wiki:

http://my.oracle.com/site/pd/fmw/products/di/odi/index.html

• ODI 12.1.3 Demo VM: http://retriever.us.oracle.com/apex/f?p=121:22:9208677187768:22:::P22_CONTAINER_ID:68263

• ODI 12.1.3 OTN VM:

http://www.oracle.com/technetwork/middleware/data-integrator/odi-demo-2032565.html

• Blogs

• Data Integration: https://blogs.oracle.com/dataintegration/

• A-Team: http://www.ateam-oracle.com/?cat=64

Page 23: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Resources

23

Oracle Data Integration Oracle Data Integration OracleGoldenGate ORCL DataIntegration blogs.oracle.com/dataintegration

Oracle Data Integrator

Oracle GoldenGate

Oracle Enterprise Data Quality

Oracle Enterprise Metadata Management

Oracle Data Services Integrator

http://www.oracle.com/us/products/middleware/data-integration/overview/index.html

Data Integration

Oracle OpenWorld 2014

Page 24: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. |

Questions and Answers

24 Oracle OpenWorld 2014

Page 25: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4

Copyright © 2014 Oracle and/or its affiliates. All rights reserved. | 25

Page 26: Oracle Data Integrator 12c - doag.org€˜s New In Oracle Data Integrator 12c? Oracle OpenWorld 2014 4