46627675 Oracle Golden Gate (1)

8/4/2019 46627675 Oracle Golden Gate (1)

1/15

Background, Overview of GG, Next Stepsetc.

Oracle GoldenGateOverview & Demo / Instructions


2/15

November 2010 2

Agenda

Oracle Golden Gate

Overview

Deep-dive Architecture

Demo


3/15

November 2010 3

Oracle GoldenGate provides real-time data integration to support high

availability capabilities across heterogeneous systems to achievebusiness continuity.

GoldenGate provides the ability to capture and deliver updates of

critical data as the changes occur and provide continuous synchronized

data across heterogeneous environments.

Use Cases: Online banking, Claims processing and Online shopping

can use GoldenGate to operate without any disruption.

Overview of Oracle Golden Gate


4/15

November 2010 4

Overview of Golden Gate Contd.

Log Based, Real-

Time Change Data

Capture

Heterogeneous

Source Systems

EDWODS

EDW

Active-Active High

Availability

Zero Downtime

Migration andUpgrades

Real-time BI

Fully Active

Distributed Database

Reporting

Database

ETL

ETL

Query Offloading

Data Distribution

New DB/OS/HW/App

Global Data Centers

SOA/EDA

GoldenGate

Provides

Low-impact capture

Routing

Transformation andDelivery of transactional data across

Heterogeneous environments in

Real time

Key Features

Non-intrusive, low-impact,

sub-second latency

Open, modular architecture

- Supports heterogeneous

sources and targets

Maintains transactionalintegrity - Resilient against

interruptions and failures

Performance

Flexible /

Extensible

Reliable


5/15

November 2010 5

Oracle GoldenGate

LAN/WAN

Internet

TCP/IP

CaptureTrail

Pump DeliveryTrail

Trail: stages and queues data for routing.

Pump: distributes data for routing to target(s).

Route: data is compressed,

encrypted for routing to target(s).

Delivery: applies data with transaction

integrity, transforming the data as required.

Source

Oracle & Non-Oracle

Database(s)

Target

Oracle & Non-O

Database(s)

Capture: Committed transactions are captured (and can be filtered) real time, from the

DB Redo logs.


6/15

November 2010 6

Oracle GoldenGate

LAN/WAN

Internet

TCP/IP

CaptureTrail

Pump Delivery (SQL)Trail

Trail: stages and queues data for routing.

Pump: distributes data for routing to target(s).

Route: data is compressed,

encrypted for routing to target(s).

Delivery: applies data with transaction

integrity, transforming the data as required.

Source

Oracle & Non-Oracle

Database(s)

Target

Database(s)Bi-directional

Capture: Committed transactions are captured (and can be filtered) real time, from the

DB Redo logs.


7/15

November 2010 7

GoldenGate Deployment Topologies

Unidirectional

Query OffloadingZero-Downtime Migration

Bi-Directional

Hot Standby orActive-Active forHA

Peer-to-Peer

Load Balancing,Multi-Master

Broadcast /Fan-out

Data Distribution

Integration/Consolidation/Fan-in

Data WarehouseData Distribution

via Messaging

BPM


8/15

November 2010 8

Architecture Components (Processes)

Log-based VAM-based

Local/Remote Queuing

Filtering

Parallel Coordination

Local/Remote

Native database API

ODBC

Batched Operations

Filtering

Transaction/Operations Grouping

Parallel Coordination

Local/Remote

CAPTURE DELIVERY

PUMP (routes from one side to other) MANAGER (Start/Stop/Report on thresholds, trail purging etc.)

Filtering

Local/Remote

TCP/IP

Compression

Encryption

Queue (Trail) Management

Process Management (Dynamic)

Monitoring and Administration (Lag reports) Reporting (Events, Errors, Thresholds)

Receive/Route requests from UI

On-disk Components

Trail Fil s ck i t Fil s ( a t r , , liv ry)

ata fi iti Fil s (S E EFS/T ET EFS)

fig

rati

Fil

s

iscar Fil s

rt/L g Fil s

Ex c ta l s


9/15

November 2010 9

TargetSource

TCP/IP

Network

Process Start

Shared Memory (for Monitoring)

TargetDatabase

Capture DeliveryPumpSource Trail Target TrailCollector

GGSCIInteractive

Interface

Manager ManagerGGSCIInteractive

Interface

Oracle GoldenGate ComponentsCommunications and Startup


10/15

November 2010 10

Capture

Memory Manager

Database Access To gather Metadata (to interpret log data) & forSQL interface for data lookup, advanced filter ing etc.

Redo API(Log reader

interface)

MetadataManager

SQLInterface

FilteringFormattingTransformationand Mapping

Checkpoint Manager

TransformationEngine

Serializationand Routing

Messaging Subsystem

Transaction

Log Reader

TransactionLogs

GoldenGateTrail Files

GoldenGateEvent Log

GoldenGateCheckpoint File

Parser

GoldenGateParameter File

Command

and Control

GoldenGateCommandInterpreter

Metadata

Manager

SourceDatabase

GoldenGate Capture


11/15

November 2010 11

Delivery

Memory Manager

Database Access For Metadata access & SQL interface

Read API

SQLInterface

FilteringFormattingTransformationand Mapping

Checkpoint Manager

TransformationEngine

SQLGeneration

Messaging Subsystem

GoldenGate

Trail Reader

GoldenGateEvent Log

GoldenGateCheckpoint File

Parser

GoldenGateParameter File

GoldenGateCommandInterpreter

Commandand Control

Metadata

Manager

GoldenGateTrail Files

TargetDatabase

GoldenGate Delivery


12/15

November 2010 12

GoldenGate Instantiation

Start delivery at or after given backup CSN (245 in this case)

Current source Commit Sequence Number (CSN) is 356

Once Delivery is current the source and target are in sync

Capture DeliveryPumpCommitOrderedSource Trail

Commit OrderedTarget Trail

Source

Database

TargetDatabase

CSNs 222

through 356

Deliver

transactions

after CSN 245

Current CSN is

356

Consistent as

of CSN 356

In Sync!

Take Backup

as of CSN

245

Apply Backup

as of CSN

245


13/15

November 2010 13

Initial Data Load

Source:

Create the Initial data extract process load1

ADD EXTRACT load1, SOURCEISTABLE

Create Parameter file for Load1: Edit params load1

Target:

Create initial data load task load2

ADD REPLICAT load2, SPECIALRUN

Create Parameter file for Load2: Edit params load2

Start Initial Load:

START EXTRACT load1

INFO EXTRACT load1


14/15

November 2010 14

Online Change Synchronization

1. Create a GoldenGate Checkpoint table

EDITPARAMS ./GLOBALS GGSCHEMA GGS_OWNER

CHECKPOINTTABLE GGS_OWNER.CHKPTAB

DBLOGIN USERID ggs_owner, PASSWORD ggs_ownerADD CHECKPOINTTABLE GGS_OWNER.CHKPTAB

2. Create an Extract groups on Source ADD EXTRACT ext1, TRANLOG, BEGIN NOW

ADD EXTRACT dpump, EXTTRAILSOURCE /ora/app/stage/GoldenGate-Zip/ggs/dirdat/lt

3. Create a parameter file for the online Extract group EDITPARAMS ext1

4. Create a Trail on Source & Target

On the source/target system, the trail files will be created having a prefix rt/ltwhich will be used by the Replicat/Extractprocesses also running on the target /source system(s)

ADD EXTTRAIL /ora/app/stage/GoldenGate-Zip/ggs/dirdat/lt, EXTRACT EXT1 ADDRMTTRAIL /ora/app/stage/GoldenGate-Zip/ggs/dirdat/rt, EXTRACT dpump

5. Create a Replicat group on the Target ADD REPLICAT rep1, exttrail /ora/app/stage/GoldenGate-Zip/ggs/dirdat/rt, checkpointtable chkptab

6. Create a parameter file for the online Replicat group EDITPARAMS rep1

Start Online Change Sync: START EXTRACT ext1

START EXTRACT dpump

STARTREPLICAT rep1


15/15

November 2010 15

Significance of Data Pump

When Data Pump is not used, the Extract process writes to a remote trail that is locatedon the target system using TCP/IP

When Data Pump is configured, Extract process writes to a local trail for Data Pump toread and write over the network to the remote trail located on the target system

The advantages of Data Pump can be seen as it protects against a network failure as inthe absence of a storage device on the local system, the Extract process writes data intomemory before the same is sent over the network

Any failures in the network could then cause the Extract process to abort (abend)

Additionally, if there are any complex data transformations or filtering, Data Pump can beused to perform these

Documents

46627675 Oracle Golden Gate (1)