Upload
eustace-wilcox
View
220
Download
0
Tags:
Embed Size (px)
Citation preview
OPS-24: Success with OpenEdge® Replication
What you don’t know can hurt you
Cyril GleimanPrincipal Technical Support Engineer
© 2008 Progress Software Corporation2 OPS-24: Success with OpenEdge Replication
Obligatory (gratuitious) Quotation
An ounce of prevention is worth a pound of cure.
28.35 grams of prevention is worth .45 kilograms of cure
© 2008 Progress Software Corporation3 OPS-24: Success with OpenEdge Replication
Success With Replication
Understand the architecture Know your requirements Analyze your system Plan, configure, deploy,monitor Succeed
© 2008 Progress Software Corporation4 OPS-24: Success with OpenEdge Replication
Agenda
Terminology and architecture – Visual Replication performance hotspots Replication availability hotspots Source and target management
© 2008 Progress Software Corporation5 OPS-24: Success with OpenEdge Replication
Replication Terminology
Primary – original production machine Secondary – original recovery machine Source – production database Target – recovery database Replication server – rpserver (source) Replication agent – rpagent (target) DBService queue – stores IPC messages Network pipe – TCP/IP rpserver -> rpagent
© 2008 Progress Software Corporation6 OPS-24: Success with OpenEdge Replication!
Architecture of Replication – Primary/Source
Primary Machine
R/WriteClients
AI Extents
DB Service
Q
Repl Server
AI ExtentsSource
Database
Database Brk/Server Processes
© 2008 Progress Software Corporation7 OPS-24: Success with OpenEdge Replication
Architecture of Replication – Secondary/Target
Secondary Machine
Repl Agent
Database Brk/ServerProcesses
TargetDatabase
R/OClients
© 2008 Progress Software Corporation8 OPS-24: Success with OpenEdge Replication
Architecture of Replication
Primary Machine Secondary Machine
Repl Agent
Database Brk/ServerProcesses
TargetDatabase
R/OClients
R/WriteClients
AI Extents
DB Service
Q
Repl Server
AI ExtentsSource
Database
Database Brk/Server Processes
© 2008 Progress Software Corporation9 OPS-24: Success with OpenEdge Replication
Agenda
Terminology and architecture - Visual Replication performance hotspots Replication availability hotspots Source and target management
© 2008 Progress Software Corporation10 OPS-24: Success with OpenEdge Replication
Performance Hotspots
Primary Machine Secondary Machine
Repl Agent
Database Brk/ServerProcesses
TargetDatabase
R/OClients
R/WriteClients
AI Extents
DB Service
Q
Repl Server
AI ExtentsSource
Database
Database Brk/Server Processes
© 2008 Progress Software Corporation11 OPS-24: Success with OpenEdge Replication
Source DB Performance Considerations
Speed/power of machine Replication Plus – offload read only clients to
target database AI files – fixed extents
AI/BI blocksizes should be the same• 16 K is the most efficient
• Need to truncate ai and bi
© 2008 Progress Software Corporation12 OPS-24: Success with OpenEdge Replication
Target DB Performance Considerations
System should not be underpowered by comparison to the source system
No need for after-imaging Read only clients
© 2008 Progress Software Corporation13 OPS-24: Success with OpenEdge Replication
Network Performance
WAN vs. LAN The bigger the pipe the better Determine size of pipe Whitepaper available on PDSN http://tinyurl.com/6xqp78
© 2008 Progress Software Corporation14 OPS-24: Success with OpenEdge Replication
Sample Network Bandwidth Calculation
Hourly After Image Blocks from 5 production databases = 713mb
By calculation:• Replication size = 1.5 * AI size
– 713mb*1.5 = 1069.5mb
• Add replication overhead (1.1)– 1069.5mb*1.1 = 1176.45mb
Throughput per second:• 1176.45mb/3600=.32679mb/b
Change to kilobytes /s:• .32679mb/s * 1000 = 326.79kb/b needed to sustain
transfer
© 2008 Progress Software Corporation15 OPS-24: Success with OpenEdge Replication
Replication Performance – DBService Buffer
DBService Buffer Queue must be sized appropriately – Solution P121969• Failure to do so may result in source
slowdown and target falling behind
Analyze AI activity prior to implementing• Use PROMON -> R&D -> Status Display (#1)-
> DB Service Manager (#16).
© 2008 Progress Software Corporation16 OPS-24: Success with OpenEdge Replication
Determining Optimal -Pica
The larger the –pica, the further behind the target may get
Monitor AI writes during busiest period for one hour
-pica = (TAIW/Blockcount) * 1.25 Blockcount depends on version
• OpenEdge 10 – blockcount is 9.16
• Progress® 9.1x – blockcount is 18.2
© 2008 Progress Software Corporation17 OPS-24: Success with OpenEdge Replication
Sample –Pica Calculation
Values up to 8192k for 10.1B01 and later 34560 TAIW over one hour Formula for OE 10:
• (34560/9.16) * 1.25 = 4716.1572052401746724890829694323
Why not just use 8192???
© 2008 Progress Software Corporation18 OPS-24: Success with OpenEdge Replication
“Houston, we have a problem” - Promon
03/03/08 Status: Database Service Manager
17:15:02
Communication Area Size : 2049.00 KB
Total Message Entries : 18733
Free Message Entries : 4
Used Message Entries : 18729
Registered Database Service Objects
Name Rdy Status Messages Locked by
OpenEdge Replication Server Y RUN 18729
OpenEdge RDBMS Y REG 0
OpenEdge DB Agent Y RUN 0
© 2008 Progress Software Corporation19 OPS-24: Success with OpenEdge Replication
Agenda
Terminology and Architecture - visual Replication performance hotspots Replication availability hotspots Source and target management
© 2008 Progress Software Corporation20 OPS-24: Success with OpenEdge ReplicationCOMP-10: OpenEdge® Management & Replication – Divide et impera!
Availability Hotspots
Primary Machine Secondary Machine
Repl Agent
Database Brk/ServerProcesses
TargetDatabase
R/OClients
R/WriteClients
AI Extents
DB Service
Q
Repl Server
AI ExtentsSource
Database
Database Brk/Server Processes
© 2008 Progress Software Corporation21 OPS-24: Success with OpenEdge ReplicationCOMP-10: OpenEdge® Management & Replication – Divide et impera!
Availability Hotspots
Secondary Machine
Repl Agent
Database Brk/ServerProcesses
TargetDatabase
R/OClients
R/WriteClients
AI Extents
DB Service
Q
Repl Server
AI ExtentsSource
Database
Database Brk/Server Processes
DOWN!
Severed
Primary Machine Loss of Database TCP or Process Failure
Locked
© 2008 Progress Software Corporation22 OPS-24: Success with OpenEdge Replication
Availability Hotspots – AI Management
Replication DOES NOT manage AI files AI Files must be emptied and backed up
• 10.1A AI archiver became available.
Database crashes when no empty AI extents.
© 2008 Progress Software Corporation23 OPS-24: Success with OpenEdge Replication
Availability Hotspots – Locked AI Files
AI “locked” when AI notes not replicated to the target db• Common issue – often caused by
simple maintenance routines and failure to monitor status of replication
Configure the rpserver and rpagent to detect and handle outages
All about the *.repl.properties file!! Configure, test well and sleep easier at night
© 2008 Progress Software Corporation24 OPS-24: Success with OpenEdge Replication
Availability – Many Locked AI Files
Use dsrutil monitor against source and target db. Check for replication shared memory
If rpagent is running, restart the replication server on the source
If rpagent is not running, restart target database and restart replication server
© 2008 Progress Software Corporation25 OPS-24: Success with OpenEdge Replication
“Houston, we have a problem” – part 2
Extent: 1 Extent: 3Status: Busy Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a3 Size: 3192 Size: 6264 Used: 3180 Used: 6200 Start: Wed May 14 14:09:34 2008 Start: Wed May 14 14:08:04 2008 Seqno: 5 Seqno: 3
Extent: 2 Extent: 4Status: Locked Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a4 Size: 230008 Size: 25208 Used: 229674 Used: 25063 Start: Wed May 14 13:45:30 2008 Start: Wed May 14 14:08:24 2008 Seqno: 2 Seqno: 4
© 2008 Progress Software Corporation26 OPS-24: Success with OpenEdge Replication
Availability – All AI Files Locked
Source database activity comes to a halt Can you afford to bring the db down?
• If not - disablesitereplication
• If yes – stop database– add new ai files– prostrct reorder– restart.
© 2008 Progress Software Corporation27 OPS-24: Success with OpenEdge Replication
Prostrct add sourcedb addai.st
Extent: 1 Extent: 4Status: Busy Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a4 Size: 3192 Size: 25208 Used: 3180 Used: 25063 Start: Wed May 14 14:09:34 2008 Start: Wed May 14 14:08:24 2008 Seqno: 5 Seqno: 4
Extent: 2 Extent: 5Status: Locked Status: Empty Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a5 Size: 230008 Size: 120 Used: 229674 Used: 0Start: Wed May 14 13:45:30 2008 Start: N/ASeqno: 2 Seqno: 0
Extent: 3 Extent: 6Status: Locked Status: Empty Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a3 Path: C:\wrk101c\repl\source.a6 Size: 6264 Size: 120 Used: 6200 Used: 0Start: Wed May 14 14:08:04 2008 Start: N/ASeqno: 3 Seqno: 0
© 2008 Progress Software Corporation28 OPS-24: Success with OpenEdge Replication
Prostrct reorder ai sourcedb
Extent: 1 Extent: 4Status: Busy Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a1 Path: C:\wrk101c\repl\source.a4 Size: 3192 Size: 230008 Used: 3180 Used: 229674Start: Wed May 14 14:09:34 2008 Start: Wed May 14 13:45:30 2008 Seqno: 5 Seqno: 2
Extent: 2 Extent: 5Status: Empty Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a2 Path: C:\wrk101c\repl\source.a5 Size: 120 Size: 6264 Used: 0 Used: 6200Start: N/A Start: Wed May 14 14:08:04 2008 Seqno: 0 Seqno: 3
Extent: 3 Extent: 6Status: Empty Status: Locked Type: Variable Length Type: Variable Length Path: C:\wrk101c\repl\source.a3 Path: C:\wrk101c\repl\source.a6 Size: 1 20 Size: 25208 Used: 0 Used: 25063Start: N/A Start: Wed May 14 14:08:24 2008 Seqno: 0 Seqno: 4
© 2008 Progress Software Corporation29 OPS-24: Success with OpenEdge Replication
Source and Target DBs Won’t Synchronize
Try restarting a few times. -Ma, -Mn, -n match? Changes to .properties files? If it continues to fail, contact support
• Source/target db log files
• AI files
• pmmgr.properties file– arguments=-logging 2
© 2008 Progress Software Corporation30 OPS-24: Success with OpenEdge Replication
Agenda
Terminology and architecture - visual Replication performance hotspots Replication availability hotspots Source and target management
© 2008 Progress Software Corporation31 OPS-24: Success with OpenEdge Replication
Managing Replication – DSRUTIL Monitor and Status
Not immediately obvious when a failure occurs
Proactively verify replication performance and status
© 2008 Progress Software Corporation32 OPS-24: Success with OpenEdge Replication
DSRUtil Monitor
DSRUTIL source/target –C monitor• Attaches to replication shared memory.
– Cannot connect to replication shared memory. Status = -1
• Checks status of Server and Agent– Server status– Agent status
© 2008 Progress Software Corporation33 OPS-24: Success with OpenEdge Replication
DSRUtil source –C monitor
OpenEdge Replication Monitor Page 1 Database: C:\wrk101c\repl\source
Database is enabled as OpenEdge Replication: Source
Server is: In Normal Processing Number of configured agents: 1 Delay Interval (current / min / max): 5 / 5 / 500 Recovery information: State: No recovery being performed
Agents needing recovery: 0
Agents connected: 0 Agents in synchronization: 0 Transition information: Type: Manual
© 2008 Progress Software Corporation34 OPS-24: Success with OpenEdge Replication
DSRUtil target –C monitor (page 1)
Database: C:\wrk101c\repl\target
Database is enabled as OpenEdge Replication: Target
Agent: Name: agent1 ID: 1 Host name: State: Normal Processing Ready: Yes Critical: No Method: Asynchronous Agent is waiting for: Nothing Maximum bytes in TCP/IP message: 8500 Server/Agent connection time: Wed May 14 13:48:43 2008 Delay Interval (current / min / max): 5 / 5 / 500 Transition information: Type: Manual The last block received at: Wed May 14 13:51:14 2008 Activity information: Blocks received: 2084 Blocks processed: 2084
© 2008 Progress Software Corporation35 OPS-24: Success with OpenEdge Replication
DSRUtil target –C monitor (page 2)
Blocks acknowledged: 0 Notes processed: 207774 Transactions started: 10269 Transactions ended: 10269 Synchronization points: 33 AI Block Information: Source RDBMS Block (Seq / Block): 2 / 2516 Last Processed Block (Seq / Block): 2 / 2494 Latency Information: Repl Server behind Source DB by: 1 second(s) Current Source Database Transaction: 13906 Last Transaction Applied to Target: 13793 Target Current as of (Target, Source): Wed May 14 13:51:13
2008, Wed May 14 13:51:13 2008 with delta of 000:00:00
© 2008 Progress Software Corporation36 OPS-24: Success with OpenEdge Replication
DSRUTIL Status
DSRUTIL source/target –C status Return code indicates current state of
replication server or agent. Good for automated scripts Grep for return code Replication User Guide provides details
© 2008 Progress Software Corporation37 OPS-24: Success with OpenEdge Replication
Managing Replication – OE Management
OpenEdge Management 10.1B02/3.1B02+ remotely monitor log files
10.1B02 AdminServer running on the remote system
Attach OE_DB_Replication Log File Ruleset to the log file monitor
Alert/email thrown when message violating rule set appears.
© 2008 Progress Software Corporation38 OPS-24: Success with OpenEdge Replication
OE Management – Replication Log File Rule Set
© 2008 Progress Software Corporation39 OPS-24: Success with OpenEdge Replication
Summary
Terminology and Architecture Replication performance hotspots Replication availability hotspots Source and target management
© 2008 Progress Software Corporation40 OPS-24: Success with OpenEdge Replication
Success With Replication
Understand the architecture Know your requirements Analyze your system Plan, Configure, Monitor Succeed
© 2008 Progress Software Corporation41 OPS-24: Success with OpenEdge Replication Session ID: Session Title
For More Information, go to…
PSDN• www.psdn.com/library/kbcategory.jspa?categoryID=21• www.psdn.com/library/kbcategory.jspa?categoryID=334
Knowledge Centrum (esupport.progress.com)• 3.1B01/10.1B01 upgrade: P122926, P123418, P123420,
P123424, P123426, P123427, P123676• sizing –pica: P121969
Documentation• OpenEdge Replication 10.1C docs (www.psdn.com)• OpenEdge Management 3.1C docs (www.psdn.com)
© 2008 Progress Software Corporation42 OPS-24: Success with OpenEdge Replication
Relevant Exchange Sessions
OPS-21: Managing multiple sites part time: Replication & OpenEdge Management Case Study
© 2008 Progress Software Corporation43 OPS-24: Success with OpenEdge Replication
Questions?
© 2008 Progress Software Corporation44 OPS-24: Success with OpenEdge Replication
Thank You
© 2008 Progress Software Corporation45 OPS-24: Success with OpenEdge Replication