B1: After Imaging New features. Greater flexibility. Richard Banville Fellow

B1: After Imaging

New features. Greater flexibility.

Richard BanvilleFellow

© 2008 Progress Software Corporation2

Agenda

Basics New features Online implementation Archival and Maintenance


Agenda

Basics• What is after imaging?

• Why use after imaging?

• How does it work?

• What is the AI archiver?


What is after imaging?

A journal of transaction data that can replay changes to a database

Referred to as a redo log Granularity of recovery

• To last completed transaction

• To a point in time

• To a specific transaction number.

Space is not reused without interaction• (as it is in the before image journal files)


Why use after imaging?

Protection• Removes single point of failure

• Media loss

• Human “error”– Table drop– File deletion

No data loss High availability & Business Continuity For replication

• Manual or OpenEdge Replication


How does it work?

Transaction data synchronously written to an after image (AI) journal

AI journal “replayed” in recovery scenarios Not automatically enabled Automation

• Some automatic maintenance features

• Recovery not automated


How the AI File Works

FOR EACH customer:

UPDATE customer.

END.

BI Note Written

AI Note Written

Transaction Begin

BI and AI Transaction End Notes Written

*** Data updated “sometime” later


Forward Processing - Physical View

DatabaseBefore Image

Shared Memory

After Image

After Image

After Image

12

3 + t

Full AI files must be managed


Forward Processing - Physical View

DatabaseBefore Image

Shared Memory

After Image

After Image

After Image

12

3 + t


The Recovery Process

Database Before Image

After Image

After Image

After Image

BackupRemote copy

1a1b

3 thru nroll forward

2Redo


Agenda

Online AI functionality Automatic AI file management

• AI archiver

New Features


New after imaging features (10.1A thru 10.1C)

Online AI functionality• Add extents

• Start/Stop (enable/disable)

• Automatic archiving– Archive extents– Change archive directory– Change archive interval

• Manage AI buffers

Guarded recovery process

Online features are a huge advance in AI’s flexibility


What is the AI archiver?

Daemon (AIMD) to manage/move “full” AI files• Timer mode (every <n> seconds)• On-demand mode (when full)• Archives & marks AI extents for reuse

Enablement• Offline via rfutil• Online via the probkup utility

Additional online features• Switch archive locations• Change archive interval

No support for archiving to a second host No support for single user activity

Introduced in 10.1A


Empty

AI Archiver

Shared Memory

DB BI A1

AI ArchiveDaemon

Archive 1

Archive 2

Archive 3

Knows when andwhere to archive

A2

Busy

Busy

FullEmpty


Agenda

Creating AI files Enabling AI Starting and managing the AI archiver

Implementing After Imaging Online


Creating AI files

New Syntax

Notes:• Ensure structure file correct before adding

• Defaults to 8K AI block size• Ensure default user permissions are correct

prior to adding extents

prostrct addonline <db> <st-file-name>

prostrct addonline <db> <st-file-name> -validate

Adding extents online


Creating AI files

Q: How many AI files to create?• A: More than one

• A2: Everyone has a different opinion

• 10 is a good starting point

Extent type: variable, fixed or vixed?• Depends on archive mechanism

• Variable – “easier” to maintain

• Fixed for “ultimate” performance– Makes some sense with fixed amount of work


Need to have:• AI files must reside on a different disk/LUN

than any other aspect of the database (db, dn, bn)

Nice to have:• AI archived to yet another disk/physical locale

– not always possible/plausible

Avoid I/O bottlenecks• Add more spindles if need be

Location, location, location

After Image File and Archive Locations


Enabling After Imaging Online

New option to the probkup utility

Notes:• Make sure your AI files are in place

• Common uses:– Oops, “forgot” to enable after imaging– AI off Recoverable maintenance AI on

probkup online <db> <backup-file> enableai


Managing AI buffers online (10.1C)

New option

Notes:• For performance, set aibufs >= bi buf pool size

– Online default same as -bibufs (8K AI buffers)

• Permissions for new shared memory

proutil <db> increase -aibufs <n>


Offline:

• AI must already be enabled ;(

Online:

• AI can be enabled in conjunction ;)

Enabling the AI archiver

rfutil <db> -C aiarchiver enable

probkup online <db> <backup> [ enableai ] enableaiarchiver -aiarcdir <dir> -aiarcinterval <n-seconds> [ -aiarcdircreate ]


New Syntax:

Can optionally create directories on the fly

Again, watch permissions

Changing the Archive Directory Online

rfutil <db> -C aiarchiver setarcdir <dir-list>

rfutil <db> -C aiarchiver setarcdir <dir-list>/ -aiarcdircreate


Changing Archive Interval Online

New Syntax:

Valid range 120 to 86,400 (in seconds)• 2 minutes to 1 day

• Suggest 15 to 60 minutes based on activity

Common uses• Correct mistake of minutes vs seconds

• Change in business constraints (lost data)

Archiver awakes every 5 seconds

rfutil <db> -C aiarchiver setinterval <#>

Forcing an extent switch automatically


Changing Archive Mode Online

On-demand Mode• No interval or interval 0

• Use with fixed length extents or manual switch

Timed Mode• Interval set 120 to 86,400 (in seconds)

• Archives regardless of activity

Notes:• Change time and/or on-demand during off hours

• Timed must be set at each invocation

• Archiver still awakes every 5 seconds

rfutil <db> -C aiarchiver setinterval <#>

On-demand vs timed archival


Agenda

Monitoring your AI files Archive log naming Backing up AI archive files Cleaning up AI archive files

Archiving and Maintenance


Manual AI Archival

Manually archive AI files while db is online Check for full extents Force extent switch Find next full extent

to archive Rinse, lather, repeat Example code: http://

wss.com/products/downloads.html

No archiver? No problem.

rfutil <db> -C aimage new

rfutil <db> -C aimage full

rfutil <db> -C aimage list


Not user definable (it should be) Naming convention

Recovery easily managed• Files relate to a particular backup

• Date and time

• Extent sequence number in the name

Directory separator “/” replaced with “~” Example:

Archive Log Naming

<db> . date . time . backup-seq . ai-seq . ext-name

usr1~richb~x.20080806.151638.00003.00000001.x.a1


Backup of Archives

Increase types of recoverable disasters How often

• Throughout the day

Where to• Separate media for media failure protection

• Secondary machine as quickly as possible

• Best if archival system is “very” far away

NOT included in the OpenEdge database backup


Cleaning Up Archives

Eventual disk space issue Retention dependant on business needs Clean up

• Create an automated process to clean up archives after a period of time (days)

• Keep at least 7 days of archives online• How often do you backup?

– Archives without a backup is useless• Where are they stored?• Maintaining a hot standby?

Archiver retains archives forever!


Cleaning Up Archives - Unix/Linux

The find command is very useful for this task

# Archive Cleaner# Should be run from cron daily# Test yourself with echo before implementingARCDIR=<your-archive-directory>KEEPDAYS=+30

find $ARCDIR -mtime $KEEPDAYS -exec \rm {} \ 2>&1 > /dev/null

• Beware of changes in $ARCDIR

• Consider e-mail rather than /dev/null

• Incorporate remote copy

• Add automatic validation


Cleaning Up Archives - Windows

Uses file from the free windows resource kit File name: %WINDOWS%\system32\forfiles.exe

REM ** Archive cleanup batch fileREM ** Have this run as a scheduled taskREM ** Test yourself with echo before implementingSet ARCDIR=your-archive-directorySet KEEP=-30

forfiles /P %ARCDIR% /m *.a* /D %KEEP% /C “cmd /C del /q @FILE”

REM OR

forfiles -p%ARCDIR% -m*.a* -d%KEEP% -c“cmd /c del /q @FILE"

//technet.microsoft.com/en-us/library/cc755872.aspx


Disaster Scenarios

Floods Fire

Bad stuff happens


Using auto AI archives to recover

In the AI archiver log “<db>.archival.log”• Find most recent backup to restore

– Look for “BACKUP_SET”

• Identify AI extent to start roll forward with– Match “aiseq” value to archived AI extent

Ignore AI extent’s backup sequence value

– Roll forward it and all later archives

Log file layout documented in DB Admin Guide Example: …

It is quite easy


AI Example

# OpenEdge RDBMS After-image Extent Manager Archival Log File

# Do not edit this file with an editor as it is automatically

# maintained by the OpenEdge RDBMS after-image extent manager.

0032,/usr1/db/x,20080807,100553,1,2,20080807,100551,1,1,x.bak,BACKUP_SET_20080807_100551

#

# After-image Extent Manager started on : Thursday August 07, 2008 at 10:05:56

#

0255,20080807,100556,100100

0001,/usr1/db/x,20080807,100853,20080807,100552,2,1,/usr1/db/x.a1,/usr2/arcdir1,/usr2arcdir1/usr1~db~x.20080807.100552.00002.00000001.x.a1

0001,/usr1/dbx,20080807,100908,20080807,100552,2,2,/usr1/db/x.a2,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00002.00000002.x.a2

0032,/usr1/db/x,20080807,100947,1,3,20080807,100946,4,1,../x.bak2,BACKUP_SET_20080807_100946

0001,/usr1/db/x,20080807,100949,20080807,100552,3,3,/usr1/db/x.a3,/usr2/arcdir1,/usr2/arcdir1/usr1~db~x.20080807.100552.00003.00000003.x.a3







A Eye Exam(ple)





#


#

0255,20080807,100556,100100











A Eye Exam(ple)





#


#

0255,20080807,100556,100100










Backup file-name, BACKUP_SET

AI Seq #4

AI Seq #4

AI file-name to start rollforward

AI Seq #4


Using auto AI archives to recover

Documentation issue: buseq aiseq Wrong AI file? No problem.

• Error message reported

• Retry with the correct one

Roll forward last active/not archived AI extent Always use AI sequence value

• Archive location within .log file is “relatively” meaningless w/respect to its BACKUP_SET

• WHY? …

Some notes on the AI archive log


Why AI Log is NOT Sequential

A1A1

DB BI

AI ArchiveDaemon

Archive 1Archive 2

Archive 3A2A2Users

Backup

DB.bakDB.bak

Archive Log

Extent S

witch

…with respect to the BACKUP_SET

No sync on

write order

BusyFullEmpty

EmptyBusy


Recovery Process Overview

Get everyone off the system (crons too!) Backup what you have (if anything)

• Database (even just parts)

• Recovery files (AI, BI,TL)

• AI Archives

Delete broken database Restore most recent backup Apply after image archives

rfutil <db> -C roll forward oplock –a <aifile> (10.1c)


Extending Archival Usefulness

Log-based replication


Log-Based Replication

Hot standby database• Good to use for backups

– Eliminates backup I/O on production– Use -norecover option with probkup

• Less recovery downtime

• Cannot be updated (subsequent rollf. will fail)

Hot standby copy (#2)• Provide for reporting (semi-stale)

• “Midnight” copy of hot standby #1

Extending archival usefulness


Log-Based Replication

Auto archived files• Deal w/awkward naming convention

• Create sub-directory under archive directory

Process looks for new archives• Applies new archives to hot-standby database

• Moves applied archives

Best if on a remote host• Less contention w/production

• Improved disaster recovery strategy

Extending archival usefulness


Summary

All OpenEdge systems should use AI Many new online AI capabilities AI management tool provided Use archives to maintain a hot-standby

• Improved high availability


Questions?


Outstanding Questions Answered

Does ai archiver use ai extract or simply copy?• It uses Ai Extract.

Is ai archiver available for Workgroup licenses?• Yes. It is functionality and not performance related.

You can specify several archive directories. Is there a way to tell which is the current archive directory?

• There is a message in the log when a dir switch is performed. There is no way to programmatically determine this. I will add an enhancement request.

When ai is locked due to replication, is the file archived and later marked empty or does the archive process stall waiting for the locked extent to become unlocked?

• The extent is archived. Once it is “unlocked” it is emptied by the watch dog. The aimgt deamon also attempts to empty locked extents.

Is there any documentation on using ai archiver with replication?• Yes, there is a callout in the ai mgt doc when using it with replication. The doc

explains what happens to an extent with ai mgt and repl running. Is it safe to change permissions of a file after add online completes?

• No. Although add online has completed, all users connected to the database have not opened the new file. They open it the next time they access it. Changing its permissions while the database is still running may cause the database to shutdown when a user attempts to open a file it does not have permission to.


Thank You


Documents

B1: After Imaging New features. Greater flexibility. Richard Banville Fellow