Backup and Recovery Technical

8/3/2019 Backup and Recovery Technical

1/21

The most important responsibility of a database administrator is to prepare for the

possibility of media, hardware and software failure. Should any of these failures occur themajor goal is to make the database available to users within an acceptable time, ensuring

that committed data is undamaged.

BACKUPThe following are the backup methodologies available:

Export Backup (Full):

This type of backup is basically a logical backup. Using this backup, recovery can be done till

the time when it was exported due to User failure by importing the needed objects and dueto Disk Failure by re-creating the database and importing the data in the database.

Considering this backup method as the primary backup method then following points needsto noted:

- Either daily full export backup has to be taken, or only the transaction tables

will have to be exported on a daily basis and static tables weekly basis.

- Recovery time needed for this type of backup would considerably high.

Offline BackupA backup taken when the database is shut down normally is known as offline or a cold

backup. Copying of the datafiles, control file and online redo log files must be done by using

an operating system copy utility. This is a considered as a complete backup of the database.Any changes made after this backup will be unrecoverable if the database is running in

NOARCHIVELOG mode. All transactions are recorded in online redo log files whetherarchiving or not. When redo logs are archived (ARCHIVELOG mode), ORACLE allows you to

apply these transactions after restoring files that were damaged (assuming an active Redo

log file was not among the files damaged).Whenever the schema of the database is changed, such as a new datafile is added or a file

is renamed or a tablespace is created or dropped, shutdown the database and at least makea copy of the control file and the newly added datafile. A complete backup of the database is

preferred.

Online BackupAt sites where a database must operate 24-hours per day and when it is not feasible to take

offline backups, then an alternative is provided by Oracle where physical backups areperformed while the database remains available for both reading and updating. For this kind

of backup the database must be in ARCHIVELOG mode. Only data files and current controlfile need to be backed up. Unlike offline backups, the unit of a online backup is a

tablespace, and any or all tablespaces can backed up whenever needed. Different datafilescan be backed up at different times.

For taking online backup you have to

Put the tablespace in begin backup mode.e.g ALTER TABLESPACE ts_name BEGIN BACKUP

Use o/s commands to copy the datafiles of this tablespace to backup destination.e.g. cp p file_name destination

Alter the status of tablespace to indicate end of backup.e.g ALTER TABLESPACE ts_name END BACKUP

All this must be done while the database is open.


2/21

What happens between BEGIN BACKUP and END BACKUP? : Once the ALTERTABLESPACE ts_name BEGIN BACKUP is issued the status in the datafile header is changed

to indicate that the datafile is being backed up. Oracle stops recording the occurrence ofcheckpoints in the header of the database files. This means that when a database file is

restored, it will have knowledge of the most recent checkpoint that occurred BEFORE thebackup, not any that occurred during the backup. This way, the system will ask for the

appropriate set of redo log files to apply should recovery be needed. Status in the datafileheader is not reset until END BACKUP is issued. On END BACKUP, the system again begins

noting the occurrence of the checkpoints in each file of the database. The checkpoint in thedatafile header is changed during the next log switch after END BACKUP is issued. The

above information will allow the tablespace to be recovered as if the database had beenoffline when the backup took place.

Frequency of online backups: In order to determine how frequently to back up the files

of a database, balance the amount of time available for taking backups and the timeavailable for recovery after media failure. The time for recovery depends on how old your

most recent copy of the damaged file is. The older your backup, the more redo log files

need to be applied, and the longer recovery will take.

RECOVERY

Restoring the files from backup and rolling forward in time is RECOVERY.

There are several kinds of recovery you can perform, depending on the type of failure and

the kind of backup you have. Essentially, if you are not running in archive log mode, thenyou can only recover the cold backup of the database and you will lose any new data and

changes made since that backup was taken. If, however, the database is in Archivelogmode you will be able to restore the database up to the time of failure.

There are three basic types of recovery:

1. Online Block Recovery.This is performed automatically by Oracle.(pmon) Occurs when a process dieswhile changing a buffer. Oracle will reconstruct the buffer using the online

redo logs and writes it to disk.2. Thread Recovery.

This is also performed automatically by Oracle. Occurs when an instance crashes whilehaving the database open. Oracle applies all the redo changes in the thread that occurred

since the last time the thread was checkpointed.3. Media Recovery.

This is required when a data file is restored from backup. The checkpoint count in the datafiles here are not equal to the check point count in the control file. This is also required

when a file was offlined without checkpoint and when using a backup control file.

Media recovery can further be classified under two scenarios :1. Database operating in Noarchivelog mode

2. Database operating in Archivelog mode.

Media Failure and Recovery in Noarchivelog Mode

The only option available in this kind of recovery is to restore from last backup taken. Youwill need to restore all the datafiles , control files , redo log files and the parameter files like

init and config files.


3/21

Media Failure and Recovery in Archivelog Mode

When the database is operating in Archivelog mode the options available for recovery are

many and the recovery depends on what has been lost.The three basic kinds of recovery are:

1. Recover database In this case you use the recover database command. The databasemust be closed and mounted but not open. Oracle will recover all datafiles that are online

after applying the necessary archives / redo logs.2. Recover tablespace In this case you use the recover tablespace command. The

database can be open but the tablespace being recovered must be offline.3. Recover datafile In this case you use the recover datafile command. The database can

be open but the datafile being recovered must be offline.

Note: You must have all archived logs since the time of backup from which you areperforming recovery else you will not have a complete recovery.

Lets understand various types recovery scenarios in brief.

a) Point in Time recovery:A typical scenario is that you dropped a table at say noon, and want to recover it. You willhave to restore the appropriate datafiles and do a point-in-time recovery to a time just

before noon.Note: you will lose any transactions that occurred after noon.

After you have recovered until noon, you must open the database with resetlogs. This isnecessary to reset the log numbers, which will protect the database from having the redo

logs that weren't used be applied.

The four incomplete recovery scenarios all work the same:

Recover database until time '1999-12-01:12:00:00';

Recover database until cancel; (you type in cancel to stop)

Recover database until change n;Recover database until cancel using backup controlfile;

Note: When performing an incomplete recovery, the datafiles must be online.

Do a select name, status from v$datafile to find out if there are any fileswhich are offline. If you were to perform a recovery on a database which has

tablespaces offline, and they had not been taken offline in a normal state, you will losethem when you issue the open resetlogs command. This is because the data file needs

recovery from a point before the resetlogs option was used.

REQUIREMENTS:

The following data structures must be available from the backup:

- All system tablespace datafiles

- All rollback segment datafiles- All datafiles in which the table to recovered is stored

- Archive logs Notes: (a) If the restored datafiles are part of a hot backup, or you wish to roll them

forward, the required archivelogs must be available. In an ideal world the partially restoreddatabase will be made available for recovery on a TEST machine. However this may not

always be possible, and it may be necessary to restore to the same machine on which the

'source' (PROD) database resides. The latter is NOT RECOMMENDED as a user error in therecovery process can corrupt the production database.


4/21

If the datafiles comprising the partially restored database were in hot backup mode at the

time the table was dropped, it is necessary to restore a previous backup. The reason for thisis that the hot backup must be rolled forward past the 'end backup' markers (the time that

the tablespaces were taken out of hot backup mode) before the database can be opened.Failure to recover the restored datafiles past their end backup markers will result in the

following error when an attempt is made to open the database:

ORA-1195 "online backup of file %s needs more recovery to be consistent"

1. Connect as a privileged user to the database and create an ASCII dump of the controlfile.For example:

SVRMGR> connect / as sysdbaSVRMGR> alter database backup controlfile to trace resetlogs;

This statement will generate a trace file in the user_dump_dest destination. Rename thistrace file to ccf.sql and save it.

2. Create a directory on the machine to store the restored database. In this example the

directory is called /RESTORE. The only stipulation is that there is enough space to easilyaccommodate the restored database and any archivelogs required for recovery. If filesystem

space is at a premium, the archivelogs may be optionally restored to /RESTORE as requiredby recovery, and removed after they have been applied.3. Restore the backup to the /RESTORE directory.

4. Optionally, restore all archivelogs required for recovery to the /RESTOREdirectory.

5. Copy the necessary parameter files (init.ora and config.ora etc.) of existing database tothe /RESTORE directory.

6. Rename the parameters files to new sid.7. Edit the init.ora and make the following modifications:

a. control_files = /RESTORE/cntrlTEMP.dbfb. if applicable, change any ifile references to point to the copied include file

c. log_archive% parameters should be changed to reflect the restored archivelogs:

log_archive_destinationlog_archive_formatFor example:

log_archive_destination=/RESTORElog_archive_format=arch_%s.dbf

d. %dump_dest parameters should be changed to point to suitable directoriese. If audit is on, turn it off.

8. Ensure that the shell environment is set correctly. At the very least, the followingenvironment variables must be defined:

a. ORACLE_HOMEb. ORACLE_SID (should be set to new sid)

9. Connect as a privileged user. For example:

SVRMGR> connect / as sysdba

10. Start the instance, specifying new sids init.ora:SVRMGR> startup nomount pfile=/RESTORE/init(newsid).ora

11. The ccf.sql file (created in Step 1) contains the syntax necessary to create a newcontrolfile. Copy this script to the /RESTORE directory and edit it to save only the text

between (and including) 'CREATE CONTROLFILE' and it's terminating semi-colon. Modify thefollowing sections:

a. LOGFILE. Edit this section to reflect the names of the online logs to be created forthe new instance. If the LIVE instance has a large number of large online log


5/21

members, it is advisable to specify a reduced number of smaller log members. YouMUST however specify at least two online log groups.

b. DATAFILE. Edit this section to reflect the names and paths of the restored datafilesonly.

c. Miscellaneous:Remove the REUSE keyword

Optionally change the ARCHIVELOG keyword to NOARCHIVELOG (so thenew database will not operate in archivelog mode)

An example of the completed ccf.sql script might be:

CREATE CONTROLFILE DATABASE "PROD" RESETLOGS NOARCHIVELOG

MAXLOGFILES 16MAXLOGMEMBERS 2

MAXDATAFILES 20MAXINSTANCES 1

MAXLOGHISTORY 337

LOGFILEGROUP 1 '/RESTORE/log1PROD.dbf' SIZE 1M,

GROUP 2 '/RESTORE/log2PROD.dbf' SIZE 1MDATAFILE

'/RESTORE/sys1PROD.dbf',

'/RESTORE/rbs1RBS.dbf','/RESTORE/users1PROD.dbf'

;Then execute ccf.sql script to create control file(s).

12. Recover the database. The database is to be recovered to a time before the table wasdropped. There are two options here:

a. Time-based incomplete recovery:- Database recovery is stopped at the specifieddate and time.

Example of time-based recovery:

SVRMGR> recover database until time '2000-01-10:12:00:00' using backupControlfile;In the above example, apply archivelogs as requested. Recovery will stop

automatically at 12:00 on January 10th, 2000.b. Cancel-based incomplete recovery:- Database recovery is stopped on an archivelog

boundary i.e. the granularity of cancel-based recovery is the archivelog.Example of cancel-based recovery:

SVRMGR> recover database until cancel using backup controlfileAs soon as you have applied the last desired archivelog, type CANCEL

to stop recovery.13. Open the database with the RESETLOGS option:

SVRMGR> alter database open resetlogs;

14. Export the table(s).

15. Import the table(s) into the PROD database.16. Once the necessary tables have been imported, the new instance can be shutdown

and all associated files removed. It is worthwhile verifying that the import has completedsuccessfully before removing the new instance.

b) Recovery without control file:If you have lost the current control file, or the current control file is inconsistent with files

that you need to recover, you need to recover either by using a backup control filecommand or create a new control file. You can also recreate the control file based on the


6/21

current one using the 'backup control file to trace' command which will create a script foryou to run to create a new one. Recover database using backup control file command must

be used when using a control file other that the current. The database must then be openedwith resetlogs option.

c) Recovery of missing datafile with rollback segment:

Recovery in this scenario is very complex and requires special care and attention. This is arecovery scenario in which a datafile in a rollback segment tablespace has been lost ordamaged to a point that Oracle cannot recognize it anymore. Trying to startup the database

will result in ORA-1157, ORA-1110, and possibly an operating system level error such asORA-7360. Trying to shut down the database in normal or immediate mode will result in

ORA-1116, ORA-1110, and possibly an operating system level error such as ORA-7368.WARNING -- PLEASE NOTE : The steps outlined are only to be used with the assistance of

Oracle Worldwide Support. Included (in step 6) is a parameter_corrupted_rollback_segments that will require that you rebuild your database. Please

explore all other options before using this parameter.

The approach to be followed depends on the specific scenario in which the loss of therollback datafile is detected:

I. THE DATABASE IS DOWN :

Attempting to startup the database will result in ORA-1157 and ORA-1110. The solutionhere depends on whether the database was cleanly shut down or not.

I.A. THE DATABASE WAS CLEANLY SHUT DOWN :

If you are ABSOLUTELY POSITIVE that the database was cleanly shutdown, i.e., it wasclosed with either shutdown NORMAL or IMMEDIATE, then the simplest solution is to offline

drop the missing datafile, open the database in restricted mode, and then drop and recreatethe rollback tablespace to which the file belonged. DO NOT follow this procedure if the

database was shut down ABORT or if it crashed.The steps are:

1. Make sure the database was last cleanly shut down.Check the alert.log file for this instance. Go to the bottom of

the file and make sure the last time you shut the database downyou got the messages:

"alter database dismount

Completed: alter database dismount"

This also includes the case of a clean shutdown followed by afailed attempt to startup the database. In that case, Oracle will

issue error messages and shut itself down abort. For the purposes

of this solution, though, this counts as a clean shutdown.

If that is not the case, i.e., if the last time YOU shut the databasedown it was in abort mode, or the database crashed itself, it is

NOT safe to proceed. You should follow the instructions for

case I.B below.2. Remove all the rollback segments in the tablespace to which the lost

datafile belongs from the ROLLBACK_SEGMENTS parameter in the init.orafile for this instance. If you are not sure about which rollbacks are

in that tablespace, simply comment out the whole ROLLBACK_SEGMENTS entry.3. Mount the database in restricted mode.


7/21

STARTUP RESTRICT MOUNT4. Offline drop the lost datafile.

ALTER DATABASE DATAFILE '' OFFLINE DROP;5. Open the database.

ALTER DATABASE OPENIf you receive the message "Statement processed," move on to Step 7.

If instead you get ORA-604, ORA-376, and ORA-1110, go to Step 6.

6. Since opening the database failed, shut the database down and edit the init.ora file forthis instance. Comment out the ROLLBACK_SEGMENTS parameter and add the

following line:_corrupted_rollback_segments = ( ,...., )

i.e., the above list should contain all the rollbacks originally listed in

the ROLLBACK_SEGMENTS parameter.

WARNING: Use this parameter ONLY IN THIS SPECIFIC SCENARIO or as

instructed by Oracle Customer Support.

Then startup the database in restricted mode:

STARTUP RESTRICT

7. Drop the rollback tablespace to which the datafile belonged.

DROP TABLESPACE INCLUDING CONTENTS;8. Recreate the rollback tablespace with all its rollback segments.

Remember to bring the rollbacks online after you create them.9. Make the database available to all users.

ALTER SYSTEM DISABLE RESTRICTED SESSION;

10. Reinclude the rollbacks you just recreated in the ROLLBACK_SEGMENTS parameter in

the init.ora file for this instance. If you had commented out the wholeROLLBACK_SEGMENTS entry, simply uncomment it now. If you had to go through Step6, REMOVE THE _CORRUPTED_ROLLBACK_SEGMENTS PARAMETER NOW.

I.B. THE DATABASE WAS NOT CLEANLY SHUT DOWN

This is the situation where the database was last shut down abort or crashed. In this case, it

is almost certain that the rollback segments that had extents in the lost datafile still contain

active transactions. Therefore, the file cannot be offlined or dropped. You must restore thelost datafile from a backup and apply media recovery to it. If the database is in

NOARCHIVELOG mode, you will only succeed in recovering the datafile if the redo to beapplied is within the range of your online logs.

These are the steps:1. Restore the lost file from a backup.

2. Mount the database.3. Issue the following query:

SELECT FILE#, NAME, STATUS FROM V$DATAFILE;

If the status of the file you just restored is "OFFLINE," you must

online it before proceeding:

ALTER DATABASE DATAFILE '' ONLINE;


8/21

4. Issue the following query:SELECT V1.GROUP#, MEMBER, SEQUENCE#, FIRST_CHANGE#

FROM V$LOG V1, V$LOGFILE V2WHERE V1.GROUP# = V2.GROUP# ;

This will list all your online redolog files and their respective

sequence and first change numbers.5. If the database is in NOARCHIVELOG mode, issue the query:

SELECT FILE#, CHANGE# FROM V$RECOVER_FILE;

If the CHANGE# is GREATER than the minimum FIRST_CHANGE# of your

logs, the datafile can be recovered. Just keep in mind that all thelogs to be applied will be online logs, and move on to step 6.

If the CHANGE# is LESSER than the minimum FIRST_CHANGE# of your logs,

the file cannot be recovered. Your options at this point include

restoring a full backup if one is available or forcing the databaseto open in an inconsistent state to get a full export out of it.

For further details and to assist you in your decision, pleasecontact Oracle Customer Support.

6. Recover the datafile:

RECOVER DATAFILE ''

7. Confirm each of the logs that you are prompted for until youreceive the message "Media recovery complete". If you are prompted for a

non-existing archived log, Oracle probably needs one or more of theonline logs to proceed with the recovery. Compare the sequence number

referenced in the ORA-280 message with the sequence numbers of your onlinelogs. Then enter the full path name of one of the members of the redo group

whose sequence number matches the one you are being asked for. Keep entering

online logs as requested until you receive the message "Media recoverycomplete".8. Open the database.

II. THE DATABASE IS UP :If you have detected the loss of the rollback datafile and the database is still up andrunning, DO NOT SHUT IT DOWN. In most cases, it is simpler to solve this problem with

the database up than with it down.Two approaches are possible in this scenario:

A) The first one involves offlining the lost datafile, restoring it frombackup, and then applying media recovery to it to make it consistent

with the rest of the database. This method can only be used if

the database is in ARCHIVELOG mode.

B) The other approach involves offlining all the rollback segments in the tablespace towhich the lost datafile belongs, dropping the tablespace, and then recreating it. You

may have to kill sessions that have transactions in the rollbacks involved to force therollbacks to go offline.

In general, approach II.A is simpler to apply. It will also be faster if the datafile and thenecessary archived logs can be quickly restored from backup. However, more user

transactions will error out and be rolled back than with approach II.B. Because of read-consistency, queries against certain tables may fail with approach II.A, since the rollback

extents from which Oracle would retrieve the data may be in the offlined datafile.


9/21

II.A: RESTORING THE DATAFILE FROM BACKUP :

As mentioned before, this approach can only be followed if the database isin ARCHIVELOG mode. Here are the steps:

1. Offline the lost datafile.

ALTER DATABASE DATAFILE '' OFFLINE;

NOTE: Depending on the current amount of database activity,

you may have to create additional rollback segments in a differenttablespace to keep the database going while you take care of the

problem.2. Restore the datafile from a backup.

3. Issue the following query:

SELECT V1.GROUP#, MEMBER, SEQUENCE#


This will list all your online redolog files and their respectivesequence numbers.

4. Recover the datafile:

RECOVER DATAFILE ''5. Confirm each of the logs that you are prompted for until you receive the message "Media

recovery complete". If you are prompted for a non-existing archived log, Oracle probablyneeds one or more of the online logs to proceed with the recovery. Compare the sequence

number referenced in the ORA-280 message with the sequence numbers of your online logs.Then enter the full path name of one of the members of the redo group whose sequence

number matches the one you are being asked for. Keep entering online logs as requested

until you receive the message "Media recovery complete".6. Bring the datafile back online.

ALTER DATABASE DATAFILE '' ONLINE;

II.B: RECREATING THE ROLLBACK TABLESPACE :This approach can be used regardless of the archival mode of the database. The steps are:

1. Try to offline all the rollback segments in the tablespace to which

the lost datafile belongs.

ALTER ROLLBACK SEGMENT OFFLINE;

Repeat this statement for all rollbacks in the tablespace.

NOTE: Depending on the current amount of database activity, you may have tocreate additional rollback segments in a different tablespace to keep the database

going while you take care of the problem.2. Check the status of the rollbacks.

They must all be offline before they can be dropped.

Issue the query:


10/21

SELECT SEGMENT_NAME, STATUS FROM DBA_ROLLBACK_SEGS

WHERE TABLESPACE_NAME = '';3. Drop all offlined rollback segments.

For each rollback returned by the query in step 2 with status

"OFFLINE," issue the statement:

DROP ROLLBACK SEGMENT ;4. Handle the rollbacks that remain online.

Repeat the query in step 2.

If any of the rollbacks you tried to offline still has an "ONLINE" status, it means

there are still active transactions in it. You may confirm that by issuing the query:

SELECT SEGMENT_NAME, XACTS ACTIVE_TX, V.STATUS

FROM V$ROLLSTAT V, DBA_ROLLBACK_SEGSWHERE TABLESPACE_NAME = '' AND SEGMENT_ID = USN;

If the above query returns no rows, it means all the rollbacks in the affectedtablespace are already offline. Repeat the query in step 2 to retrieve the names of

the rollbacks that just became offline and then drop them as described in step 3.If the above query returns one or more rows, they should show status "PENDING

OFFLINE". Next, check the ACTIVE_TX column for each rollback. If it has a valueof 0, it implies there are no pending transactions left in the rollback, and it should

go offline shortly. Repeat the query in step 2 a few more times until it shows therollback being offline and then drop it as described in step 3. Move on to step 6.

If any of the "pending offline" rollbacks has a value of 1 or greater in the

ACTIVE_TX column, move on to step 5.

5. Force rollbacks with active transactions to go offline.

At this point, the only way to move forward is to have the "pending offline"

rollbacks released. The active transactions in these rollbacks must either becommitted or rolled back. The following query shows which users have transactions

assigned to which rollbacks:

SELECT S.SID, S.SERIAL#, S.USERNAME, R.NAME "ROLLBACK"FROM V$SESSION S, V$TRANSACTION T, V$ROLLNAME R

WHERE R.NAME IN ('', ... , '')AND S.TADDR = T.ADDR AND T.XIDUSN = R.USN;

You may directly contact the users with transactions in the "pending offline"

rollbacks and ask them to commit (preferably) or rollback immediately. If that isnot feasible, you can force that to happen by killing their sessions. For each of the

entries returned by the above query, issue the statement:

ALTER SYSTEM KILL SESSION ', ';

where and are those returned by the previous query.After the sessions are killed, it may take a few minutes before Oracle finishes

rolling back and doing cleanup work. Go back to step 2 and repeat the query in


11/21

there periodically until all rollbacks in the affected tablespace are offline and readyto be dropped.

6. Drop the rollback tablespace.

DROP TABLESPACE INCLUDING CONTENTS;

If this statement fails, please contact Oracle Customer Support.Otherwise, proceed to step 7.

7. Recreate the rollback tablespace.8. Recreate the rollback segments in the tablespace and bring them online.

d) Recovery of missing datafile without rollback segment:There are three ways to recover in this scenario, as mentioned above.

1. recover database2. recover datafile 'c:\orant\database\usr1orcl.ora'

3. recover tablespace user_data

e) Recovery with missing online redo logs:

Missing online redo logs means that somehow you have lost your redo logs before they hada chance to archived. This means that crash recovery cannot be performed, so media

recovery is required instead. All datafiles will need to be restored and rolled forwarded untilthe last available archived log file is applied. This is thus an incomplete recovery, and as

such, the recover database command is necessary. (i.e. you cannot do a datafile ortablespace recovery). As always, when an incomplete recovery is performed, you must open

the database with resetlogs.Note: the best way to avoid this kind of a loss, is to mirror your online log

files.

f) Recovery with missing archived redo logs:If your archives are missing, the only way to recover the database is to restore from yourlatest backup. You will have lost any uncommitted transactions which were recorded in the

lost archived redo logs. Again, this is why Oracle strongly suggests mirroring your onlineredo logs and duplicating copies of the archives.

g) Recovery with resetlogs option:Reset log option should be the last resort, however, as we have seen from above, it may be

required due to incomplete recoveries. (recovery using a backup control file, or a point in

time recovery). It is imperative that you backup up the database immediately after youhave opened the database with reset logs. The reason is that oracle updates the control file

and resets log numbers, and you will not be able to recover from the old logs. The nextconcern will be if the database crashes after you have opened the database with resetlogs,

but have not had time to backup the database.

How to recover?Shut down the databaseBackup all the datafiles and the control file

Startup mountAlter database open resetlogs

This will work, because you have a copy of a control file after the resetlogs point.

Media failure before a backup after resetlogs.


12/21

If a media failure should occur before a backup was made after you opened the

database using resetlogs, you will most likely lose data. The reason is because restoring alost datafile from a backup prior to the resetlogs will give an error that the file is from a

point in time earlier, and you don't have its backup log anymore.

h) Recovery with corrupted/missing rollback segments:If a rollback segment is missing or corrupted, you will not be able to open the database. Thefirst step is to find out what object is causing the rollback to appear corrupted. If we can

determine that, we can drop that object. If we can't we will need to log an iTar to engagesupport.

So, how do we find out if it's actually a bad object?

1. Make sure that all tablespaces are online and all datafiles are online.

This can be checked through v$datafile, under the status column.For tablespaces associatedwith the datafiles, look in dba_tablespaces. If this doesn't show us anything, i.e., all are

online, then

2. Put the following in the init.ora:event = "10015 trace name context forever, level 10"

This event will generate a trace file that will reveal information about the

transaction Oracle is trying to roll back and most importantly, what objectOracle is trying to apply the undo to.

Stop and start the database.

3. Check in the directory that is specified by the user_dump_dest parameter (in theinit.ora or show parameter command) for a trace file that was generated at startup time.

4. In the trace file, there should be a message similar to:error recovery tx(#,#) object #.

TX(#,#) refers to transaction information.

The object # is the same as the object_id in sys.dba_objects.

5. Use the following query to find out what object Oracle is trying to perform recovery on.

select owner, object_name, object_type, statusfrom dba_objects where object_id = ;

6. Drop the offending object so the undo can be released. An export or relying on a backup

may be necessary to restore the object after the corrupted rollback segment goes away.

7. After dropping the object, put the rollback segment back in the init.ora parameterrollback_segments, remove the event, and shutdown and startup the database.

In most cases, the above steps will resolve the problematic rollback segment.

If this still does not resolve the problem, it may be likely that thefcorruption is in the actual rollback segment. If in fact the rollback segment itself is

corrupted, we should see if we can restore from a backup. However, that isn't alwayspossible, there may not be a recent backup etc. In this case, we have to force the database


13/21

open with the unsupported, hidden parameters, you will need to log an iTar to engagesupport.

Please note, that this is potentially dangerous!

When these are used, transaction tables are not read on opening of the database Becauseof this, the typical safeguards associated with the rollback segment are disabled. Their

status is 'offline' in dba_rollback_segs. Consequently, there is no check for activetransactions before dropping the rollback segment. If you drop a rollback segment which

contains active transactions then you will have logical corruption. Possibly this corruptionwill be in the data dictionary.

If the rollback segment datafile is physically missing, has been offlined dropped, or the

rollback segment header itself is corrupt, there is no way todump the transaction table to check for active transactions. So the only thing to do is get

the database open, export and rebuild. Log an iTar to engage support to help with thisprocess.

If you cannot get the database open, there is no other alternative thanrestoring from a backup.

i) Recovery with System Clock change:You can end up with duplicate timestamps in the datafiles when a system clockchanges. A solution here is to recover the database until time 'yyyy-mm-dd:00:00:00', and

set the time to be later than the when the problem occurred. That way it will roll forwardthrough the records that were actually performed later, but have an earlier time stamp due

to the system clock change. Performing a complete recovery is optimal, as all transactionswill be applied.

j) Recovery with missing System tablespace:The only option is to restore from a backup.

k) Media Recovery of offline tablespace:When a tablespace is offline, you cannot recover datafiles belonging to this tablespace using

recover database command. The reason is because a recover database command will onlyrecover online datafiles. Since the tablespace is offline, it thinks the datafiles are offline as

well, so even if you recover database and roll forward, the datafiles in this tablespace willnot be touched. Instead, you need to perform a recover tablespace command.

Alternatively, you could restored the datafiles from a cold backup, mount the database andselect from the v$datafile view to see if any of the datafiles are offline. If they are, bring

them online, and then you can perform a recover database command.

l) Recovery of Read-Only tablespaces:When a tablespace is altered to read-only the datafiles for this tablespace are marked with a

read-only status in the control file with a stop scn. In addition, the datafile header is alsomarked with the same stop scn. Finally, the data dictionary table TS$ is also updated.

When you go to the backup of the file and try to bring it online, Oracle will compare the stopscn from the datafile header to that of the control file. If they are the same, no recovery is

needed. When using the current control file, recovery is simple. You need to offline theread-only datafiles and start recovery.


14/21

If there is a discrepancy between the two, then some sort of recovery will need to applied.This is why it is ESSENTIAL to make a backup of the datafile(s) once a tablespace is altered

to be read-only. After this time, no other backups need to be taken unless the tablespace isaltered back to read-write.

When using backup control files, if you do not know if the read-only tablespace has toggled

between read-only and read-write, you may want to play it safe by recreating your controlfile with the resetlogs option. This is described in detail as scenario #4 below.

When using a backup controlfile, oracle has no recognition of the stop SCN,

therefore it expects the DBA to tell it when to stop. It will also not be able to do instancerecovery, so at some point it will ask for an archive file you may not have. At this point,

you can submit the online redo log file. If you select the wrong one, you may see an errorlike:

Applying logfile...

ORA-00310: archived log contains sequence 35; sequence 36 required

Try again until you get the right sequence number.

The following are some scenarios of read-only tablespace recovery:

1. You have:backup of read-only tablespace file when it is read-only.

current control file showing tablespace is read-only

To recover you must:take read-only tablespace offline

restore the datafile backupalter the tablespace online

In this scenario, no recovery is needed for the tablespace regardless of how old the backupis. No redo is needed to bring the datafile(s) to current status since it was read-only and nochanges could have taken place in it. If you try to issue recovery, you will receive an error

indicating that no recovery is needed.

SQLDBA> recover tablespace readonly;ORA-00283: Recovery session canceled due to errors

ORA-00264: no recovery required

2. You have:backup of read-only tablespace when it was read-only.

current control file shows tablespace as read-write.

tablespace is currently read-write.

To recover you must:

take the tablespace offlineissue the recover tablespace command

Oracle will prompt you for all archive files since the tablespace was made writeable to the

current log. It will apply information from the online logs, so it is possible that it will notask for all archives generated.


15/21

3. You have:backup of read-only tablespace when it was read-write.

current control file shows tablespace as read-onlytablespace is currently read-only

To recover you must:

take the tablespace offlineissue the recover tablespace command

Oracle will ask for archive files from datafile backup to when it was altered to read-only. If

during the time between the backup and the time of recovery, the tablespace was maderead-only, then oracle will also ask for archives when the file was read-only. For example,

lets say that the datafile backup at 11 (read-write), and the tablespace was read-only from16-19, then read-write from 20-24, finally read-only again at 24. Recovery needs 11-24.

The point is that although the tablespace was read-only from 16-19, those archives are notskipped, they will still be required for recovery.

4. You have:backup of control file showing the tablespace read-write (20)

the datafile backup is when tablespace was read-only(16)The tablespace is currently read-only(24)archives up to 27 --

to recover:

recover database using backup controlfile

It will request the first archive file when the tablespace was read-write and continue throughthe current redo log file. To open you must use resetlogs option, 'alter database open

resetlogs'. When the resetlogs option is used, this negates all previous backups. Thesequence number is reset to 1. If you must use this option, it is essential to take a full

backup of the database.

5. You have:backup of control file showing the tablespace read-only

the datafile backup is when the tablespace was read-writeThe tablespace is currently read-only

You will not be able to recover using this control file, if you try to issue recovery, you will

get an error like:

SQLDBA> recover database using backup controlfileORA-00283: Recovery session canceled due to errors

ORA-01233: file 3 is read only - cannot recover using backup controlfile

ORA-01110: data file 3: '/u06/bugmnt/tar9212155.4_2/reonly.dbf'

To recover you must recreate the control file:

alter database backup controlfile to trace;

A trace file will be generated with the create controlfile statement. Delete all other

information from this trace file. It is essential to add the read-only datafile with the others.It will be listed further down from the create statement indicating the file is MISSING.

Finally, be sure to create


16/21

the control file with 'resetlogs' option.

If don't specify resetlogs with the create control file you may see:

SQLDBA> @controlORA-01503: CREATE CONTROLFILE failed

ORA-01229: data file 2 is inconsistent with logsORA-01110: data file 2: '/u06/bugmnt/tar9212155.4_2/rbsT716.dbf'

Once the control file has been recreated, to recover:

Issue recover database using backup control file

Oracle will ask for archive for the read-only datafile when it was read-write to the current

log. Once recovery has completed, issue 'alter database open resetlogs'. When theresetlogs option is used, this negates all previous backups. The sequence number is reset

to 1. If you must use this option, it is essential to take a full backup of the database.

It is important to note that the initial check of the control file for read-only tablespaces will

effect all recovery, not just if you are recovering the read-only tablespace. For example, ifyou are recovering a different tablespace with the backup control file, the recovery will notbe able to proceed because of the read-only tablespace. If your intention is to recover a

different tablespace, you have two options.

- If you know that the read-only tablespace is consistent with the control file, you cansimply take the datafile offline before proceeding with the recovery. You can use the

'alter database datafile 'xxx' offline'. After the database is restarted, then you needto alter the read-only tablespace online to allow access of it's data.

- You can recreate the control file as described above.

J) Recovering from a lost datafile in user tablespace:This is a recovery scenario in which a datafile in a user tablespace has been lost or damagedto a point that Oracle cannot recognize it anymore. Trying to startup the database will result

in ORA-1157, ORA-1110, and possibly an operating system level error such as ORA-7360.Trying to shut down the database in normal or immediate mode will result in ORA-1116,

ORA-1110, and possibly an operating system level error such as ORA-7368.

Two approaches are possible:I. Recreate the tablespace to which the datafile belongs

II. Restore the file from a backup and do media recovery on it

I. THE USER TABLESPACE CAN BE EASILY RECREATED

This means that reasonably recent exports of the objects in the tablespace are available, or

that the tables in the tablespace can be repopulated by running a script or program,loading the data through SQL*Loader, etc. In this case, the simplest solution is to offline

drop the datafile, drop the tablespace, recreate it, and then recreate all the objects in it.The steps involved are:

1. If the database is down, mount it.

2. Offline drop the datafile.

ALTER DATABASE DATAFILE '' OFFLINE DROP;


17/21

3. If the database is at mount point, open it.4. Drop the user tablespace.

DROP TABLESPACE INCLUDING CONTENTS;

5. Recreate the tablespace.6. Recreate all the previously existing objects in the tablespace.

II. THE USER TABLESPACE CANNOT BE EASILY RECREATED

In many situations, recreating the user tablespace is impossible or too laborious. The

solution then is to restore the lost datafile from a backup and do media recovery on it. Ifthe database is in NOARCHIVELOG mode, you will only succeed in recovering the datafile if

the redo to be applied to it is within the range of your online logs.

These are the steps:

1. Restore the lost file from a backup.2. If the database is down, mount it.

3. Issue the following query:

SELECT V1.GROUP#, MEMBER, SEQUENCE#, FIRST_CHANGE#


This will list all your online redolog files and their respectivesequence and first change numbers.

4. If the database is in NOARCHIVELOG mode, issue the query:

SELECT FILE#, CHANGE# FROM V$RECOVER_FILE;

If the CHANGE# is GREATER than the minimum FIRST_CHANGE# of yourlogs, the datafile can be recovered. Just keep in mind that all the

logs to be applied will be online logs, and move on to step 5.

If the CHANGE# is LESSER than the minimum FIRST_CHANGE# of your logs,the file cannot be recovered. Your options at this point would be to

restore the most recent full backup (and thus lose all changes

to the database since) OR recreate the tablespace as explained in scenario I.5. Recover the datafile:

RECOVER DATAFILE ''6. Confirm each of the logs that you are prompted for until you receive the message "Media

recovery complete". If you are prompted for a non-existing archived log, Oracle probablyneeds one or more of the online logs to proceed with the recovery. Compare the sequence

number referenced in the ORA-280 message with the sequence numbers of your online logs.Then enter the full path name of one of the members of the redo group whose sequence

number matches the one you are being asked for. Keep entering online logs as requesteduntil you receive the message "Media recovery complete".

7. If the database is at mount point, open it..

K) HOW TO RECOVER A DATABASE HAVING ADDED A DATAFILE SINCE THE LASTBACKUP:

A. Current controlfile, backup of datafile exists (Oracle release 7.x)


18/21

A valid (either hot or cold) backup of the datafiles exists, except for the datafile createdsince the backup was taken. The current controlfile exists. The database is in archivelog

mode (see note (c) at bottom of page).1. Restore ONLY the datafiles (those that have been lost or damaged) from

the last hot or cold backup. The current online redo logs and control file(s) must beintact.

2. Mount the database3. Create a new datafile using the 'ALTER DATABASE CREATE DATAFILE' command.

a. The datafile can be created with the same name as the originalfile. For example,

SQLDBA> alter database create datafile

2> '/dev1/oracle/dbs/testtbs.dbf';Statement processed.

b. The datafile can be created with a different filename to the original.

This option might be chosen if the original file was lost due to disk

failure and the failed disk was still unavailable; the new file wouldthen be created on a different device. For example,

SQLDBA> alter database create datafile2> '/dev1/oracle/dbs/testtbs.dbf'

3> as4> '/dev2/oracle/dbs/testtbs.dbf';

Statement processed.

The above command creates a new datafile on the dev2 device. The fileis created using information, stored in the control file, from the

original file. The command implicitly renames the filename in thecontrol file.

NOTE: IT IS VERY IMPORTANT TO SPECIFY THE CORRECT FILENAME WHENRECREATING THE LOST DATAFILE. IF YOU SPECIFY AN EXISTINGORACLE DATAFILE, THAT DATAFILE WILL BE INITIALISED AND WILL

ITSELF REQUIRE RECOVERY.4. Recover the database.

SQLDBA> recover database

ORA-00279: Change 6677 generated at 06/03/97 15:20:24 needed for thread 1ORA-00289: Suggestion : /dev1/oracle/dbs/arch/arch000074.arc

ORA-00280: Change 6677 for thread 1 is in sequence #74Specify log: {=suggested | filename | AUTO | CANCEL}

At this point the recovery procedure will wait for the user to supply the

information requested regarding the name and location of the archived logfiles. For example, entering AUTO directs Oracle to apply the suggested

redo log and any others that it requires to recover the datafiles.

Applying suggested logfile...Log applied.

::


19/21

::

Media recovery complete.5. Open the database

SQLDBA> alter database open;

Statement processed.

B. Old controlfile, no backup of datafile (Oracle release 7.3.x)A valid (either hot or cold) backup of the datafiles exists, except for the datafile created

since the backup was taken. The controlfile is a backup from before the creation of the newdatafile. The database is in archivelog mode (see note (c) at bottom of page).

1. Restore the datafiles (those that have been lost or damaged) from the last hot or coldbackup. Also restore the old copy of the controlfile. The current online redo logs must be

intact.2. Mount the database

3. Start media recovery, specifying backup controlfile

SVRMGR> recover database using backup controlfile

ORA-00279: Change 6677 generated at 06/03/97 15:20:24 needed for thread 1ORA-00289: Suggestion : /dev1/oracle/dbs/arch/arch000074.arcORA-00280: Change 6677 for thread 1 is in sequence #74

Specify log: {=suggested | filename | AUTO | CANCEL}

At this point, apply the archived logs as requested. Eventually Oraclewill encounter redo to be applied to the non-existent datafile. The

recovery session will exit with the following message, and will returnthe user to the Server Manager prompt:

ORA-00283: Recovery session canceled due to errors

ORA-01244: unnamed datafile(s) added to controlfile by media recovery

ORA-01110: data file 5: '/dev1/oracle/dbs/testtbs.dbf'4. Recreate the missing datafile. To do this, select the relevant filenamefrom v$datafile:

SVRMGR> select name from v$datafile where file#=5;

NAME-------------------------------------------------------

UNNAMED0005

Now recreate the file:

SVRMGR> alter database create datafile

2> 'UNNAMED0005'

3> as4> '/dev1/oracle/dbs/testtbs.dbf';

5. Restart recovery

SVRMGR> recover database using backup controlfileORA-00279: Change 6747 generated at 09/24/97 16:57:18 needed for thread 1

ORA-00289: Suggestion : /dev1/oracle/dbs/arch/arch000079.arcORA-00280: Change 6747 for thread 1 is in sequence #79



20/21

Apply archived logs as requested. Prior to Oracle8, recovery must apply

the complete log which was current at the time of the datafile creation(in the above example, this would be log sequence 79). A recovery to a

point in time before the end of this log would result in errors:

ORA-01196: file 1 is inconsistent due to a failed media recovery sessionORA-01110: data file 1: '/dev1/oracle/dbs/systbs.dbf'

If this happens, re-recover the database and ensure that the complete log

is applied (plus any further redo if required). This limitation doesnot exist from Oracle 8.0+.

Eventually, Oracle will request the archived log corresponding to the

current online log. It does this because the (backup) controlfile has noknowledge of the current log sequence. If an attempt is made to apply the

suggested log, the recovery session will exit with the following message:

ORA-00308: cannot open archived log '/dev1/oracle/dbs/arch/arch000084.arc'

ORA-07360: sfifi: stat error, unable to obtain information about file.SVR4 Error: 2: No such file or directory

At this stage, simply restart the recovery session and apply the currentonline log. The best way to do this is to try applying the online redo

logs one by one until Oracle completes media recovery:

SVRMGR> recover database using backup controlfileORA-00279: Change 6763 generated at 09/24/97 16:57:59 needed for thread 1

ORA-00289: Suggestion : /dev1/oracle/dbs/arch/arch000084.arcORA-00280: Change 6763 for thread 1 is in sequence #84


/dev1/oracle/dbs/log2.dbfLog applied.Media recovery complete.

6. Open the database

SVRMGR> alter database open resetlogs;

The resetlogs option must be chosen to resynchronize the controlfile.

NOTES:a) These techniques can be used whether the database was closed either

cleanly or uncleanly (aborted).

b) If the database is recovered using an incomplete recovery technique (either time-based,

cancel-based, or change-based), and is recovered to a point in time before the datafile wasoriginally created, any references to that datafile will be removed from the database when

the database is opened.

Oracle handles this situation as follows:

- The 'alter database create datafile....' command creates a reference inthe controlfile for the datafile.

- Incomplete recovery terminates before applying redo that would create a


21/21

corresponding row for the datafile in the file$ dictionary table.- When the database is opened, Oracle detects an inconsistency between file$ and the

controlfile and resolves in favour of file$, deleting the entry from the controlfile.

c) It may be possible to recover the datafile using this technique even if the database is notin archivelog mode. However, this relies on the required redo being available in the online

redo logs.

Documents

Backup and Recovery Technical