View
222
Download
3
Tags:
Embed Size (px)
Citation preview
How to do a Painless QA DB Refresh
Session 911A
Who we are…..
• Tracy Ferrell – Mindworks Consulting. Senior Basis Administrator assigned to ChevronTexaco for 2.5 years. Originally with IBM and then Accenture- about 11 years with SAP R/3 basis
• Rich Bernat – ChevronTexaco. Senior Basis Administrator. Crack DBA and RMAN expert.
What we do…..
ChevronTexaco Basis Team
• Manage 6 - SAP Production Systems (R/3 4.6B and 4.6C, BW 2.1)• 100GB – 700GB Database sizes• HP-UX, Oracle 8.1.7.4• FI/CO, SD, Purchasing, HR, BW, IS-Oil, MM/PP and PS
Why do we care about DB Refreshes?
• Customers expect us to deliver environments in a narrow time window and they have to be functional .• ChevronTexaco has a fairly strict regression test requirement for all production changes
Why do we care about DB Refreshes?
• All QA environments are refreshed monthly • Project environments are refreshed on demand• Projects create, copy, refresh, restore many times for different project phases• All QA refreshes occur on the same day …6+Tb of data moved over a weekend!!!
What will we talk about ???
DB refresh strategies Assumptions that have been made What can be “saved” from the old QA
database What are the various restore strategies
that can be used What are the tools that can be used What can be automated and how to get
to that point
Data Refresh Strategies
Client Export/Import – works well with small clients
SAP DB Migration tool Automated Tools (Loadrunner, Autotester,
CATT, etc.) – requires maintenance to scripts when changes are made
ALE – works well for master data not so well for transaction data
Full database restore – requires pre and post work, but is complete and is the strategy used by this presentation
Database Restore Strategies
Full DB Restore Strategy is further defined by the backup/restore tools that are being utilized
Full off-line backupsThe easiest method, however requires production downtime (brbackup, rcp, Legato, etc)
On-line backupsNot as straight forward – but can be managed
Third mirror systems
Database Restore Strategies (cont.)
Full DB Restore Strategy is further defined by the backup/restore tools that are being utilized
Oracle RMANThe easiest to manage, however, requires extensive RMAN knowledge and will not be covered in this presentation – catch Rich Bernat’s presentation handouts - 910A – “Reducing Costs of Environment Copies Using Oracle - Recovery Manager” which were presented on Tuesday at 2pm
This QA DB Refresh Strategy
This presentation will cover the strategy of using an off-line backup of the source database.
Slight changes can be made to this process to use an on-line backup
ChevronTexaco had been using this strategy for about 7 years.
Assumptions
QA database (disk) is sized sufficiently to take a full copy of the production/source DB
The existing QA database is up and running. Oracle is installed and at the same
version as production SAP executables exist and are
functioning QA database is not co-located with
production database
Assumptions (cont.)
If refreshing the development database, much more work is required to “preserve” versioning information
User has root authority oraNEW, oraOLD, NEWadm and OLDadm
accounts all have access to target database server
These hints and tips work well for SAP versions 4.0- 4.6 and Oracle 8.1.x
Where are we going????
Gotta have a plan !!
Prep Work DB copy Post WorkPost DB
copywork
Database Refresh Flow
General flow of refresh
Preparation of target server Export of tables that can be preserved Shutdown of target R/3 system Removal of all target system Oracle
datafiles Restore of source datafiles to target server Rename of source database to target
database Import of items that were preserved Clean-up
Where are we??
Gotta have a plan !!
Prep Work DB copy Post WorkPost DB
copy work
Preparation for Refresh
Export items that can be preserved and imported back into the QA database to save manual post configuration steps
SM59 tables RZ12/SMLG tables RZ04 tables User masters and activity groups R/3 Profiles (RZ10) Transport status tables (next transport
number) Other custom tables (ChevronTexaco pulls
custom table that control interfaces)
Preparation for Refresh
Things that should be done before DB restore
Validate that the size of the sapdatas on the target server can accept the sapdatas from the source system
Create control file from source database Modify control file for later use Ensure that an initOLD.ora file exits on
target server Create a new initNEW.ora file or leave the
original one from the target database if sufficient (beware of rollback segments)
Preparation for Refresh (cont.)
Shutdown the target DB Remove all sapdatas, archive files, online
redo log files from the target database Create a link from the “OLD” SID to the
“NEW” SID (for restore purposes and to bring up the “OLD” database to run the create control file script – a key point!)
Where are we??
Gotta have a plan !!
Prep WorkDB
restorePost Work
Post DB copywork
Database Restore
Completely quiesce and shutdown source database
shutdown startup restrict (to ensure all activity is gone) switch log files – with the current option shutdown
Backup and then restore all the datafiles from the source database to the target database with the tool of choice – brrestore, Omniback, ADSM, rcp, etc.
Remember: This method assumes an offline backup of the source database
Where are we??
Gotta have a plan !!
Prep Work DB copy Post WorkPost DB
copywork
Database Rename Work
This is the toughest step!!!
• Startup “OLD” database on new server• Execute create control file from sql• Rename datafiles• Rename logfiles• Execute other SQL based changes – ChevronTexaco at this point manages the printers (server assign and disable), batch jobs, transport system • Backup new control file• Shutdown “New” instance
Database Rename Work (How?)
Startup the “OLD” database on the new server in mount mode
SQL> connect as sysdba; SQL> startup nomount; SQL> @create_control_file.sql SQL> alter database open resetlogs;
Should have new database name at this point
Note: This NEW database has the same DBID as the OLD database.
Altered control file SQL script
Database Rename Work
Shutdown – just for fun and testing Restart to make sure it really worked Backup control file of the NEW database Create new OPS$sidadm account Reset oracle system, sys, sapr3 passwords
if required. Execute other SQL scripts as required
Where are we??
Gotta have a plan !!
Prep Work DB copy Post WorkPost DB
copywork
Post Refresh Work
Startup R/3 Import security
tp import SIDK9nnnnn SID u1 Sync user buffers (from se38 RSUSR405)
Import items that can were preserved SM59 tables RZ12/SMLG tables RZ04 tables R/3 Profiles (RZ10)Transport status tables (next transport
number) Other custom tables
Post Refresh Work (cont)
Remove all batch jobs except housekeeping Fix print server assignment Clean up
New Saplicense with saplicense –install –p <ifile>
Userid access to Unix Permissions on oracle files Remove extraneous Oracle ID’s
(OPS$OLDadm, etc.) ALE re-setup TMS configuration for NEW SID (run SE06
with DB copy option)
Where are we??
DONE!!!!!!!!!
Prep Work DB copy Post WorkPost DB
copywork
Automation Opportunities
ChevronTexaco has automated the entire process
But how can you????
I can’t give you all our scripts, but here are some tips
1. Start small and work you way up2. Start by creating scripts without error
recovery and little automation, i.e., query the person executing the script for the old SID, new SID, new print server name, etc.
Automation Opportunities
4. Modularize, Modularize, modularize – did I say modularize?
5. Group SQL commands in a .sql script that are generic per SID or can be easily modified with a vi command
6. Make a checklist and follow it religiously for each refresh!!!!!
Modified Control file (SID refers to the target)
Hints….
CREATE CONTROLFILE REUSE REUSE DATABASE “NEWSID" RESETLOGS ARCHIVELOG MAXLOGFILES 255 MAXLOGMEMBERS 3 MAXDATAFILES 1000 MAXINSTANCES 50 MAXLOGHISTORY 9302LOGFILE GROUP 1 ( '/oracle/SID/sapdata1/log_1/loga.dbf', '/oracle/SID/sapdata3/log_1/logb.dbf' ) SIZE 20M, GROUP 2 ( '/oracle/SID/sapdata2/log_2/loga.dbf', '/oracle/SID/sapdata4/log_2/logb.dbf' ) SIZE 20M,DATAFILE '/oracle/SID/sapdata6/system_1/system.data1', '/oracle/SID/sapdata21/roll_1/roll.data1', '/oracle/SID/sapdata24/temp_1/temp.data1', '/….etc '/….etc '/….etc '/oracle/SID/sapdata190/btabd_20/btabd.data20', '/oracle/SID/sapdata190/zza01ai_29/zza01ai.data29', '/oracle/SID/sapdata191/glpcad_31/glpcad.data31'; Key semi-colon
Hints…… (cont.)
Table exports from R/3:
We use R3trans on most of our table exports:
1. Create an R3trans control file2. As the sidadm, execute “R3trans –w logfile
export_r3trans_control_file”
3.Some of the tables are managed with SQL, others with tp (like security)…..this is mostly just historical and most could managed with R3trans.
Hints…… (cont.)
R3trans export control file format:
ExportClient=nnnFile=‘/usr/sap/trans/tmp/export_db_refresh.data’select * from RFCDES select * from RZLLITAB select * from QSENDDEST select * from TVARV select * from TPFET select * from TPFHT select * from TPFID select * from BTCOMSET select * from CCC_CHECK select * from CCCFLOW select * from DBCON
Hints…… (cont.)
R3trans import control file format:
importclient=nnnfile=‘/usr/sap/trans/tmp/export_db_refresh.data’
R3trans command format:
hostname:sidadm 3> cd /usr/sap/trans/tmphostname:sidadm 3> R3trans –w …/log/import_data_nnn.log
import_control_file_name
Hints…… (cont.)
How to find tables that could/should be exported:
Decide what areas need to be manually adjusted after each refresh…From the transaction in R/3, press F1, then F9. This will display the table that contains the field that you have to export/import.Add that table to the R3trans export list.TestAnd more Testing (there are probably more tables)
Note: This is probably not supported by SAP
Hints…… (cont.)
Here are some script samples (all in ksh!):
Prompting for parameters:##################################################################
echo "\n Enter SID of source system: \c" read OLD echo "\n Enter SID of target system: \c" read NEW echo "\nDo you want to delete all scheduled jobs? [y] \c " read answer delete_scheduled_jobs=${answer:-y}
Hints…… (cont.)
Deleting scheduled batch jobs: (put this in a SQL script - it is generic!!!)
##################################################################-- TBTCO: job status overview tabledelete from sapr3.tbtcowhere jobname not like 'SAP_%‘ and jobname not like 'DBA%';-- TBTCS: batch schedule tabledelete from sapr3.tbtcswhere jobname not like 'SAP_%‘ and jobname not like 'DBA%';-- TBTCP: batch log tabledelete from sapr3.tbtcpwhere jobname not like 'SAP%’ and jobname not like 'DBA%';
Hints…… (cont.)
Making sure there is enough space in the sapdatas before attempting the restore
################################################################### Create temporary file that contains the source systems list of sapdatasremsh $source -l ora$old -n "bdf -l | grep OLD | grep sapdata" > /tmp/OLD_sapdatas
2> /dev/null
# Compare with target sapdatasfor i in /oracle/SID/sapdata* do new_sapdata=`basename $i` free_space=`bdf -l | grep ${i}$ | awk '{print $2}'` needed_space=`grep ${new_sapdata}$ /tmp/OLD_sapdatas | awk '{print $3}'` if [[ $needed_space -gt $free_space ]] then echo sapdata $i needs $needed_space more space fidone
Hints…… (cont.)
Creating a SQL on the fly that can be used to rename the datafiles:
###########################################################################spool &3/chglogs.sql
select 1 orderit, 'alter database rename file ' from dual unionselect 2 orderit, ''''||member||''''||',' from v$logfile unionselect 3 orderit, 'to' from dual unionselect 4 orderit, ''''||rtrim(replace(member,upper('&&1'),upper('&&2')))||''''||',' from
v$logfile order by 1;
spool off;
Note: This chglogs.sql must be modified to make it executable……see next slide…..
Hints…… (cont.)
Sample SQL to rename log files:
###########################################################################alter database rename file'/oracle/oldSID/origlogA/log_g11m1.dbf','/oracle/oldSID/origlogA/log_g13m1.dbf','/oracle/oldSID/origlogB/log_g12m1.dbf','/oracle/oldSID/origlogB/log_g14m1.dbf',to'/oracle/newSID/origlogA/log_g11m1.dbf','/oracle/newSID/origlogA/log_g13m1.dbf','/oracle/newSID/origlogB/log_g12m1.dbf','/oracle/newSID/origlogB/log_g14m1.dbf‘;
Hints…… (cont.)
Renaming log files:
dbshutdbstart mountremsh localhost -l ora$new -n "\$ORACLE_HOME/bin/svrmgrl <<EOF connect internal; --- change log file references @${NEW}/sql/chglogs.sql --- change file references @${NEW}/sql/chgfiles.sql alter database open; --- reset transport number @${NEW}/sql/tadir.sql – updates header record in tadir exitEOF" >> /dev/null 1>$SQLERR_LOG_FILE 2>&1
Hints…… (cont.)
Creating a SQL on the fly that reflects the new SID and then can then be executed:
###########################################################################echo "\n Generating the change_printer_profiles.sql script ...\n“
echo "connect internal;" > ${NEW}/sql/change_printer.sqlecho "update sapr3.tsp03c set pamsserver='${new_print_server}' where pamsserver='$
{old_print_server};" >> ${NEW}/sql/change_printer.sqlecho "update sapr3.tsp03d set pamsserver='${new_print_server}’ where pamsserver='$
{old_print_server}';" >> ${NEW}/sql/change_printer_profiles.sqlecho "commit;" >> ${NEW}/sql/change_printer.sql
Hints…… (cont.)
This the script can then be executed as :
##################################################################remsh localhost -l ora$new -n "\$ORACLE_HOME/bin/svrmgrl <<EOF connect internal; --- change printer profiles @${NEW}/sql/change_printer_profiles.sql --- backup controlfile alter database backup controlfile to trace;EOF" >> /dev/null 1>$SQLERR_LOG_FILE 2>&1
Created on previous slide dynamically
Hints…… (cont.)
Interesting OSS Notes :
Note #549828 – Database Copy with an Online BackupNote #179149 – Inst 4.6B R/3 Inst. Homogeneous System Copy – install new system with DBload from another systemNote #147243 – R3Copy under NT Oracle – install new system with DB copy – talks about the control file work that needs to be done.
QUESTIONS ???????
Thank you for attending!Please remember to complete and return your evaluation form following this session.
Session Code: 911A