Embed Size (px)
DESCRIPTION
Citation preview
Using Virtualization to setup Oracle QA/Development Database Environments
2/16/11 1
Using Virtualization to setup Oracle QA/Development Database
Environments
Rocky Mountain Training DaysFeb. 16, 2011
Mike Gangler – Ithaka / JSTOR
Presentation Agenda
• Who am I• Describe QA / Development Challenges• Pre-Virtual Oracle Environment layout• Solaris Zones and ZFS• ZFS Commands Used• Virtual Oracle Environment layout• Steps required – Zones• Scripts required• Demo if time / Wifi• Recap and QA
16-Feb-2011 3
Quick Survey
• How many people are using a form of Virtualization ?
16-Feb-2011
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?
16-Feb-2011
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?
16-Feb-2011
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?
16-Feb-2011
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?– Other ?
16-Feb-2011
Quick Survey
• How many people are using a form of Virtualization ?– VMWARE ?– Solaris ?– Oracle VM ?– Other ?
• How many are running oracle instances on these VM machines ?
16-Feb-2011
Michael J. Gangler (Mike)
• Database Specialist – Ithaka– Working with Oracle, MSSQL, MySQL
• Working with Oracle since Oracle Version 5• IOUG
– Executive Vice President – Finance– Director of Membership services– www.ioug.org– April 10 – 14 – Collaborate –Orlando
• Southeast Michigan Oracle Professionals (SEMOP)– Past President and Current Director– www.semop.org
• Michigan Oracle User Summit (MOUS)– Database / Tech. director– www.mous.us– Wednesday November 30th – Schoolcraft College
16-Feb-2011
Ithaka’s mission
• ITHAKA is a not-for-profit organization dedicated to helping the academic community take full advantage of rapidly advancing information and networking technologies. We serve scholars, researchers, and students by providing the content, tools, and services needed to preserve the scholarly record and to advance research and teaching in sustainable ways. We are committed to working in collaboration with other organizations to maximize benefits to our stakeholders.
16-Feb-2011 11
Ithaka’s Services
• Ithaka S+R works with initiatives and organizations to develop sustainable business models and conducts research and analysis on the impact of digital media on the academic community as a whole. www.ithaka.org
• JSTOR helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive of over 1,200 academic journals and other content. JSTOR uses information technology and tools to increase productivity and facilitate new forms of scholarship. www.jstor.org
• Portico preserves scholarly literature published in electronic form—more than 10,000 e-journals and 28,000 e-books—and ensures that these materials remains accessible to future scholars, researchers, and students. www.portico.org
16-Feb-2011 12
16-Feb-2011
Pre-Virtual Environment
16-Feb-2011
Database / Application Specifics
• Oracle 10.2.0.4• SGA – 11 gb• Oracle Dataguard / standby used• Data is replicated via application and Daemons• 1 schema – 160 – 200 gb• Holds only pointers versus actual files• Files are stored at each data center• Software is a 3rd party application
16-Feb-2011 15
QA / Development Challenges
• QA requires restoration from Production - Long export / Import times – (6-8 Hours)
• QA restore every 1 – 2 weeks.• Multi-Data Center requires additional work and
headaches• Development needs many restores due to changes
and testing • Application written data replication (Versus Database)
16-Feb-2011 16
Pre-Virtual QA Environment
16-Feb-2011 17
WebApp
Master
Web App
Slave
Web App
Slave
DataCenter
1
DataCenter
2
Data Center
3
Qa.jstor.org
Data Replication
Data Replication
Pre-Virtual QA Steps
• Export Production database – 2 hours• Zip production export – 30 minutes• Copy production export file to sites
– Manchester, England – 3 hours– Princeton , NJ – 1.5-2 hours– Ann Arbor – 1 hour
• Unzip export file – 30 minutes• Import export file – 3 – 5 hours
• Total Time for all 3 sites (approx) – 10 – 12 hours
16-Feb-2011 18
Virtual EnvironmentZones & ZFS
16-Feb-2011
20
Solaris Zones
16-Feb-2011
21
ZFS
16-Feb-2011
ZFS is a combined file system and logical volume manager designed by Sun Microsystems.
ZFS includes :data integrity (protection against bit rot, etc.)Support for high storage capacities Integration of the concepts of file system and volume
managementCan provide snapshots and copy-on-write clonesContinuous integrity checking and automatic repairRAID-Z and native NFSv4 ACLs. ZFS is implemented as open-source software
22
Solaris ZFS
16-Feb-2011
23
ZFS Commands Used
16-Feb-2011
“zfs snapshot” - performs a copy of the data “zoneadm” – halts and starts the local zones “zfs destroy” – removing a clone or zpool “zfs clone” - Writeable snapshots , resulting in
two independent file systems that share a set of blocks.
“zfs list” – lists out all the snapshots “zfs set mountpoint” – allocates an area data
to a mountpoint “zfs set zoned” - turns on/off the local zones
ZFS Snapshot
An advantage of copy-on-write is that when ZFS writes new data, the blocks containing the old data can be retained, allowing a snapshot version of the file system to be maintained. ZFS snapshots are created very quickly, since all the data composing the snapshot is already stored; they are also space efficient, since any unchanged data is shared among the file system and its snapshots.
• Runs from the Master site• Need Root password• All oracle processes need to be shutdown• User must be in NON- /u01 directory
16-Feb-2011 24
ZFS Snapshot - Example
• [oracle@aa2vps217]$ su• Password: • # zfs snapshot tank/zones/aa2vps217-u01@20100701• # zfs list• NAME USED AVAIL REFER MOUNTPOINT• tank 849G 624G 267K /tank• tank/zones 821G 624G 26K /tank/zones• tank/zones/aa2vps217-u01 677G 624G 400G /u01• tank/zones/aa2vps217-u01@20100602 133G - 162G -• tank/zones/aa2vps217-u01@20100701 0G - 447G -
16-Feb-2011
Virtual EnvironmentProcess and setup
16-Feb-2011
2716-Feb-2011
•Qa.jstor.org
•Web •Master
WebSlave
•Web•Slave
•ff•Db•Master
•Db•Slave
•Db•slave
1 physical Machine
1 Global Zone3 Zones
1 physical Machine1 Global Zone3 Zones
Virtual QA environment
Virtual QA Steps
• Export Production database – Not Required – Using Previous nights backup * cron job copies it to master site nightly
• Zip production export – N/A• Unzip Export File – 30 Minutes• Import Export File to MASTER ONLY ! - 2 – 3 Hours• Run ZFS SNAPSHOT Master Only – 5 Minutes• Run ZFS CLONE to slave sites (run from global Zone) - 10 Mins• Run script to change database names (Each site) – 5 Minutes• Total Time for all 3 sites (approx) – 3 – 4 Hours
16-Feb-2011 28
ZFS Cloning Process
• Runs from the Global Zone Site – Must be root user
• Cloning steps: Halt the zones
zoneadm –z <slave> halt Destroy the old u01 directories (slave machines only)
zfs destroy –r tank/zones/<slave>-u01 Clone the master u01 directory to the slaves
Zfs clone tank/zones/<master>-u01@`date +%Y%m%d` tank/zones/<slave1>-u01
16-Feb-2011 29
ZFS Cloning Process (Con’t)
List out Zone copiesZfs list –o name,origin
Set the mount pointsZfs set mountpoint=/u01 tank/zones/<slave>-u01
Set the zone back onZfs set zoned=on tank/zones/<slave>-u01
Reboot the zonesZoneadm –z <slave> boot
List out Zone copies – same as above
16-Feb-2011 30
3116-Feb-2011
Example Clone Script – Page 1
#!/usr/bin/env bash
export PATH=/usr/local/bin:$PATH
host=aa2x4240-04echo =====Must be run as SU======echo =====Cloning omega - 217, 218, 219 ======
echo =====listing out the zone copies=======echo zfs list -o name,originzfs list -o name,origin
echo =====halting zones=======zoneadm -z aa2vps218 haltzoneadm -z aa2vps219 halt
ZFS Clone Example
3216-Feb-2011
Example Clone Script – Page 2 (cont)
echo =====destroying old u01 directories======zfs destroy -r tank/zones/aa2vps218-u01zfs destroy -r tank/zones/aa2vps219-u01
echo =====cloning snapshots===========zfs clone tank/zones/aa2vps217-u01@`date +%Y%m%d` tank/zones/aa2vps218-u01zfs clone tank/zones/aa2vps217-u01@`date +%Y%m%d` tank/zones/aa2vps219-u01
echo =====listing out the new zone copies======echo zfs list -o name,originzfs list -o name,origin
ZFS Clone Example – Page 2
3316-Feb-2011
Example Clone Scripts – Page 3 (cont)
echo =====setting the mount points========zfs set mountpoint=/u01 tank/zones/aa2vps218-u01zfs set mountpoint=/u01 tank/zones/aa2vps219-u01
echo =====set zoned=on===========zfs set zoned=on tank/zones/aa2vps218-u01zfs set zoned=on tank/zones/aa2vps219-u01
echo =====boot machines=======zoneadm -z aa2vps218 bootzoneadm -z aa2vps219 boot
echo =====listing out the new zone copies======zfs list -o name,origin
ZFS Clone Example – Page 3
After Clone Script - Slaves
• NOTE – used if the database name needs to be the same
• Rename listener.ora, tnsnames.ora with new machine name
• Clean up Grid Control –
– Remove entries from sysman/emd directory– Change sysman/emd/targets.xml – new machine– Change sysman/emd/emd.proprties – new machine name
16-Feb-2011 34
After Clone Example Script - Slaves
• NOTE – used if the database name needs to be the same
• Runs on Slave Sites – as Oracle User• Needs to be in $HOME
• #!/bin/bash
• ### set -xv• oldmach=$1• newmach=$2
• for FILE in $ORACLE_HOME/network/admin/*.ora• do• sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out• mv tmp.out $FILE• done
16-Feb-2011 35
After Clone Example Script – Slaves – part 2
rm -r -f $AGENT_HOME/sysman/emd/core*.*rm -r -f $AGENT_HOME/sysman/emd/state/*rm -r -f $AGENT_HOME/sysman/emd/upload/*rm -r -f $AGENT_HOME/sysman/emd/recv/*rm -r -f $AGENT_HOME/sysman/emd/collection/*rm -r -f $AGENT_HOME/sysman/emd/lastupld.xmlrm -r -f $AGENT_HOME/sysman/emd/agntstmp.txt
for FILE in $AGENT_HOME/sysman/emd/targets.xmldo sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out mv tmp.out $FILEDonefor FILE in $AGENT_HOME/sysman/config/emd.propertiesdo sed -e 's/'"$oldmach"'/'"$newmach"'/g' $FILE > tmp.out mv tmp.out $FILEdone
exit 0
16-Feb-2011 36
3716-Feb-2011
# ./clone-presentation.sh=====Must be run as SU===========Cloning omega - 97, 98, 99 and ===========halting zones=======zoneadm -z aa2vps98 haltzoneadm -z aa2vps99 halt=====destroying old u01 directories======zfs destroy -r tank/zones/aa2vps98-u01zfs destroy -r tank/zones/aa2vps99-u01=====cloning snapshots===========zfs clone tank/zones/aa2vps97-u01@20101012 tank/zones/aa2vps98-u01zfs clone tank/zones/aa2vps97-u01@20101012 tank/zones/aa2vps99-u01=====setting the mount points========zfs set mountpoint=/u01 tank/zones/aa2vps98-u01zfs set mountpoint=/u01 tank/zones/aa2vps99-u01cannot mount '/u01': directory is not emptyproperty may be set but unable to remount filesystem=====set zoned=on===========zfs set zoned=on tank/zones/aa2vps98-u01zfs set zoned=on tank/zones/aa2vps99-u01=====boot machines=========zoneadm -z aa2vps98 bootzoneadm -z aa2vps99 boot
Example Run – ZFS Clone – Page 1
3816-Feb-2011
=====listing out the new zone copies======zfs list -o name,originNAME ORIGINtank -tank/snaps -tank/swap1 -tank/zones -tank/zones/aa2vps191 -tank/zones/aa2vps191-u01 -tank/zones/aa2vps247-v01 -tank/zones/aa2vps247-v01@20081106_detached -tank/zones/aa2vps247-v02 -tank/zones/aa2vps247-v02@20081105_detached -tank/zones/aa2vps248-v01 tank/zones/aa2vps247-v01@20081106_detachedtank/zones/aa2vps248-v02 tank/zones/aa2vps247-v02@20081105_detachedtank/zones/aa2vps249-v01 tank/zones/aa2vps247-v01@20081106_detachedtank/zones/aa2vps249-v02 tank/zones/aa2vps247-v02@20081105_detachedtank/zones/aa2vps97 tank/zones/aa2vps98@200909131322-migrationtank/zones/aa2vps97-u01 -tank/zones/aa2vps97-u01@20101008 -tank/zones/aa2vps98 tank/zones/aa2vps99@200909131307-migrationtank/zones/aa2vps98@200909131322-migration -tank/zones/aa2vps98-u01 tank/zones/aa2vps97-u01@20101008tank/zones/aa2vps99 -tank/zones/aa2vps99@200909131307-migration -tank/zones/aa2vps99-u01 tank/zones/aa2vps97-u01@20101008#
Example Run – ZFS Clone – Page 2
3916-Feb-2011
TIPS
o Keep the zone root, Oracle Binaries and Oracle data on the SANo Keep the resource management simple – Use dedicated CPU
feature in lieu resource poolso To Prevent potential problems in the global zone:
Create a separate file system for each local zone root Avoid using bock/raw devices in non-global zones
o For Greater Isolation of Environments: Keep the zone root, Oracle Binaries and Oracle data on
separate file systems and mounts. Install Oracle inside each non-global zone rather than
sharing binaries from the global zone.
40
Conclusion
16-Feb-2011
Virtualization can reduce down time Very Efficient with Test and QA
databases I haven’t tried it with Production
databases, but we do use it for Production Web Servers
Need to understand all the applications on the local zones
You can clone across local zones and global zones
Concepts can be used with any virtualization software (VMWARE, etc).
