Virtualizing Oracle 11g/R2 RAC Database on Oracle VM: Methods/Tips

IOUG Collaborate 12, Virtualization Boot Camp Session #328, Monday April 23rd, 3:45-4:45pm

Kai Yu Oracle Solutions Engineering Lab Dell Inc.

• Kai Yu, kai_yu@dell.com

– 17 years with Oracle Technology: DBA, Apps DBA, Architect

– Hands-on experience on IT solution stack: hardware to software

– Focus on Oracle RAC, Oracle VM and Oracle EBS

– Oracle ACE Director, author and frequent presenter

– IOUG Oracle RAC SIG President (2009-2010)

– IOUG Virtualization SIG Webinar Chair

– 2011 OAUG Innovator of Year Award Winner

– Oracle Blog: http://kyuoracleblog.wordpress.com/

• Dell Oracle Solutions Engineering: www.dell.com/oracle

– Oracle Technology Solutions on Dell systems/storages

– Dell | Oracle Solutions Components

– Solutions stack: servers, storage, network, OS, Oracle VM, Oracle RAC, Oracle Applications

About Author

2

Introduction to Oracle VM and Oracle RAC

Establishing Oracle VM Infrastructure

Creating Virtual Machines

Configuring 11g R2 RAC database on Oracle VM

High Availability of Oracle RAC on VM

Recorded Demo: Oracle VM 3 Virtualization Environment

3

Agenda

• Introduction of Oracle VM Virtualization Solution. – Software based server Virtualization solutions for X86 – Server partitions and consolidation – High availability and scalability – Fast deployment and standardization – Provide the virtualization support for Grid /Cloud

• Oracle VM Architecture – Oracle VM server: Xen Hypervisor, management Linux kernel with support of devices, IO, networking, etc. – VM: guest OS with applications running on domU

4

Introduction to Oracle VM and RAC

5

Virtual Server Pool: an autonomous region of VM servers

Oracle VM Manager: GUI Management solution for Oracle VM

Oracle VM Agent: installed with VM server and communicating with Oracle VM manager

• Oracle VM 3 New Features – Application Driven Virtualization – Scalability: up to 160 physical CPUs, 2 TB of memory 128 virtual CPUs and 1TB memory per guest VM – Oracle VM Manager provides centralized management

Fusion middleware Apps on Weblogic and Database Provide Brower-based GUI Interface Provide completed network and storage management Cloning, sharing, configuring and migrating VMs. Dynamic Resource Management for VMs All VM operations are submitted as jobs Managed by Oracle Enterprise Manager

– Centralized network management Separate logic networks; management through GUI – Centralized Storage Configuration Storage Connect, Automaticity storage repository configuration

7

• Oracle Real Application Clusters (RAC) – Multiple instances running interconnected by high speed private network

share a single database – Oracle Cache Fusion Technology for node synchronization – All Instances can execute transactions simultaneously on the single

database in the shared storage – High Availability and scalability for the applications

8

• Why running RAC On Oracle VM – Server partition and consolidation – Sub-capacity licensing – Virtual cluster on single physical server – Template base deployment

• Overall Architecture – VM servers connect to physical networks and storage. – Virtual Infrastructure provides CPUs, Memory, Network, Storage

resources – Guest VMs as Oracle RAC Nodes – All the network and storage go through VM servers

10

• Deployment Method of RAC On Oracle VMs – Guest VMs are running OS just like physical host – Required Shared storage and public and private Network – Oracle 11.R2 GI and Oracle RAC are running on Guest OS – RAC VM templates or Manual RAC Configuration

11

• Virtual System Architecture Design

• Implementation Tasks Overview – VM server Installation and VM Manager Installation – Connect VM servers to the VM Manager – Virtual Infrastructure : network and storage configuration. – Create guest VM using VM template – Configure network and shared storage on VMs

Establishing Oracle VM Infrastructure

12

VM Server Installation – Bare metal Installation, turn on the virtualization in BIOS – Change Dom 0 memory configuration to 1024GB in VM2.2 – Oracle VM 3.0: no manual work on VM server after install

– Ensure VM agent working: #service ovs-agent status

VM Manager Installation Discover/connect VM servers vs OVS agent

–Virtual Infrastructure based on Oracle VM 3 (FC storage)

VM server network configuration :

– Oracle VM 3: separated networks, managed through VM Manager: Logic networks vs physical networks

.

15

3 Logical networks: public, management and private

Public Network mapped to physical ports

Map the public network to physical network ports

Map the management network to physical ports:

Private Network mapped to physical ports

Map the private network to physical ports

• Storage Configuration – Connect to shared storage and setup multipath devices Oracle VM 3.0: Oracle VM manager connects to the storage through a set of plug-ins (storage connect) – Oracle generic iSCSI plug-in and SCSI plug-in – Also created the multipath devices

• Create VM Repository in manual steps (Oracle VM 2.2) – Configure 02cb service :$service o2cb configure – Create OCFS2 cluster file system on the storage volume

mkfs.ocfs2 -T datafiles -N 8 -L "OVS" /dev/mapper/ovsp1 – Create VM repository:

$/opt/ovs-agent-2.3/utils/repos.py -n /dev/mapper/ovsp1 – make the newly created repository the cluster root repository

$/opt/ovs-agent-2.3/utils/repos.py -r <uuid> • VM Repository structure

• Create Storage Repository in Oracle VM 3 – Creation step automated by Oracle VM Manager: automated tasks: Created the partition, create the OCFS2 file system, created the repository

.

• Oracle VM 3

• Prepare the physical disks for database on VMs (VM 3)

.

• Different ways to create virtual machines Using ISO file of an Installation DVD, Booting VM via PXE Using Oracle VM templates or Assemblies

• Creating virtual machines from VM templates – Download the VM templates, Import the VM templates:

– Create virtual machines by cloning VM template

Creating and Configuring Virtual Machines

• Configure Network on Virtual Machines (VM 3.0) Create virtual NICs Assign the virtual NICs to VM

', ]

Port 1

Guest VM

Bridge

eth0:

00:21:f6:00:00:02

eth1:

00:21:f6:00:00:03

bond3

Bridge

VM Server

port3 Port 4

Public Private

Private Network

Public Network

Configure Network on Virtual Machines (VM 3.0) Add the shared physical disks and local disk to VMs :

vvdb and xvdc for Data and FRA

physical volumes of the virtual machine: xvdb and xvdc for shared storage Data and FRA xvdd for local disk

xvdd for local disk

26

• Methods to create Oracle RAC Database on OVM – Manual Installation and configuration – Automated Provisioning using EM Provisioning Procedure – Based on Oracle RAC 11g R2 Oracle VM Template: refer to Saar’s

Power of the New Oracle RAC 11g Release 2 Oracle VM Templates • Installing & configuring Oracle RAC OVM

– Network configuration: private, private, SCAN IPs, VIPs – Disable firewall service: $service iptables stop $chkconfig iptables off – Prerequisite checks: rpms, kernel configs, ntpd, etc limits in /etc/security/limits.conf for grid user and oracle user ./runcluvfy.sh stage -pre crsinst -n owirac1,owirac2 -r 11gR2 –verbose – Create ASM disks on the virtual disks: $service oracleasm configure -- configure ASM (all nodes) $fdisk /dev/xvdc, kpartx /dev/xvdc -- partition virtual disks $service oracleasm createdisk OCR1 /dev/xvdc1 (all nodes)

Configuring 11gR2 RAC Database on OVM

27

– Install 11gR2 Grid Infrastructure = Clusterware + ASM

28

• Specify the network interfaces:

Specify ASM disks for OCR and Votingdisk files:

29

Install Oracle RAC software ./cluvfy stage -pre dbinst -fixup -n owirac1,owirac2 -r 11gR2 -osdba dba -verbose ./runInstaller as Oracle user

Select two RAC nodes Select the software location

30

– Create ASM diskgroups for Database: as grid user: ./asmca

– Create RAC database : ./cluvfy stage -pre dbcfg -fixup -n

owirac1,owirac2, -d $ORACLE_HOME -verbose run ./dbca as oracle

Select Oracle Real Application Clusters database Select two RAC nodes Select +DATA for database area, +FRA for flash recovery area Specify the database configuration

31

• Two kinds of High Availability (HA) – Oracle VM HA (external HA) : Restart VM on another physical server – Oracle Clusterware base HA (internal HA) : Restart or relocate applications connections to another DB instance – For production database, Oracle RAC HA without VM HA must be used. For production database, it is not supported to co-locate VMs that runs RAC Instances in same physical servers. *Refer to Oracle Whitepaper: Oracle Real Applications Clusters in Oracle VM Environments Oracle VM Environment.

High Availability of Oracle RAC on VM

32

RAC One Node Database in Oracle VM environment. • RAC One Node fully supported in Oracle VM environment. • Run RAC One Node on Virtual Machine: • Ensure the high availability of RAC One Node database in VM The database will failover to another VM if the this VM node has issue

High Availability of Oracle RAC on VM

33

Oracle VM Live Migration: Migrate VM from one physical server to 2nd physical server RAC One Node Online Relocation Relocate the database from 1st VM node to 2nd node (VM) instance

High Availability of Oracle RAC on VM

34

Thank You and QA Visit my Oracle Blog at http://kyuoracleblog.wordpress.com

IOUG Virtualization SIG: www.ioug.org/virtualization

My Collaborate 12 Sessions and Panel

#920, Enterprise Manager 12c Deep Dive Sunday 9:am-3pm

Mandalay K

#840, Alternative Options to Enhance the High Availability of your Oracle Database

Monday, 12:15pm Palm C

#328, Virtualization Boot Camp: Virtualizing Oracle 11g/R2 RAC Database on Oracle VM 3.0 Monday, 3:45pm Palm D #312,Configuring and Managing a Private Cloud with Oracle Enterprise Manager 12c Tuesday,4:30pm Mandalay K Oracle on Oracle VM - Expert Panel, Oracle VM Expert Panel Wed, 9:30am Mandalay L