Managing Oracle Enterprise Manager Cloud Control 12c with Oracle Clusterware

Managing Oracle Enterprise Manager Cloud Control 12c

with Oracle ClusterwareSession ID#UGF9794

Leighton L. NelsonMercy

About Me

• Oracle DBA for 10+ years• Oracle ACE• Oracle Certified Expert: RAC and Grid Infrastructure

11gR2• Co-Author Expert Oracle Enterprise Manager Cloud

Control 12c – Apress 2013• Oracle RAC SIG US Events Chair/IOUG Liaison

Agenda

• OEM 12c Architecture Overview• OEM High Availability• Level 2 HA Configuration• Oracle Clusterware Setup• OEM Installation• OEM Failover • Management Repository Configuration

OEM 12c Architecture Overview

OEM High Availability Levels

Level Description Load Balancer Required Cost1 OMS and Repository on separate hosts.

No redundancy. No $

2 OMS installed on shared storage with VIP used in active/passive failover.Repository Database using local Data Guard.

No $$

3 Multiple OMSs in active/active configuration. Repository using RAC Database with local Data Guard

Yes, At primary site $$$

4 Primary OMS in active/active configuration with RAC Repository database. Standby OMS at DR site in active/active configuration. Standby RAC database at DR site

Yes. At pimary and standby sites

$$$$

Level 2 – Active/Passive HA

• A minimum of 2 Servers Required• OMS binaries installed on shared filesystem

• NFS/OCFS2/DBFS/ACFS

• OMS can run on one node in cluster at any given time

• Data Guard for Management Repository

Level 2 – Active/Passive OMS

Level 2 – Active/Passive OMS

Oracle Clusterware Setup

• Clusterware can be used to create/manage VIP• 11.2+ uses appvipcfg for VIP

<GRID_HOME>/bin/appvipcfg create -network=1 \ -ip=192.168.1.0 \ -vipname=omsvip \ -user=root

• VIP can be created on non-default network• In Oracle 12c Flex Clusters app vips can be create on

leaf nodes• Allow Oracle Grid Infrastructure software owner (e.g.

grid) to run the script to start the VIP.<GRID_HOME>/bin/crsctl setperm resource omsvip -u user:grid:r-x


• Start the VIP as the GI owner e.g. grid<GRID_HOME>/bin/crsctl start resource omsvip

• Check the status of the VIP<GRID_HOME>/bin/crsctl status resource omsvip

The status of the output should be similar to the following:

NAME=omsvip

TYPE=app.appvip_net1.type

TARGET=ONLINE

STATE=ONLINE on oms1


• Check if virtual hostname and VIP are resolvable

nslookup <omsvip>• Also do a reverse lookup of the IP address.

nslookup <virtual IP address>• Verify that the IP address returned from the nslookup

output is running on the OMS host.

ifconfig –a | grep <virtual IP address>

OEM Installation

• Create ORACLE_HOME for the OMS on the shared storage on all nodes in the cluster

mkdir –p /u01/app/oms_share

• Create Oracle Inventory directory under ORACLE_HOME for the OMS on all nodes

mkdir /u01/app/oms_share/oraInventory

• Create the inventory pointer in the oraInventory directoryvi oraInst.loc

inventory loc=/u01/app/oracle/oms_share/oraInventory

inst_group=oinstall

OEM Installation

• Install OEM by setting ORACLE_HOSTNAME environment variable

runInstaller -invPtrloc /u01/app/oms_share/oraInst.loc ORACLE_HOSTNAME=omsvip.example.com –debug

OEM Failover

• To manually relocate the VIP to another host in the cluster issue the following command.

$crsctl relocate res omsvipCRS-2673: Attempting to stop 'omsvip' on 'oms1'CRS-2677: Stop of 'omsvip' on 'oms1' succeededCRS-2672: Attempting to start 'omsvip' on 'oms2'CRS-2676: Start of 'omsvip' on 'oms2' succeeded

• Check if the IP address associated with the VIP is running on the relocated host.

ifconfig –a|grep <vip>

OEM Failover

• Establish IP on failover server (done through Clusterware)• Start listener (if part of same failover group)• Start database (if required)• Set the ORACLE_HOSTNAME environment variable to the

virtual hostname. Continuing with our example we use the command below.

export ORACLE_HOSTNAME=omsvip.example.com• Start the OMS on the new node

$OMS_HOME/bin/emctl start oms

Add OEM Clusterware resource

• OEM can be added as Clusterware resource• Administrator managed (static, 2 – nodes)• Policy managed (dynamic, > 2 nodes)

• Handled through Agent Framework• C/C++

• Create Action Script• Store on shared storage• Specify START/STOP/CHECK/ABORT routines

Example

$ crsctl add server pool oem_sp -attr "PARENT_POOLS=Generic, SERVER_NAMES=oms1 oms2”

$ crsctl add resource oem -type cluster_resource -attr "ACTION_SCRIPT=/opt/cluster/scripts/oem12c.scr, PLACEMENT='restricted', SERVER_POOLS=oem_sp, CHECK_INTERVAL='30', RESTART_ATTEMPTS='2', START_DEPENDENCIES='hard(omsvip)', STOP_DEPENDENCIES='hard(omssvip)'"

Clusterware resource

• Oracle 12c introduced a new generic_application resource type

crsctl add resource oem -type generic_application -attr "START_PROGRAM='$OMS_HOME/bin/emctl start oms', STOP_PROGRAM='$OMS_HOME/bin/emctl stop oms', CLEAN_PROGRAM='$OMS_HOME/bin/emctl stop oms –all

-force', CHECK_PROGRAM='$OMS_HOME/bin/emctl status oms'

Management Repository

• Can be configured using Data Guard with Fast-Failover• Also can be configured with RAC/RAC One Node

• Storage becomes SPOF

• Listeners should be reachable from all nodes in cluster• Database should be started before starting OEM

Configure OEM with Data Guard Repo

Create database service for fast failover.

Configure OEM with Data Guard Repo

emctl config oms -store_repos_details -repos_conndesc '"(DESCRIPTION=(FAILOVER=ON)(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=repo1)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HOST=repo2)(PORT=1521)))(CONNECT_DATA=(SERVICE_NAME=emrep))(FAILOVER_MODE=(TYPE=select)(METHOD=basic)))"' -repos_user sysman

Configure OEM with RAC/RAC One Node Repo

emctl config oms -store_repos_details -repos_conndesc "

(DESCRIPTION= (ADDRESS=(PROTOCOL=TCP)

(HOST=emrep-scan.example.com)(PORT=1521) )

(CONNECT_DATA=(SERVER=DEDICATED) (SERVICE_NAME = emrep)))"

-repos_user sysman

References

• How to Configure Grid Control OMS in Active/Passive CFC Environments failover / HA (Doc ID 405642.1)

• How to Configure the OMS Connect String when Repository is in a Dataguard setup [ID 1328768.1]

• Oracle® Clusterware Administration and Deployment Guide12c Release 1

COLLABORATE14.IOUG.ORGCOLLABORATE14.IOUG.ORG

• Network with 5,000+ database and application professionals

• 5 days of in-depth education built by users, for users

• Complimentary Pre-Conference Workshops for IOUG registrants

April 7 – 11, 2014The Venetian Resort,

Las Vegas, NV

Attend for free!IOUG’s Call for Speakers is now open

collaborate14.ioug.org/call-for-speakers

Communities

Training Close to HomeTraining Close to Home

Maximum Availability Architecture with Oracle’s Larry Carpenter and Joe MeeksOctober 15: Milwaukee, WIOctober 16: Rochester, NY

Performance and Internals with Craig Shallahamer November 6: Chicago, ILNovember 8: Atlanta, GA

Get more information at www.ioug.org/masterclass or visit the IOUG kiosk in Moscone West, 2nd floor

IOUG Master Class Series ReturnsComing soon to a city near you!

One Day Interactive Training Events to meet your educational needs

Come See IOUG at the User Group PavilionCome See IOUG at the User Group Pavilion

Win a Free Registration to COLLABORATE 14!

Stop by the IOUG kiosk in the User Group Pavilion in Moscone West, 2nd floor, to pick up a lucky poker chip and

see if you are a winner of a free registration to COLLABORATE 14 in Las Vegas!

Copyright © 2013, Oracle and/or its affiliates. All rights reserved.27

Complimentary eBook Register Now

www.mhprofessional.com/dbsec

Use Code: db12c

THANK YOU

http://blogs.griddba.com

linkedin.com/pub/leighton-nelson/4/6b8/a7a

@leight0nn

[email protected]

Technology

Managing Oracle Enterprise Manager Cloud Control 12c with Oracle Clusterware