oracle 11.2.0.2 RAC for HPUX.pdf

Version 변경일자 변경자(작성자) 주요내용

1 2011-01-30 조 훈 문서 최초 작성

Author 조 훈

Creation Date 2011-01-30

Last Updated

Version 1.0

Copyright(C) 2004 Goodus Inc.

All Rights Reserved

ORACLE 11gR2 (11.2.0.2)

RAC install

for HP-UX

ORACLE 10g RAC install patch failover ON DELL

- 2 –

본 산출물의 내용은 굿어스㈜의 사전승인 없이

외부유출 및 공개를 금지합니다.

Contents

1. 들어가며 ............................................................................................................................... 3

2. System Overview ................................................................................................................ 3

2.1. H/W and OS Information .........................................................................................3 2.1.1. Hardware ..................................................................................................................... 3 2.1.2. OS ............................................................................................................................... 3 2.1.3. Shared Storage ........................................................................................................... 3

2.2. IP 구성정보 ...............................................................................................................4

2.3. OS USER 정보 ...........................................................................................................4

2.4. 패키지 .......................................................................................................................4

3. Post Installation Steps ....................................................................................................... 6

3.1. Preparing the system ...............................................................................................6 3.1.1. Host file Configuration ................................................................................................. 6 3.1.2. Kernel Configuration ................................................................................................... 6 3.1.3. Check pach ................................................................................................................. 7 3.1.4. RSH Configuration ...................................................................................................... 7 3.1.5. Oracle User Limits Value Configuration ...................................................................... 7 3.1.6. User OS enviropment Configuration ........................................................................... 7

3.2. Network Configure Check ........................................................................................8 3.3. Raw Device for OCR, VOTEDISK ..............................................................................9

3.3.1. Raw device Creation ................................................................................................... 9

4. Install steps ...................................................................................................................... 10

4.1. Clusterware Installation ....................................................................................... 10 4.1.1. Clusterware Installation ............................................................................................. 10

4.1.2. OCR 변경 ................................................................................................................. 24

4.1.3. votedisk 변경 ............................................................................................................ 24

4.2. Database Installation ........................................................................................... 26 4.2.1. Change Permission ................................................................................................... 26 4.2.2. Database engine install ............................................................................................. 26 4.2.3. DBCA ........................................................................................................................ 34

4.2.4. ASM resource 제거................................................................................................... 46

4.3. Check Configuration ............................................................................................. 46

5. 설치 후 발견된 이슈 ........................................................................................................... 47

5.1. Error while reading ADR file after upgraded to 11.2.0.2 ...................................... 47 5.2. HP-UX Time Mismatch Between Ocssd.Log And Sysdate ..................................... 47


- 3 –



1. 들어가며

이 내용은 HP-UX Itanium 에 Oracle 11.2.0.2 신규 설치에 대한 내용으로 ASM 이나 CFS 가 아닌 raw

device 를 사용하여 RAC 구성을 목적으로 합니다.

2. System Overview

2.1. H/W and OS Information

2.1.1. Hardware

항 목 NODE1 ( goodus1 ) NODE2 ( goodus2 )

MODEL HP BL870c HP BL870c

CPU 8CPU Quad Core 8CPU Quad Core

MEMORY 32GB 32GB

Swap 64GB 64GB

2.1.2. OS

항 목 NODE1 ( goodus1 ) NODE2 ( goodus2 )

OS HP-UX 11i v3 HP-UX 11i v3

Hostname goodus1 goodus2

Kernel B.11.31 U ia64 B.11.31 U ia64

2.1.3. Shared Storage

항 목 위치 SIZE

OCR /dev/dbs20/rocr01, /dev/dbs20/rocr02 300M

VOTE /dev/dbs20/rvot01 , /dev/dbs20/rvot02 , /dev/dbs21/rvote03 300M

ASM /dev/rdisk/disk378 1,024M

CONTROL /dev/dbs22/rcontrol1 , /dev/dbs22/rcontrol2 , /dev/dbs22/rcontrol3 300M

SYSTEM /dev/dbs22/rsystem01 4,096M

REDO (thread 1) /dev/dbs22/rredo111 , /dev/dbs22/rredo121 , /dev/dbs22/rredo131 ,

/dev/dbs22/rredo112 , /dev/dbs22/rredo122 , /dev/dbs22/rredo132 1,024M

REDO (thread 2) /dev/dbs22/rredo211 , /dev/dbs22/rredo221 , /dev/dbs22/rredo231 ,

/dev/dbs22/rredo212 , /dev/dbs22/rredo222 , /dev/dbs22/rredo232 1,024M

UNDOTBS1, 2 /dev/dbs23/rundotbs1 , /dev/dbs23/rundotbs2 10,240M

TEMP /dev/dbs23/rtemp 1,024M

USERS /dev/dbs22/ruser 2,048M


- 4 –



SYSAUX /dev/dbs22/rsysaux 10,240M

SPFILE /dev/dbs22/rspfile 300M

2.2. IP 구성정보

항 목 NODE1 (goodus1) NODE2 (goodus2) 비 고

lan900 ***.***.***.58 ***.***.***.59 Public IP

lan902 192.168.200.1 192.168.200.2 Interconnect IP

lan900:1 ***.***.***.158 ***.***.***.159 Service IP

lan900:2 ***.***.***.160 SCAN IP

2.3. OS USER 정보

USERNAME GROUP SHELL Home Direcoty PROFILE

grid oinstall(700), dba(701), asmdba(702) /bin/ksh /oracle/app/grid .profile

oracle oinstall(700), dba(701) /bin/ksh /oracle/app/oracle .profile

2.4. 패키지

Oracle 에서 권고하는 package 목록과 설치된 package 목록 HP-UX Operating System Itanium:

» HP-UX 11iV3 patch Bundle Sep/ 2008 (B.11.31.0809.326a)

All installations for HP-UX 11i V3 (11.31) requires the following patches (or newer versions – the

current successor status March 2010):

» PHCO_40381 11.31 Disk Owner Patch

» PHKL_38038 VM patch - hot patching/Core file creation directory

→ replaced by PHKL_40942

» PHKL_38938 11.31 SCSI cumulative I/O patch


» PHKL_39351 Scheduler patch : post wait hang


» PHSS_36354 11.31 assembler patch

→ replaced by PHSS_40546

» PHSS_37042 11.31 hppac (packed decimal)

» PHSS_37959 Libcl patch for alternate stack issue fix (QXCR1000818011)


» PHSS_39094 11.31 linker + fdp cumulative patch


- 5 –




» PHSS_39100 11.31 Math Library Cumulative Patch


» PHSS_39102 11.31 Integrity Unwind Library


» PHSS_38141 11.31 aC++ Runtime


Patch to run 11gR2 RAC on SG/SGeRAC

» PHSS_40886 11.31 Serviceguard for RAC A.11.19.00

Pro*C/C++, Oracle Call Interface, Oracle C++ Call Interface, Oracle XML Developer’s Kit (XDK)

Patch for HP-UX 11i V3 (11.31) on HP-UX Itanium:

» PHSS_39824 - 11.31 HP C/aC++ Compiler (A.06.23) patch

→ Replaced by PHSS_40631

VERITAS File System

» PHKL_39773: 11.31 VRTS 5.0 GARP6 VRTSvxfs Kernel Patch

→ Replaced by PHKL_40650


- 6 –



3. Post Installation Steps

3.1. Preparing the system

3.1.1. Host file Configuration

/etc/hosts 파일에 각노드의 Pulic IP 주소 및 VIP, Interconnect 주소를 등록합니다.

(root 사용자로 양쪽 노드 모두 수행합니다.)

[root@goodus1 ~]# cat /etc/hosts

127.0.0.1 localhost.localdomain localhost

***.***.***.58 goodus1

***.***.***.59 goodus2

***.***.***.158 goodus1-vip

***.***.***.159 goodus2-vip

192.168.200.1 goodus1-priv

192.168.200.2 goodus2-priv

***.***.***.160 goodus-scan

3.1.2. Kernel Configuration

Oracle 을 기동하기 위한 OS kernel 값 설정합니다.

Parameter Minimum Required Value

ksi_alloc_max 32768

executable_stack 0

maxfiles 1024

maxfiles_lim 63488

max_thread_proc 1024

maxdsiz 1073741824 (1 GB)

maxdsiz_64bit 2147483648 (2 GB)

maxssiz 134217728 (128 MB)

maxssiz_64bit 1073741824 (1 GB)

maxuprc 3686

msgmap 4096

msgmni 4096

msgtql 4096

ncsize 35840

nflocks 4096

ninode 34816

nkthread 7184


- 7 –



nproc 4096

semmni 4096

semmns 8192

semmnu 4096

semvmx 32767

shmmax The size of memory or 1073741824 (0X40000000), whichever is greater.

shmmni 4092

shmseg 512

3.1.3. Check pach

OS 패치 설치 유무를 확인 합니다.


# /usr/sbin/swlist -l patch

3.1.4. RSH Configuration

RAC 설치를 위한 rsh 설정을 합니다.

RAC 는 설치중 노드갂 Binary File 을 Copy 합니다. 이를 위해 rsh 나 ssh 의 설치가 필요합니다.

rsh, rcp, rlogin 을 패스워드 없이 작동하기 위한 설정

양쪽 host 갂 oracle 유져에 대한 remote 접근을 password 없이 가능하도록 설정합니다.


[root@goodus1 ~]# su - oracle

/u01/app/oracle> cat .rhosts

+

/u01/app/oracle>

3.1.5. Oracle User Limits Value Configuration

Oracle 사용자가 생성하는 최대 process 개수와 open 할 수 있는 최대 file 개수를 설정합니다.

/etc/security/limits.conf 파일에 아래 함목을 추가합니다.


oracle soft nproc 2047

oracle hard nproc 16384

oracle soft nofile 1024

oracle hard nofile 65536

3.1.6. User OS enviropment Configuration

사용자의 shell 홖경을 설정 합니다. .profile 을 아래와 같이 생성합니다.

grid user


- 8 –



set -o vi

umask 022

export PS1=[`hostname`:\$PWD]$

export LANG=C

export EDITOR=vi

export GRID_HOME=/oracle/app/11.2.0/grid

export ORACLE_BASE=/oracle/app/grid

export ORACLE_HOME=$GRID_HOME

export ORACLE_SID=+ASM1

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib:$ORACLE_HOME/rdbms/lib

export SHLIB_PATH=$ORACLE_HOME/lib:$ORACLE_HOME/rdbms/lib

export PATH=$PATH:$ORACLE_HOME/bin:/usr/local/bin:$ORACLE_HOME/OPatch:

export

CLASSPATH=$ORACLE_HOME/jre:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib:$ORACLE_HOME/network/jlib:$OR

ACLE_HOME/OPatch

oracle user

export ORACLE_BASE=/oracle/app/oracle

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db

export GRID_HOME=/oracle/app/11.2.0/grid

export ORACLE_SID=goodus1

export ORACLE_TERM=xterm

export PATH=$ORACLE_HOME/bin:$GRID_HOME/bin:$PATH

export EDITOR=vi

3.2. Network Configure Check

11.2.0.2 에서 Oracle Grid Infrastructure 는 cluster interconnec 에 대한 오라클 공급 중복 허용 인터커넥트 이중화라

는 새로운 기능을 소개합니다. 이 새로운 기능으로, 개인 상호 연결 네트워크에서 멀티 캐스트 네트워크 통신은 클

러스터의 피어 노드와 통신을 설정하는 부트 스트랩에 홗용, 일단 커뮤니케이션이 설립 네트워크 통신은 다음 유니

캐스트로 젂홖됩니다. 이 multicast 통신은 private interconnect network 의 230.0.1.0 address(port 42424)를 사용합니

다. 그러므로 private interconnect network 에 multicast 가 홗성화 되어 있어야만 multicast 주소 230.0.1.0 에 대한 모

든 cluster node 에서 재대로 작동합니다. 오라클 문서 ID 1212703.1 참고

metalink 에서 mcasttest.pl perl 프로그램을 다운받아 확인해 볼 수 있습니다.

$perl mcasttest.pl –n goodus1, goodus2 -i lan902

########### Setup for node goodus1 ##########

Checking node access 'goodus1'

Checking node login 'goodus1'

Checking/Creating Directory /tmp/mcasttest for binary on node 'goodus1'

Distributing mcast2 binary to node 'goodus1'

########### Setup for node goodus2 ##########

Checking node access 'goodus2'

Checking node login 'goodus2'


- 9 –



Checking/Creating Directory /tmp/mcasttest for binary on node 'goodus2'

Distributing mcast2 binary to node 'goodus2'

########### testing Multicast on all nodes ##########

Test for Multicast address 230.0.1.0

Jan 17 17:18:21 | Multicast Succeeded for lan902 using address 230.0.1.0:42000

Test for Multicast address 224.0.0.251

Jan 17 17:18:22 | Multicast Succeeded for lan902 using address 224.0.0.251:42001

3.3. Raw Device for OCR, VOTEDISK

3.3.1. Raw device Creation

Clusterware 에 필요한 OCR 과 VOTEDISK 를 위한 raw device 를 생성하고 renaming 합니다.


chown root:dba /dev/dbs20/rocr01 /dev/dbs20/rocr02

chown oracle:dba /dev/dbs20/rvot01 /dev/dbs20/rvot02 /dev/dbs21/rvote03 /dev/rdisk/disk378

chmod 660 /dev/dbs20/* /dev/dbs21/* /dev/rdisk/disk378


- 10 –



4. Install steps

4.1. Clusterware Installation

4.1.1. Clusterware Installation

Clusterware 는 각 노드갂의 동기화 및 클러스터링을 제공합니다. Clusterware 를 설치하기 위해 아래의 젃차대로

수행합니다.

(oracle 사용자로 goodus1 에서 수행합니다.).

$ export ORACLE_HOME=/oracle/app/11.2.0/grid

$ ./runInstaller &

1.

2


- 11 –



3.

4.


- 12 –



5

6


- 13 –



7.

8.


- 14 –



9.

10.


- 15 –



11.

12.


- 16 –



13.

14.


- 17 –



15.

16.


- 18 –



17.

각 노드에서 root 유저로 orainstRoot.sh 와 root.sh 를 실행합니다.


- 19 –



goodus1: /oracle/app/oraInventory/orainstRoot.sh

#/oracle/app/oraInventory/orainstRoot.sh

Changing permissions of /oracle/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /oracle/app/oraInventory to oinstall.

The execution of the script is complete.

goodus2: /oracle/app/oraInventory/orainstRoot.sh

#/oracle/app/oraInventory/orainstRoot.sh

Changing permissions of /oracle/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /oracle/app/oraInventory to oinstall.

The execution of the script is complete.

goodus1: /oracle/app/11.2.0/grid/root.sh

#/oracle/app/11.2.0/grid/root.sh

Running Oracle 11g root script...

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /oracle/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /oracle/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

LOCAL ADD MODE

Creating OCR keys for user 'root', privgrp 'sys'..

Operation successful.

OLR initialization - successful

root wallet


- 20 –



root wallet cert

root cert export

peer wallet

profile reader wallet

pa wallet

peer wallet keys

pa wallet keys

peer cert request

pa cert request

peer cert

pa cert

peer root cert TP

profile reader root cert TP

pa root cert TP

peer pa cert TP

pa peer cert TP

profile reader pa cert TP

profile reader peer cert TP

peer user cert

pa user cert

Adding daemon to inittab

CRS-2672: Attempting to start 'ora.mdnsd' on 'goodus1'

CRS-2676: Start of 'ora.mdnsd' on 'goodus1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'goodus1'

CRS-2676: Start of 'ora.gpnpd' on 'goodus1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'goodus1'

CRS-2672: Attempting to start 'ora.gipcd' on 'goodus1'

CRS-2676: Start of 'ora.cssdmonitor' on 'goodus1' succeeded

CRS-2676: Start of 'ora.gipcd' on 'goodus1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'goodus1'

CRS-2672: Attempting to start 'ora.diskmon' on 'goodus1'

CRS-2676: Start of 'ora.diskmon' on 'goodus1' succeeded

CRS-2676: Start of 'ora.cssd' on 'goodus1' succeeded

ASM created and started successfully.

Disk Group ASM created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys.



CRS-4256: Updating the profile

Successful addition of voting disk 8fd08aa2236e4f42bf6d75b33ae020d7.


- 21 –



Successfully replaced voting disk group with +ASM.


CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 8fd08aa2236e4f42bf6d75b33ae020d7 (/dev/rdisk/disk378) [ASM]

Located 1 voting disk(s).

CRS-2672: Attempting to start 'ora.asm' on 'goodus1'

CRS-2676: Start of 'ora.asm' on 'goodus1' succeeded

CRS-2672: Attempting to start 'ora.ASM.dg' on 'goodus1'

CRS-2676: Start of 'ora.ASM.dg' on 'goodus1' succeeded

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

goodus2: /oracle/app/11.2.0/grid/root.sh

#/oracle/app/11.2.0/grid/root.sh

Running Oracle 11g root script...

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /oracle/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /oracle/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

LOCAL ADD MODE



OLR initialization - successful

Adding daemon to inittab

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on

node unknown, number unknown, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

Configure Oracle Grid Infrastructure for a Cluster ... succeeded


- 22 –



node2 에서 root.sh 돌릴때, CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS

daemon on node unknown, number unknown, and is terminating 에러가 발생합니다.

이 에러는 HP 10 Gigabit Ethernet NIC driver 사용시 10GigEthr-02 (iexgbe) B.11.31.1009.01 or later 필요합니다.

오라클 문서 ID 1276346.1 참고

#ioscan -fnC lan

Class I H/W Path Driver S/W State H/W Type Description

=====================================================================

lan 0 0/0/0/3/0/0/0 iexgbe CLAIMED INTERFACE HP PCIe 2-p 10GbE Built-in








#swlist -l product|grep -i iex

IEXGBE-DRV B.11.31.1011 HP PCIe 10GigEthr Driver

18.

19.


- 23 –



CRS 설치가 완료 되었는지 확인합니다.

(root 사용자로 양쪽 노드 모두 수행합니다.).

$crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora.ASM.dg ora....up.type ONLINE ONLINE goodus1

ora....ER.lsnr ora....er.type ONLINE ONLINE goodus1

ora....N1.lsnr ora....er.type ONLINE ONLINE goodus1

ora.asm ora.asm.type ONLINE ONLINE goodus1

ora.cvu ora.cvu.type ONLINE ONLINE goodus1

ora.gsd ora.gsd.type OFFLINE OFFLINE

ora....network ora....rk.type ONLINE ONLINE goodus1

ora.oc4j ora.oc4j.type ONLINE ONLINE goodus1

ora.ons ora.ons.type ONLINE ONLINE goodus1

ora....SM1.asm application ONLINE ONLINE goodus1

ora....P1.lsnr application ONLINE ONLINE goodus1

ora....mp1.gsd application OFFLINE OFFLINE

ora....mp1.ons application ONLINE ONLINE goodus1

ora....mp1.vip ora....t1.type ONLINE ONLINE goodus1

ora....SM2.asm application ONLINE ONLINE goodus2

ora....P2.lsnr application ONLINE ONLINE goodus2

ora....mp2.gsd application OFFLINE OFFLINE

ora....mp2.ons application ONLINE ONLINE goodus2


- 24 –



ora....mp2.vip ora....t1.type ONLINE ONLINE goodus2

ora.scan1.vip ora....ip.type ONLINE ONLINE goodus1

[goodus1:/work/patchset/grid]$

4.1.2. OCR 변경

해당 설치문서는 ASM 이 아닌 raw 를 사용하므로 ASM 에 설치된 OCR 을 raw device 로 변경합니다.

#[/oracle/app/11.2.0/grid/bin]./ocrconfig -add /dev/ dbs20/rocr01

#[/oracle/app/11.2.0/grid/bin]./ocrconfig -add /dev/ dbs20/rocr02

#[/oracle/app/11.2.0/grid/bin]./ocrconfig -delete +ASM

#[/oracle/app/11.2.0/grid/bin]./ocrcheck

Status of Oracle Cluster Registry is as follows :

Version : 3

Total space (kbytes) : 262120

Used space (kbytes) : 2356

Available space (kbytes) : 259764

ID : 642797593

Device/File Name : /dev/ dbs20/rocr01

Device/File integrity check succeeded

Device/File Name : /dev/ dbs20/rocr02

Device/File integrity check succeeded

Device/File not configured



Cluster registry integrity check succeeded

Logical corruption check succeeded

4.1.3. votedisk 변경

해당 설치문서는 ASM 이 아닌 raw 를 사용하므로 ASM 에 설치된 votedisk 를 raw device 로 변경합니다.

#[/oracle/app/11.2.0/grid/bin]./crsctl replace votedisk /dev/ dbs20/rvot01

Now formatting voting disk: /dev/ dbs20/rvot01.


Successful addition of voting disk ae4387b930394f76bf17923430bae8e3.


- 25 –



Successful deletion of voting disk 8fd08aa2236e4f42bf6d75b33ae020d7.


CRS-4266: Voting file(s) successfully replaced

#[/oracle/app/11.2.0/grid/bin]./crsctl add css votedisk /dev/ dbs20/rvot02


CRS-4603: Successful addition of voting disk /dev/ dbs20/rvot02.

#[/oracle/app/11.2.0/grid/bin]./crsctl add css votedisk /dev/ dbs21/rvot03


CRS-4603: Successful addition of voting disk /dev/ dbs21/rvot03.

#[/oracle/app/11.2.0/grid/bin]

#[/oracle/app/11.2.0/grid/bin]./crsctl query css votedisk

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE ae4387b930394f76bf17923430bae8e3 (/dev/ dbs20/rvot01) []

2. ONLINE 02818a19cc1d4ffebff79b96aa2ddea2 (/dev/ dbs20/rvot02) []

3. ONLINE 63210c7dc5fe4f5cbf682e3dbd9cdd8f (/dev/ dbs21/rvot03) []

Located 3 voting disk(s).


- 26 –



4.2. Database Installation

Database 를 raw device 로 생성합니다.

4.2.1. Change Permission

Database 를 생성하기위해 raw device 의 소유권 및 권한을 변경합니다. (양쪽 노드에서 모두 적용)

$chown oracle:dba /dev/dbs22/rcontrol1 /dev/dbs22/rcontrol2 /dev/dbs22/rcontrol3

$chown oracle:dba /dev/dbs22/rsystem01

$chown oracle:dba /dev/dbs22/rredo111 /dev/dbs22/rredo121 /dev/dbs22/rredo131 /dev/dbs22/rredo112

/dev/dbs22/rredo122 /dev/dbs22/rredo132

$chown oracle:dba /dev/dbs22/rredo211 /dev/dbs22/rredo221 /dev/dbs22/rredo231 /dev/dbs22/rredo212


$chown oracle:dba /dev/dbs23/rundotbs1 /dev/dbs23/rundotbs2

$chown oracle:dba /dev/dbs23/rtemp

$chown oracle:dba /dev/dbs22/ruser

$chown oracle:dba /dev/dbs22/rsysaux

$chown oracle:dba /dev/dbs22/rspfile

$chmod 660 /dev/dbs22/rcontrol1 /dev/dbs22/rcontrol2 /dev/dbs22/rcontrol3

$chmod 660 /dev/dbs22/rsystem01

$chmod 660 /dev/dbs22/rredo111 /dev/dbs22/rredo121 /dev/dbs22/rredo131 /dev/dbs22/rredo112


$chmod 660 /dev/dbs22/rredo211 /dev/dbs22/rredo221 /dev/dbs22/rredo231 /dev/dbs22/rredo212


$chmod 660 /dev/dbs23/rundotbs1 /dev/dbs23/rundotbs2

$chmod 660 /dev/dbs23/rtemp

$chmod 660 /dev/dbs22/ruser

$chmod 660 /dev/dbs22/rsysaux

$chmod 660 /dev/dbs22/rspfile

HP Itanium 에서 11.2.0.2 이상 설치 혹은 업그레이드시 raw device 로 생성한 controlfile, datafile, redo log file 들은

마운트 단계에서 block device 오픈시 에러를 발생하므로 반드시 아래 패치를 적용해줍니다.

HPUX patch PHCO_41479 - 오라클 문서 ID 1269346.1 참고

4.2.2. Database engine install

(oracle 사용자로 goodus1 에서 수행합니다.).

$ export ORACLE_HOME=/oracle/app/oracle/product/11.2.0/db

$ ./runInstaller &

1.


- 27 –



2.

3.


- 28 –



4.

5.


- 29 –



6.

7.


- 30 –



8.

9.


- 31 –



10.

11.


- 32 –



12.

13.


- 33 –



14.


- 34 –



4.2.3. DBCA

raw device 를 사용시 OUI 를 사용할 수 없으므로 DBCA 를 실행하여 생성스크립트를 추출하여 editing 합니다.

1.

2.


- 35 –



3.

4.

5.


- 36 –



6.

7.


- 37 –



8.

9.


- 38 –



10.

11.


- 39 –



12.

13.


- 40 –



14.

15.


- 41 –



16.

17.


- 42 –



18.

19. Create Database 체크를 해제하고 스크립트 생성을 체크해서 스크립트만 생성하도록 합니다.


- 43 –



생성된 스크립트 폴더에 들어가서 다음 파일들을 수정합니다.

init.ora - controlfile 을 추가하고, ASM 과련 설정을 주석처리 합니다.

control_files='/dev/dbs22/rcontrol1','/dev/dbs23/rcontrol2','/dev/dbs24/rcontrol3'

#db_create_file_dest="+ASM"

goodus1.sql - ASM 관련 부분을 수정합니다.

set verify off

ACCEPT sysPassword CHAR PROMPT 'Enter new password for SYS: ' HIDE

ACCEPT systemPassword CHAR PROMPT 'Enter new password for SYSTEM: ' HIDE

ACCEPT sysmanPassword CHAR PROMPT 'Enter new password for SYSMAN: ' HIDE

ACCEPT dbsnmpPassword CHAR PROMPT 'Enter new password for DBSNMP: ' HIDE

ACCEPT asmSysPassword CHAR PROMPT 'Enter ASM SYS user password: ' HIDE

host /oracle/app/oracle/product/11.2.0/db/bin/orapwd

file=/oracle/app/oracle/product/11.2.0/db/dbs/orapwGOODUS1 force=y

host /oracle/app/11.2.0/grid/bin/setasmgidwrap o=/oracle/app/oracle/product/11.2.0/db/bin/oracle

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl add database -d GOODUSDB -o

/oracle/app/oracle/product/11.2.0/db -p /dev/dbs22/rspfile -n GOODUSDB

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl add instance -d GOODUSDB -i GOODUS1 -n goodus1

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl add instance -d GOODUSDB -i GOODUS2 -n goodus2

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl disable database -d GOODUSDB

@/oracle/app/oracle/admin/GOODUSDB/scripts/CreateDB.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/CreateDBFiles.sql


- 44 –



@/oracle/app/oracle/admin/GOODUSDB/scripts/CreateDBCatalog.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/context.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/xdb_protocol.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/emRepository.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/apex.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/CreateClustDBViews.sql

host echo "SPFILE='/dev/dbs22/rspfile " > /oracle/app/oracle/product/11.2.0/db/dbs/initGOODUS1.ora

@/oracle/app/oracle/admin/GOODUSDB/scripts/lockAccount.sql

@/oracle/app/oracle/admin/GOODUSDB/scripts/postDBCreation.sql

CreateDB.sql - datafile 경로를 수정합니다.

SET VERIFY OFF

connect "SYS"/"&&sysPassword" as SYSDBA

set echo on

spool /oracle/app/oracle/admin/GOODUSDB/scripts/CreateDB.log append

startup nomount pfile="/oracle/app/oracle/admin/GOODUSDB/scripts/init.ora";

CREATE DATABASE "GOODUSDB"

MAXINSTANCES 32

MAXLOGHISTORY 1

MAXLOGFILES 192

MAXLOGMEMBERS 3

MAXDATAFILES 2048

DATAFILE '/dev/dbs22/rsystem01' SIZE 4000M

EXTENT MANAGEMENT LOCAL

SYSAUX DATAFILE '/dev/dbs22/rsysaux' SIZE 10200M

SMALLFILE DEFAULT TEMPORARY TABLESPACE TEMP TEMPFILE '/dev/dbs23/rtemp01' SIZE 1000M

SMALLFILE UNDO TABLESPACE "UNDOTBS1" DATAFILE '/dev/dbs23/rundotbs1' SIZE 10200M

CHARACTER SET AL32UTF8

NATIONAL CHARACTER SET AL16UTF16

LOGFILE GROUP 1 ('/dev/dbs22/rredo111','/dev/dbs23/rredo112') SIZE 1000M,

GROUP 2 ('/dev/dbs22/rredo121','/dev/dbs23/rredo122') SIZE 1000M,

GROUP 3 ('/dev/dbs22/rredo131','/dev/dbs23/rredo132') SIZE 1000M

USER SYS IDENTIFIED BY "&&sysPassword" USER SYSTEM IDENTIFIED BY "&&systemPassword";

set linesize 2048;

column ctl_files NEW_VALUE ctl_files;

select concat('control_files=''', concat(replace(value, ', ', ''','''), '''')) ctl_files from v$parameter where name

='control_files';

host echo &ctl_files >>/oracle/app/oracle/admin/GOODUSDB/scripts/init.ora;

spool off

CreateDBFiles.sql – datafile 경로를 수정합니다.

SET VERIFY OFF


set echo on


- 45 –



spool /oracle/app/oracle/admin/GOODUSDB/scripts/CreateDBFiles.log append

CREATE UNDO TABLESPACE "UNDOTBS2" DATAFILE '/dev/dbs23/rundotbs2' SIZE 10200M;

CREATE TABLESPACE "USERS" LOGGING DATAFILE '/dev/dbs22/ruser' SIZE 2000M EXTENT MANAGEMENT

LOCAL SEGMENT SPACE MANAGEMENT AUTO;

ALTER DATABASE DEFAULT TABLESPACE "USERS";

spool off

postDBCreation.sql – datafile 경로 수정 및 ASM 관련 설정을 제거합니다.

SET VERIFY OFF


set echo on

spool /oracle/app/oracle/admin/GOODUSDB/scripts/postDBCreation.log append

@/oracle/app/oracle/product/11.2.0/db/rdbms/admin/catbundle.sql psu apply;

select 'utl_recomp_begin: ' || to_char(sysdate, 'HH:MI:SS') from dual;

execute utl_recomp.recomp_serial();

select 'utl_recomp_end: ' || to_char(sysdate, 'HH:MI:SS') from dual;

select group# from v$log where group# =3;

select group# from v$log where group# =4;

ALTER DATABASE ADD LOGFILE THREAD 2 GROUP 4 ('/dev/dbs22/rredo211','/dev/dbs23/rredo212') SIZE

1000M,

GROUP 5 ('/dev/dbs22/rredo221','/dev/dbs23/rredo222') SIZE 1000M,

GROUP 6 ('/dev/dbs22/rredo231','/dev/dbs23/rredo232') SIZE 1000M

ALTER DATABASE ENABLE PUBLIC THREAD 2;

host echo cluster_database=true >>/oracle/app/oracle/admin/GOODUSDB/scripts/init.ora;

host echo remote_listener=goodus-cluster-scan:1521>>/oracle/app/oracle/admin/GOODUSDB/scripts/init.ora;


set echo on

create spfile='SPFILE='/dev/dbs22/rspfile‘ FROM pfile='/oracle/app/oracle/admin/GOODUSDB/scripts/init.ora';

shutdown immediate;

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl enable database -d GOODUSDB;

host /oracle/app/oracle/product/11.2.0/db/bin/srvctl start database -d GOODUSDB;


host /oracle/app/oracle/product/11.2.0/db/bin/emca -config dbcontrol db -silent -cluster -CLUSTER_NAME

goodus-cluster -LOG_FILE /oracle/app/oracle/admin/GOODUSDB/scripts/emConfig.log -SID PIS001 -

DB_UNIQUE_NAME GOODUSDB -EM_HOME /oracle/app/oracle/product/11.2.0/db -SERVICE_NAME GOODUSDB

-PORT 1521 -LISTENER_OH /oracle/app/11.2.0/grid -LISTENER LISTENER -ORACLE_HOME

/oracle/app/oracle/product/11.2.0/db -HOST goodus1;

spool off

exit;

해당파일들을 수정후 goodus1.sh 를 실행시켜 database 를 생성합니다.


- 46 –



4.2.4. ASM resource 제거

필요없어진 ASM resource 를 제거합니다.

$srvctl stop diskgroup -g ASM

$srvctl stop asm -n goodus1

$srvctl stop asm -n goodus2

$srvctl disable diskgroup -g ASM

$srvctl disable asm -n goodus1

$srvctl disable asm -n goodus2

4.3. Check Configuration

설치가 완료되면 현재 OCR 에 등록된 구성 정보를 확인합니다.

Oifcfg getif 조회 값과 x$ksxpia 조회값이 반드시 올바른 값으로 일치해야 합니다.

(oracle 사용자로 양쪽 노드 모두 수행합니다.)

[goodus1:/home/grid]$crs_stat -t

Name Type Target State Host

------------------------------------------------------------

ora.ASM.dg ora....up.type OFFLINE OFFLINE

ora....ER.lsnr ora....er.type ONLINE ONLINE goodus1

ora....N1.lsnr ora....er.type ONLINE ONLINE goodus2

ora.asm ora.asm.type OFFLINE OFFLINE

ora.cvu ora.cvu.type ONLINE ONLINE goodus2

ora.gsd ora.gsd.type OFFLINE OFFLINE

ora....network ora....rk.type ONLINE ONLINE goodus1

ora.oc4j ora.oc4j.type ONLINE ONLINE goodus2

ora.ons ora.ons.type ONLINE ONLINE goodus1

ora....01db.db ora....se.type ONLINE ONLINE goodus1

ora....SM1.asm application OFFLINE OFFLINE

ora....S1.lsnr application ONLINE ONLINE goodus1

ora....us1.gsd application OFFLINE OFFLINE

ora....us1.ons application ONLINE ONLINE goodus1

ora....us1.vip ora....t1.type ONLINE ONLINE goodus1

ora....SM2.asm application OFFLINE OFFLINE

ora....S2.lsnr application ONLINE ONLINE goodus2

ora....us2.gsd application OFFLINE OFFLINE

ora....us2.ons application ONLINE ONLINE goodus2

ora....us2.vip ora....t1.type ONLINE ONLINE goodus2

ora.scan1.vip ora....ip.type ONLINE ONLINE goodus2


- 47 –



5. 설치 후 발견된 이슈

5.1. Error while reading ADR file after upgraded to 11.2.0.2

Database 를 startup 할 때 alert.log 에서 다음과 같은 에러를 확인할 수 있습니다.

…

Error with dbgriap_init_adr_pga: 48178

ORA-48178: error encountered while reading an ADR block file during ADR initialization

[/oracle/app/orapis/diag/rdbms/pis001/PIS0011/metadata/ADR_INTERNAL.mif]

ORA-48122: error with opening the ADR block file

[/oracle/app/orapis/diag/rdbms/pis001/PIS0011/metadata/ADR_INTERNAL.mif] [0]

ORA-27037: unable to obtain file status

HPUX-ia64 Error: 251: Function is not available

Additional information: 46

Wed Jan 19 13:00:43 2011

ERROR: The process is unable to create the ADR schema in the diagnostic_dest directory

ERROR: because of a disk issue or OS platform issue

Wed Jan 19 13:00:43 2011

ERROR: Reverting back to using the user_dump_dest and background_dump_dest

ERROR: as the location for the traces and logs

Starting ORACLE instance (normal)

…

이 에러는 11.2.0.2 에서 'ADR_INTERNAL.mif' 파일을 오픈할 때 HP-UX directio code 를 사용하는데 HP OnLineJFS

product 이 설치되지 않아 발생하는 이슈입니다.

해결 방법은 HP OnLineJFS product 을 설치하거나 init.ora 에 _DIAG_ADR_ENABLED 를 FALSE 로 설정합니다.


5.2. HP-UX Time Mismatch Between Ocssd.Log And Sysdate

cssd.log 를 확인하면 다음 메시지를 4 초 마다 볼 수 있다.

2011-01-19 15:52:05.243: [ CSSD][38]clssnmSendingThread: sending status msg to all nodes

2011-01-19 15:52:05.243: [ CSSD][38]clssnmSendingThread: sent 4 status msgs to all nodes





해당 메시지는 HP-UX 의 sysdate 와 ocssd.log 사이에 time 이 맞지않아 발생하는 에러로 HP-UX 설치 후에


- 48 –



“/etc/default/tz”에 기본 TZ 설정이 포함되는데 이 값이 실제 시갂과 일치하지 않아 발생합니다.

해결방법은 실제 timezone 에 /etc/default/tz 를 설정합니다.


$date

Wed Jan 19 15:57:13 KST 2011

$cat /etc/default/tz

KST-9

Documents

oracle 11.2.0.2 RAC for HPUX.pdf