Vbox virtual box在oracle linux 5 - shoug 梁洪响

Vbox VirtualBox 在 Oracle Linux 5.7 上安装 Oracle

Database 11g Release 2 RAC （11.2.0.3）

前言：

本文档使用虚拟机搭建 RAC 学习环境，希望对大家的学习 RAC有所帮

助。

前期规划：

1.RAC 逻辑架构图

逻辑架构图可以描述整个架构，对系统有个整体的印象。

在搭建环境前，我们还需要注意以下问题： 1）系统部署安装所需的磁盘空间， CPU 和内存容量。

2）由于 11gR2 的安装在内存使用上比 11gR1 大, 每台虚拟机至少保证 3G RAM（推荐 4G 以上）内存容量。

3）对于虚拟机安装时磁盘空间分配，推荐使用动态分配模式来建立磁盘空间，因为这会造成不必要的空间

浪费。不过在创建共享磁盘时，则必须使用固定大小分配空间模式建立。

4）在安装中 SCAN (Single Client Access Name)需要提前在 DNS 或 GNS 进行设置，从而在多个 IP 地址

上进行轮询(round-robin)。为避免复杂性，本文不会进行 DNS 服务器配置, 所以为了完成安装，我们在

/etc/hosts 文件中定义一个 SCAN IP 地址，这可能会造成预安装 DNS 检查失败，但这并不影响 RAC 之后的

安装。 5）本文使用 64 位 Oracle Enterprise Linux 5.7

2. RAC 安装规划表

安装前最好有个规划，这样在真个安装过程中，做到心中有数，可以

有条不紊的推荐整安装过程。

3.相关软件

Oracle Linux Release 5 Update 7 Media Pack for x86_64 (64 bit)

11.2.0.3.0 Patch Set for Oracle Database Server (Patchset)

FileZilla

VirtualBox

备注：相关软件请自行搜索获取

创建虚拟机：

1.虚拟机目录设置

安装 vbox 后，请设置 Default Machine Folder，这样可以使得创建

的虚拟机文件保存在你设置的目录中。

管理 -> 全局设定…

2.创建新的虚拟机

需要有 2 个虚拟 RAC 节点。可以先建立 1 个节点，然后通过克隆

来建立第二个节点，这样可以节省时间。

01 按工具条上'新建'按钮新建虚拟机。

02 内存设置 2560, 下一步（N）.可以动态调整，根据自己机器情况

调整，最好 3GB以上。

03 选择现在创建虚拟硬盘（C）

04 使用默认 VDI 文件存储虚拟机文件即可,下一步

05 非共享存储使用动态分配节省磁盘空间。下一步

06 设置保存路径及虚拟磁盘大小，这里设置 25GB 创建

07 点击右边网络, 我们需要设置 2 张网卡(NIC)，这里我选择

host-only 方式。

08 进入系统，设置启动顺序 Boot Order, 取消软驱的勾选，并重

新设置启用顺序如下。

到这里就先配置好了一个虚拟机。

3.操作系统安装:

01 启动刚建立的虚拟机 tony1.选择 Oracle Linux 5u7 镜像文件装入。

02 Enter

03 跳过 CD 介质测试。

04 Next

05 选择安装时使用语言 English, Next

06 Keyboard 选择 U.S. English, Next.

07 初始化安装，清空所有数据。按 Yes

08 勾选'Review and modify partitioning layout'， Next.

09 选 Yes

10 由于 Oracle Linux 5u7 是 Oracle 11gR2 认证的系统，所以其实

我们可以不需要检查 swap 分配，不过还是请确保 swap 3G+，这样才

能避免 Oracle RAC 安装验证失败。 Next.

11 Next.

12 根据安装规划，这里我们设置 eth0, eth1，并设置主机名

编辑 eth0

取消勾选 IPv6 support，并手工设置 IP 地址及其掩码：

192.168.8.20/255.255.255.0

编辑 eth1

取消勾选 IPv6 support, 并手工设置 IP 地址及其掩码:

10.10.10.2/255.255.255.0

设置网关及 DNS:

Gateway: 192.168.8.1

Primary DNS: 8.8.8.8

Secondary DNS: 8.8.4.4

Next.

选 Asia/Shanghai 时区，取消 System clock uses UTC 的勾选(由

于本文 RAC 使用 ctss，所以关闭 ntp 服务)。

13 设置 root 密码（这里我们使用 oracle 作为密码）

为了保证安装顺利，请点选下列组件以确保相关必要工具存在：

?Desktop Environments > GNOME Desktop Environment

?Applications > Editors

?Applications > Graphical Internet

?Development > Development Libraries

?Development > Development Tools

?Servers > Server Configuration Tools

?Base System > Administration Tools

?Base System > Base

?Base System > System Tools

?Base System > X Window System

Next.

其它软件包件，可以忽略，暂时不装，一来加快操作系统的安装进

度，二来其余的软件包对于配置 RAC 11gR2 Oracle 不需要，如果在

将来的安装过程中，系统提示缺失软件包的：时候，我们也可以手工

从安装光盘中自行安。

Next

入格式化文件系统，安装软件包，直到最后

14 根据提示 reboot 系统：

禁用 Firewall.

禁用 SELinux.

Oracle 安装前预设置：

在操作系统安装后，还需要对系统进行进一步设置安装以满足 Oracle

安装需要。使用 root 登陆 tony1。

1.修改/etc/hosts 文件

[root@tony1 ~]# vi /etc/hosts

# Do not remove the following line, or various programs

# that require network functionality will fail.

127.0.0.1 localhost.localdomain localhost

::1 localhost6.localdomain6 localhost6

#tony1

192.168.8.20 tony1.yellowsand.com tony1

192.168.8.22 tony1-vip.yellowsand.com tony1-vip

10.10.10.2 tony1-priv.yellowsand.com tony1-priv

#tony2

192.168.8.21 tony2.yellowsand.com tony2

192.168.8.23 tony2-vip.yellowsand.com tony2-vip

10.10.10.3 tony2-priv.yellowsand.com tony2-priv

#scan ip

192.168.8.202 scan-cluster.yellow_sand.com scan-cluster

2.添加用户、用户组

配置 11g R2 RAC 需要安装 Oracle Grid Infrastructure 软件、

Oracle 数据库软件，其中 Grid 软件件等同于 10g Oracle 的

Clusterware 集群件。Oracle 建议不同的用户分别安装 Grid

Infrastructure 软件、Oracle 数据库软件， grid 用户安装 Oracle

Grid Infrastructure ，oracle用户安装 Oracle 数据库。并且

grid、oracle 用户需要属于不同的用户组。在配置 RAC时，还需求这

两个用户在 RAC 的不同节点上 UID， GID 要一致。用户、组的对应信

息表。

Group Name Group ID Group Info Oracle User(400) Grid User(500)

oinstall 4000 Inventory Group Y Y

dba 4001 OSDBA Group Y

oper 4002 OSOPER Group Y

asmadmin 5000 OSASM Y

asmdba 5001 OSDBA for ASM Y Y

asmoper 5002 OSOPER for ASM Y

groupadd -g 4000 oinstall

groupadd -g 4001 dba

groupadd -g 4002 oper

groupadd -g 5000 asmadmin

groupadd -g 5001 asmdba

groupadd -g 5002 asmoper

useradd -u 500 -g oinstall -G asmadmin,asmdba,asmoper -d /home/gird -s /bin/bash -c "Grid

Infrastructure Owner" grid

useradd -u 400 -g oinstall -G dba,oper,asmdba -d /home/oracle -s /bin/bash -c "Oracle Software

Owner" oracle

user grid, password is oracle

[root@tony1 ~]# passwd grid

Changing password for user grid.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

[root@tony1 ~]# passwd oracle

Changing password for user oracle.

New UNIX password:

BAD PASSWORD: it is based on a dictionary word

Retype new UNIX password:

passwd: all authentication tokens updated successfully.

3.建立 Oracle 对应文件目录

mkdir /u01

mkdir /g01

mkdir -p /u01/app/oracle

mkdir -p /g01/app/grid

chown –R oracle:oinstall /u01

chown -R grid:oinstall /g01

4.检查网络状态

service iptables status

service ntpd status

因为 Oracle Cluster Time Synchronization Service (ctssd)能同步 RAC 节点时间。因此

禁用 NTP 且清理相关文件:

chkconfig ntpd off

mv /etc/ntp.conf /etc/ntp.conf.orig

rm /var/run/ntpd.pid

[root@tony1 ~]# service iptables status

Firewall is stopped.

[root@tony1 ~]# service ntpd status

ntpd is stopped

[root@tony1 ~]# ls /etc/ntp.conf

/etc/ntp.conf

[root@tony1 ~]# mv /etc/ntp.conf /etc/ntp.conf.orig

[root@tony1 ~]# rm /var/run/ntpd.pid

rm: cannot lstat `/var/run/ntpd.pid': No such file or directory

5.配置 grid profile

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_SID=+ASM1 在节点 2 上修改成 ASM2

export ORACLE_BASE=/g01/app/grid

export ORACLE_HOME=/g01/app/11.2.0/grid

export ORACLE_TERM=xterm

export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'

export TNS_ADMIN=$ORACLE_HOME/network/admin

export PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JER:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

export EDITOR=vi

export LANG=en_US

export NLS_LANG=american_america.AL32UTF8

umask 022

6.配置 oracle profile

export TMP=/tmp

export TMPDIR=$TMP

export ORACLE_HOSTNAME=tony1.yellowsand.com

export ORACLE_SID=prod1 在节点 2 上修改成 prod2

export ORACLE_UNQNAME=PROD

export ORACLE_BASE=/u01/app/oracle

export ORACLE_HOME=$ORACLE_BASE/product/11.2.0/db_1

export ORACLE_TERM=xterm

export NLS_DATE_FORMAT='yyyy/mm/dd hh24:mi:ss'

export TNS_ADMIN=$ORACLE_HOME/network/admin

export PATH=/usr/sbin:$PATH

export PATH=$ORACLE_HOME/bin:$PATH

export LD_LIBRARY_PATH=$ORACLE_HOME/lib:/lib:/usr/lib

export CLASSPATH=$ORACLE_HOME/JER:$ORACLE_HOME/jlib:$ORACLE_HOME/rdbms/jlib

export EDITOR=vi

export LANG=en_US

export NLS_LANG=american_america.AL32UTF8

umask 022

7.配置 yum 并安装 Oracle 相关前提应用包

1）将 cdrom 挂载到/media/disk 以便于访问：

# ls /media

# mkdir /media/disk

[root@tony1 media]# mount /dev/cdrom /media/disk

mount: block device /dev/cdrom is write-protected, mounting read-only 2）检查是否以安装 yum 并设置

# rpm -q yum # touch /etc/yum.repos.d/public-yum-el5u7.repo

# vi /etc/yum.repos.d/public-yum-oel5u7.repo

[root@tony1 media]# rpm -q yum

yum-3.2.22-37.0.1.el5

[root@tony1 media]# touch /etc/yum.repos.d/public-yum-el5u7.repo

[root@tony1 media]# vi /etc/yum.repos.d/public-yum-el5u7.repo

[oel5u7]

name = Oracle Enterprise Linux 5.7-64bit DVD

baseurl=file:///media/disk/Server/

gpgcheck=0

enable=1

3）安装 oracle-validated 包并检查其安装配置

# yum install oracle-validated

[root@tony1 media]# yum install oracle-validated

Loaded plugins: rhnplugin, security

This system is not registered with ULN.

ULN support will be disabled.

oel5u7 | 1.1 kB 00:00

oel5u7/primary | 1.4 MB 00:00

oel5u7 3275/3275

Setting up Install Process

Resolving Dependencies

--> Running transaction check

---> Package oracle-validated.x86_64 0:1.1.0-14.el5 set to be updated

oel5u7/filelists | 3.1 MB 00:01

--> Processing Dependency: sysstat for package: oracle-validated

--> Processing Dependency: /usr/lib/libaio.so for package: oracle-validated

--> Processing Dependency: unixODBC-devel for package: oracle-validated

--> Processing Dependency: /usr/lib64/libodbccr.so for package: oracle-validated

--> Processing Dependency: libaio-devel for package: oracle-validated

--> Processing Dependency: /usr/lib/libodbc.so.1 for package: oracle-validated

--> Processing Dependency: /usr/lib64/libaio.so for package: oracle-validated

--> Processing Dependency: /usr/lib/libodbccr.so for package: oracle-validated

--> Processing Dependency: libXp.so.6 for package: oracle-validated

--> Processing Dependency: libdb-4.2.so()(64bit) for package: oracle-validated

--> Processing Dependency: libodbc.so.1()(64bit) for package: oracle-validated

--> Running transaction check

---> Package compat-db.x86_64 0:4.2.52-5.1 set to be updated

---> Package libXp.i386 0:1.0.0-8.1.el5 set to be updated

---> Package libaio-devel.i386 0:0.3.106-5 set to be updated

---> Package libaio-devel.x86_64 0:0.3.106-5 set to be updated

---> Package sysstat.x86_64 0:7.0.2-11.el5 set to be updated

---> Package unixODBC.i386 0:2.2.11-7.1 set to be updated

---> Package unixODBC.x86_64 0:2.2.11-7.1 set to be updated

---> Package unixODBC-devel.i386 0:2.2.11-7.1 set to be updated

---> Package unixODBC-devel.x86_64 0:2.2.11-7.1 set to be updated

--> Finished Dependency Resolution

Dependencies Resolved

================================================================================================

====================================================

Package Arch Version Repository Size

================================================================================================

====================================================

Installing:

oracle-validated x86_64 1.1.0-14.el5 oel5u7 24 k

Installing for dependencies:

compat-db x86_64 4.2.52-5.1 oel5u7 1.6 M

libXp i386 1.0.0-8.1.el5 oel5u7 22 k

libaio-devel i386 0.3.106-5 oel5u7 12 k

libaio-devel x86_64 0.3.106-5 oel5u7 11 k

sysstat x86_64 7.0.2-11.el5 oel5u7 187 k

unixODBC i386 2.2.11-7.1 oel5u7 830 k

unixODBC x86_64 2.2.11-7.1 oel5u7 835 k

unixODBC-devel i386 2.2.11-7.1 oel5u7 743 k

unixODBC-devel x86_64 2.2.11-7.1 oel5u7 799 k

Transaction Summary

====================================================================================================================================================

Install 10 Package(s)

Upgrade 0 Package(s)

Total download size: 5.0 M

Is this ok [y/N]: y

Downloading Packages:

------------------------------------------------------------------------------------------------

----------------------------------------------------

Total 250 MB/s | 5.0 MB 00:00

Running rpm_check_debug

Running Transaction Test

Finished Transaction Test

Transaction Test Succeeded

Running Transaction

Installing : unixODBC 1/10

Installing : sysstat 2/10

Installing : compat-db 3/10

Installing : unixODBC 4/10

Installing : unixODBC-devel 5/10

Installing : unixODBC-devel 6/10

Installing : libaio-devel 7/10

Installing : libaio-devel 8/10

Installing : libXp 9/10

Installing : oracle-validated 10/10

Installed:

oracle-validated.x86_64 0:1.1.0-14.el5

Dependency Installed:

compat-db.x86_64 0:4.2.52-5.1 libXp.i386 0:1.0.0-8.1.el5 libaio-devel.i386 0:0.3.106-5 libaio-

devel.x86_64 0:0.3.106-5

sysstat.x86_64 0:7.0.2-11.el5 unixODBC.i386 0:2.2.11-7.1 unixODBC.x86_64 0:2.2.11-7.1 unixODBC-

devel.i386 0:2.2.11-7.1

unixODBC-devel.x86_64 0:2.2.11-7.1

Complete!

[oracle@tony1 ~]$ ulimit -a;

core file size (blocks, -c) 0

data seg size (kbytes, -d) unlimited

scheduling priority (-e) 0

file size (blocks, -f) unlimited

pending signals (-i) 20010

max locked memory (kbytes, -l) 50000000

max memory size (kbytes, -m) unlimited

open files (-n) 131072

pipe size (512 bytes, -p) 8

POSIX message queues (bytes, -q) 819200

real-time priority (-r) 0

stack size (kbytes, -s) 8192

cpu time (seconds, -t) unlimited

max user processes (-u) 131072

virtual memory (kbytes, -v) unlimited

file locks (-x) unlimited

由于 oracle-validated 在/etc/security/limits.conf 中仅对 oracle 账户进行了设置。

因此我们还要添加相关 grid 账户设置。

# vi /etc/security/limits.conf

# grid-Validated setting for nofile soft limit is 131072

grid soft nofile 131072

# grid-Validated setting for nofile hard limit is 131072

grid hard nofile 131072

# grid-Validated setting for nproc soft limit is 131072

grid soft nproc 131072

# grid-Validated setting for nproc hard limit is 131072

grid hard nproc 131072

# grid-Validated setting for core soft limit is unlimited

grid soft core unlimited

# grid-Validated setting for core hard limit is unlimited

grid hard core unlimited

# grid-Validated setting for memlock soft limit is 50000000

grid soft memlock 50000000

# grid-Validated setting for memlock hard limit is 50000000

grid hard memlock 50000000

克隆虚拟机

1.克隆一台新虚拟机 tony2 作为第二个节点。

2.节点 2设置

01 取消 ISO镜像

启动并登陆 tony2

02 以 root 登陆后修改网络设置：

开启 terminal：

# system-config-network

删除.bak 的 2 行网卡设置。

03 重新编辑 eth1, eth0

04 修正 DNS Hostname: tony2.yellowsand.com

05 Activate eth0 eth1

建立共享磁盘

1.创建共享磁盘

E:\VirtualBox VMs\RACShareDisk\rac_share01.vdi

本文试验建立 8 块共享盘：

点菜单管理 -> 虚拟介质管理，我们需要将这 8 块盘的属性修改为

可共享

这样我们进入 tony2 -> 存储将这 8 块磁盘添加到 tony2 主机下

2.使用 udev 绑定 ASM 磁盘

01 启动并登陆节点(tony1, tony2)检查系统是否看到这些共享盘：

ls -l /dev/sd*

[root@tony1 ~]# ls -l /dev/sd*

brw-r----- 1 root disk 8, 0 Apr 8 2015 /dev/sda

brw-r----- 1 root disk 8, 1 Apr 8 09:55 /dev/sda1

brw-r----- 1 root disk 8, 2 Apr 8 2015 /dev/sda2

brw-r----- 1 root disk 8, 16 Apr 8 2015 /dev/sdb

brw-r----- 1 root disk 8, 32 Apr 8 2015 /dev/sdc

brw-r----- 1 root disk 8, 48 Apr 8 2015 /dev/sdd

brw-r----- 1 root disk 8, 64 Apr 8 2015 /dev/sde

brw-r----- 1 root disk 8, 80 Apr 8 2015 /dev/sdf

brw-r----- 1 root disk 8, 96 Apr 8 2015 /dev/sdg

brw-r----- 1 root disk 8, 112 Apr 8 2015 /dev/sdh

brw-r----- 1 root disk 8, 128 Apr 8 2015 /dev/sdi

02 通过运行一下脚本生成 udev 规则设置(8 块共享盘 sdb~ sdi)：

for i in b c d e f g h i

do

echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id -g -u -s %p\", RESULT==\"`scsi_id -g -u -s /block/sd$i`\", NAME=\"asm-disk$i\",

OWNER=\"grid\", GROUP=\"asmadmin\", MODE=\"0660\""

done

[root@tony1 ~]# for i in b c d e f g h i ;

> do

> echo "KERNEL==\"sd*\", BUS==\"scsi\", PROGRAM==\"/sbin/scsi_id -g -u -s %p\",

RESULT==\"`scsi_id -g -u -s /block/sd$i`\", NAME=\"asm-disk$i\", OWNER=\"grid\",

GROUP=\"asmadmin\", MODE=\"0660\""

> done

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB3b37c5e7-7c762b7f_", NAME="asm-diskb", OWNER="grid",

GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p",

RESULT=="SATA_VBOX_HARDDISK_VB560ec96b-21e08a44_", NAME="asm-diskc", OWNER="grid",



RESULT=="SATA_VBOX_HARDDISK_VB826344d7-4ece5df0_", NAME="asm-diskd", OWNER="grid",



RESULT=="SATA_VBOX_HARDDISK_VB9d5f4648-7305d1ba_", NAME="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBe76e4b36-d0781797_", NAME="asm-diskf", OWNER="grid",



RESULT=="SATA_VBOX_HARDDISK_VB76e71262-1bf7cd15_", NAME="asm-diskg", OWNER="grid",



RESULT=="SATA_VBOX_HARDDISK_VB5ad8c402-fe0ad9cb_", NAME="asm-diskh", OWNER="grid",



RESULT=="SATA_VBOX_HARDDISK_VBe33295ae-accb6c63_", NAME="asm-diski", OWNER="grid",


03 将以上内容添加到 udev rules 文件中:

cd /etc/udev/rules.d

touch 99-oracle-asmdevices.rules

vi 99-oracle-asmdevices.rules

[root@tony1 rules.d]# cat 99-oracle-asmdevices.rules

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB3b37c5e7-7c762b7f_", NAME="asm-diskb", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB560ec96b-21e08a44_", NAME="asm-diskc", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB826344d7-4ece5df0_", NAME="asm-diskd", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB9d5f4648-7305d1ba_", NAME="asm-diske", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBe76e4b36-d0781797_", NAME="asm-diskf", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB76e71262-1bf7cd15_", NAME="asm-diskg", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VB5ad8c402-fe0ad9cb_", NAME="asm-diskh", OWNER="grid", GROUP="asmadmin", MODE="0660"

KERNEL=="sd*", BUS=="scsi", PROGRAM=="/sbin/scsi_id -g -u -s %p", RESULT=="SATA_VBOX_HARDDISK_VBe33295ae-accb6c63_", NAME="asm-diski", OWNER="grid", GROUP="asmadmin", MODE="0660"

[root@tony1 rules.d]#

04 启用此 rule:

start_udev

[root@tony1 rules.d]# start_udev

Starting udev: [ OK ]

05 检查磁盘：

[root@tony1 rules.d]# ls -l /dev/asm*

brw-rw---- 1 grid asmadmin 8, 16 Apr 8 10:01 /dev/asm-diskb

brw-rw---- 1 grid asmadmin 8, 32 Apr 8 10:01 /dev/asm-diskc

brw-rw---- 1 grid asmadmin 8, 48 Apr 8 10:01 /dev/asm-diskd

brw-rw---- 1 grid asmadmin 8, 64 Apr 8 10:01 /dev/asm-diske

brw-rw---- 1 grid asmadmin 8, 80 Apr 8 10:01 /dev/asm-diskf

brw-rw---- 1 grid asmadmin 8, 96 Apr 8 10:01 /dev/asm-diskg

brw-rw---- 1 grid asmadmin 8, 112 Apr 8 10:01 /dev/asm -diskh

brw-rw---- 1 grid asmadmin 8, 128 Apr 8 10:01 /dev/asm -diski

成功后将 parnassusdata1 的实现复制到 2 节点上。

scp 99-oracle-asmdevices.rules tony2:/etc/udev/rules.d

[root@tony1 rules.d]# scp 99-oracle-asmdevices.rules tony2:/etc/udev/rules.d

The authenticity of host 'tony2 (192.168.8.21)' can't be established.

RSA key fingerprint is 87:2b:39:ef:4c:55:80:24:5c:ee:89:59:1d:af:59:d9.

Are you sure you want to continue connecting (yes/no)? yes

Warning: Permanently added 'tony2,192.168.8.21' (RSA) to the list of known hos ts.

root@tony2's password:

99-oracle-asmdevices.rules 100% 1432 1.4KB/s 00:00

[root@tony1 rules.d]#

在 2 节点上重新启动 udev:

start_udev

06 检查 2 节点磁盘：

ls -l /dev/asm*

[root@tony2 ~]# start_udev

Starting udev: [ OK ]

[root@tony2 ~]# ls -l /dev/asm*

brw-rw---- 1 grid asmadmin 8, 16 Apr 8 10:03 /dev/asm-diskb

brw-rw---- 1 grid asmadmin 8, 32 Apr 8 10:03 /dev/asm-diskc

brw-rw---- 1 grid asmadmin 8, 48 Apr 8 10:03 /dev/asm-diskd

brw-rw---- 1 grid asmadmin 8, 64 Apr 8 10:03 /dev/asm-diske

brw-rw---- 1 grid asmadmin 8, 80 Apr 8 10:03 /dev/asm-diskf

brw-rw---- 1 grid asmadmin 8, 96 Apr 8 10:03 /dev/asm-diskg

brw-rw---- 1 grid asmadmin 8, 112 Apr 8 10:03 /dev/asm -diskh

brw-rw---- 1 grid asmadmin 8, 128 Apr 8 10:03 /dev/asm -diski

安装 Grid Infrastructure

1.root 身份登录 tony1 调整显示屏幕大小，防止显示器不能全部包含 OUI界面

2.将安装介质上传到 tony1虚拟主机。

用 grid 用户登陆

解压 GI 安装压缩包：

cd /g01

mkdir 11203_oinstall

cd 11203_oinstall

upzip /soft/p13390677_112040_Linux-x86-64_3of7.zip

3.在 tony1 启动 runInstaller

$ cd /g01/11203_oinstall/grid $ ./runInstaller

01 skip software update, Next

02 选择'Install and Configure Oracle Grid Infrastructure for

a Cluster'

03 选 'Advanced Installation'

04 选择语言: English

05 SCAN Name: scan-cluster.yellowsand.com

取消对'Configure GNS'的勾选

6) 在'Cluster Node Information' 页:

增加 public hostname: tony2.yellowsand.com, virtual hostname:

tony2-vip.yellowsand.com

点'SSH Connectivity...'，建立 1 节点和 2 节点的 ssh 等价联通:

07 确认网络 eth0, eth1 设置, Next：

08 选择 Oracle ASM 来配置 OCR and voting disk

09 创建磁盘组 Disk Group 'GRIDDG' 存放 OCR VoteDisk

Redundancy: Normal (High 需要至少 5 块盘， Normal 至少 3 块)

AU Size: 1MB

Change Discovery Path: /dev/asm*

(/dev/asm-diskb, /dev/asm-diskc, /dev/asm-diskd)

10 设置 SYS 和 ASMSNMP 密码：oracle

11 选择 'Do not use IPMI'.

12 对 OSASM, OSDBA, OSOPER 组进行设置：

13 设置

Oracle Base: /g01/app/grid

Software Location: /g01/app/grid/11.2.0/grid

14 设置 Inventory Directory: /g01 /app/oraInventory

15 进行预安装检查:

# Package: cvuqdis-1.0.9-1 (install it can fix the problem)

# Device Checks for ASM (由于我们使用 udev 而没有用 ASMLib 来做磁盘绑定, 可忽略此错误)

# Task resolv.conf integrity (没有检查到 DNS/GNS 设置, 可忽略此错误)

[root@tony1 rpm]# pwd

/g01/11203_install/grid/rpm

[root@tony1 rpm]# rpm -ivh cvuqdisk-1.0.9-1.rpm

Preparing... ########################################### [100%]

Using default group oinstall to install package

1:cvuqdisk ########################################### [100%]

[root@tony1 rpm]# pwd

/g01/11203_install/grid/rpm

将 cvuqdisk-1.0.9-1.rpm 包传送到 tony2 并安装

[root@tony1 rpm]# ll

total 12

-rwxr-xr-x 1 grid oinstall 8551 Sep 22 2011 cvuqdisk-1.0.9-1.rpm

[root@tony1 rpm]# scp cvuqdisk-1.0.9-1.rpm tony2:/soft/

root@tony2's password:

cvuqdisk-1.0.9-1.rpm 100% 8551 8.4KB/s 00:00

[root@tony1 rpm]#

[root@tony2 ~]# cd /soft/

[root@tony2 soft]# ll

total 12

-rwxr-xr-x 1 root root 8551 Apr 8 15:05 cvuqdisk-1.0.9-1.rpm

[root@tony2 soft]# rpm -ivh cvuqdisk-1.0.9-1.rpm

Preparing... ########################################### [100%]

Using default group oinstall to install package

1:cvuqdisk ########################################### [100%]

重新检查忽略其余告警！

16 概要：

17 安装

18 在每个节点上用 root 用户运行一下脚本:

/g01/app/oraInventory/orainstRoot.sh

/g01/app/11.2.0/grid/root.sh

Tony1

[root@tony1 g01]# /g01/app/oraInventory/orainstRoot.sh

Changing permissions of /g01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /g01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@tony1 g01]#

[root@tony1 g01]# /g01/app/11.2.0/grid/root.sh

Performing root user operation for Oracle 11g

The following environment variables are set as:

ORACLE_OWNER= grid

ORACLE_HOME= /g01/app/11.2.0/grid

Enter the full pathname of the local bin directory: [/usr/local/bin]:

Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...

Entries will be added to the /etc/oratab file as needed by

Database Configuration Assistant when a database is created

Finished running generic part of root script.

Now product-specific root actions will be performed.

Using configuration parameter file: /g01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

OLR initialization - successful

root wallet

root wallet cert

root cert export

peer wallet

profile reader wallet

pa wallet

peer wallet keys

pa wallet keys

peer cert request

pa cert request

peer cert

pa cert

peer root cert TP

profile reader root cert TP

pa root cert TP

peer pa cert TP

pa peer cert TP

profile reader pa cert TP

profile reader peer cert TP

peer user cert

pa user cert

Adding Clusterware entries to inittab

CRS-2672: Attempting to start 'ora.mdnsd' on 'tony1'

CRS-2676: Start of 'ora.mdnsd' on 'tony1' succeeded

CRS-2672: Attempting to start 'ora.gpnpd' on 'tony1'

CRS-2676: Start of 'ora.gpnpd' on 'tony1' succeeded

CRS-2672: Attempting to start 'ora.cssdmonitor' on 'tony1'

CRS-2672: Attempting to start 'ora.gipcd' on 'tony1'

CRS-2676: Start of 'ora.gipcd' on 'tony1' succeeded

CRS-2676: Start of 'ora.cssdmonitor' on 'tony1' succeeded

CRS-2672: Attempting to start 'ora.cssd' on 'tony1'

CRS-2672: Attempting to start 'ora.diskmon' on 'tony1'

CRS-2676: Start of 'ora.diskmon' on 'tony1' succeeded

CRS-2676: Start of 'ora.cssd' on 'tony1' succeeded

ASM created and started successfully.

Disk Group GRIDDG created successfully.

clscfg: -install mode specified

Successfully accumulated necessary OCR keys.

Creating OCR keys for user 'root', privgrp 'root'..

Operation successful.

CRS-4256: Updating the profile

Successful addition of voting disk 77e0b000bb3c4ff5bf19fd8344c5719e.

Successful addition of voting disk 368ed4194dd94f22bf24a89769045fd1.

Successful addition of voting disk 90731ae3a8a34fd0bf7d4ccf694742b7.

Successfully replaced voting disk group with +GRIDDG.

CRS-4256: Updating the profile

CRS-4266: Voting file(s) successfully replaced

## STATE File Universal Id File Name Disk group

-- ----- ----------------- --------- ---------

1. ONLINE 77e0b000bb3c4ff5bf19fd8344c5719e (/dev/asm-diskb) [GRIDDG]

2. ONLINE 368ed4194dd94f22bf24a89769045fd1 (/dev/asm-diskc) [GRIDDG]

3. ONLINE 90731ae3a8a34fd0bf7d4ccf694742b7 (/dev/asm-diskd) [GRIDDG]

Located 3 voting disk(s).

CRS-2672: Attempting to start 'ora.asm' on 'tony1'

CRS-2676: Start of 'ora.asm' on 'tony1' succeeded

CRS-2672: Attempting to start 'ora.GRIDDG.dg' on 'tony1'

CRS-2676: Start of 'ora.GRIDDG.dg' on 'tony1' succeeded

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

Tony2

[root@tony2 g01]# /g01/app/oraInventory/orainstRoot.sh

Changing permissions of /g01/app/oraInventory.

Adding read,write permissions for group.

Removing read,write,execute permissions for world.

Changing groupname of /g01/app/oraInventory to oinstall.

The execution of the script is complete.

[root@tony2 g01]#

[root@tony2 g01]# /g01/app/11.2.0/grid/root.sh



ORACLE_OWNER= grid

ORACLE_HOME= /g01/app/11.2.0/grid


Copying dbhome to /usr/local/bin ...

Copying oraenv to /usr/local/bin ...

Copying coraenv to /usr/local/bin ...

Creating /etc/oratab file...





Using configuration parameter file: /g01/app/11.2.0/grid/crs/install/crsconfig_params

Creating trace directory

User ignored Prerequisites during installation

OLR initialization - successful

Adding Clusterware entries to inittab

CRS-4402: The CSS daemon was started in exclusive mode but found an active CSS daemon on node

tony1, number 1, and is terminating

An active cluster was found during exclusive startup, restarting to join the cluster

Configure Oracle Grid Infrastructure for a Cluster ... succeeded

安装时可以查看一下文件:

#/g01/app/11.2.0/grid/log/tony1/cssd # tail -f ocssd.log

每个节点都执行完后，就可以回到 1 节点，点 OK 继续之后的安装。

SCAN 报错是因为 DNS 未设置的原因。可忽视。

点 Skip, Next.

19 grid 安装结束。

crsctl stat res -t

crsctl query crs activeversion

crsctl check crs

crsctl check cluster

[grid@tony1 ~]$ asmcmd

ASMCMD> lsdg

State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks

Voting_files Name

MOUNTED NORMAL N 512 4096 1048576 15360 14434 5120 4657 0 Y GRIDDG/

ASMCMD> exit

[grid@tony1 ~]$

[grid@tony1 ~]$

[grid@tony1 ~]$ crsctl stat res -t

--------------------------------------------------------------------------------

NAME TARGET STATE SERVER STATE_DETAILS

--------------------------------------------------------------------------------

Local Resources

--------------------------------------------------------------------------------

ora.GRIDDG.dg

ONLINE ONLINE tony1

ONLINE ONLINE tony2

ora.LISTENER.lsnr

ONLINE ONLINE tony1

ONLINE ONLINE tony2

ora.asm

ONLINE ONLINE tony1 Started

ONLINE ONLINE tony2 Started

ora.gsd

OFFLINE OFFLINE tony1

OFFLINE OFFLINE tony2

ora.net1.network

ONLINE ONLINE tony1

ONLINE ONLINE tony2

ora.ons

ONLINE ONLINE tony1

ONLINE ONLINE tony2

--------------------------------------------------------------------------------

Cluster Resources

--------------------------------------------------------------------------------

ora.LISTENER_SCAN1.lsnr

1 ONLINE ONLINE tony1

ora.cvu


ora.oc4j


ora.scan1.vip


ora.tony1.vip


ora.tony2.vip


[grid@tony1 ~]$ crsctl query crs activeversion

Oracle Clusterware active version on the cluster is [11.2.0.3.0]

[grid@tony1 ~]$ crsctl check crs

CRS-4638: Oracle High Availability Services is online

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

[grid@tony1 ~]$ crsctl check cluster

CRS-4537: Cluster Ready Services is online

CRS-4529: Cluster Synchronization Services is online

CRS-4533: Event Manager is online

创建 ASM 磁盘组

重新以 grid 用户登陆以启用最新的 profile 设置，打开 terminal： .

$ asmca

创建'DATA' disk group

External Redundancy (/dev/asm-diske, diskf, diskg)

创建 'BACKUPDG' disk group

External Redundancy (/dev/asm-diskh, diski)

完成：

安装数据库

1.Oracle 数据库软件安装

01 以 oracle 用户重新登陆 1 节点

$ cd /u01

$ mkdir 11203_install

$ cd 11203_install

$unzip /soft/p10404530_112030_Linux-x86-64_1of7.zip

$unzip /soft/p10404530_112030_Linux-x86-64_1of7.zip

[oracle@tony1 11203_install]$ ll

total 4

drwxr-xr-x 8 oracle oinstall 4096 Sep 22 2011 database

[oracle@tony1 11203_install]$ pwd

/u01/11203_install

$ cd database

$ ./runInstaller

进入 Oracle 数据库安装界面后， Security Updates 跳过, Next.

02 跳过更新， Next.

03 仅安装软件'Install database software only'

04 选择 Oracle Real Application Clusters database

installation

05 建立 Oracle 用户节点等价性 - SSH 互通

06 语言选择: English

07 版本选 Enterprise edition

08 设置安装目录：

Oracle Base: /u01/app/oracle

Software Location: /u01/app/oracle/product/11.2.0/db_1

09 设置 OSDBA: dba, OSOPER: oper

忽略以下错误并开始安装：

#Clock Synchronization

#Task resolv.conf Integrity

#Single Client Access Name(SCAN) -> DNS issue

10 在每个节点以 root 用户运行以下脚本.

/u01/app/oracle/product/11.2.0/db_1/root.sh

节点 1：

[root@tony1 db_1]# /u01/app/oracle/product/11.2.0/db_1/root.sh



ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1


The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.





Finished product-specific root actions.

节点 2

[root@tony2 g01]# /u01/app/oracle/product/11.2.0/db_1/root.sh



ORACLE_OWNER= oracle

ORACLE_HOME= /u01/app/oracle/product/11.2.0/db_1


The contents of "dbhome" have not changed. No need to overwrite.

The contents of "oraenv" have not changed. No need to overwrite.

The contents of "coraenv" have not changed. No need to overwrite.





Finished product-specific root actions.

回到安装界面点 OK 后完成。

2.创建 Oracle 数据库

01 重新以 oracle 用户登录 1 节点并启动 dbca:

选择'Oracle Real Application Clusters (RAC) database', Next.

02 选择 Create New Database -> Custom Database

选择 Configuration Type: Admin-Managed (或者 Policy-Managed)

Global Database Name: PROD

03 全选数据节点：

04 默认即可， Next

05 设置账号密码统一为 oracle

06 存储类型

Storage Type: ASM

Storage Locations: +DATA

07 Next 后输入密码并 OK。

08 启用归档并设置归档日志位置：

Enable Archiving -> Archive Log Destination: +BACKUPDG

09 选择组件，组件可按需勾选 (Enterprise mananger Repository

is one MUST)

Memory Size: 1024 MB

Character Sets(all): UTF8

10 Finish 并开始安装：

等待安装结束后，到此所有 RAC环境搭建完成。

参考资料：

http://www.askmaclean.com/

http://www.hellodba.com/