51

42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

Embed Size (px)

Citation preview

Page 1: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC
Page 2: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 2/51

42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RACMartin Klier

Performing Databases GmbHMitterteich / Germany

Page 3: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 3/51

Speaker

● Martin Klier● Solution Architect and

Database Expert

● My focus– Performance Optimization– High Availability– Architecture DBMS

● Linux since 1997● Oracle Database since 2003

Page 4: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 4/51

Performing Databases

● Experts for the Oracle Database

– Concept– Planning & Sizing– Licensing– Implementation and Troubleshooting

● Get in touch

– Performing Databases GmbHWiesauer Straße 2795666 Mitterteich, GERMANY

– Web: http://www.performing-databases.com– Twitter: @PerformingDB

Page 5: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 5/51

● Design

● Licensing

● Implementation

● Tuning

● Troubleshooting

● Service

● Upgrade

● Migration

International

http://www.performing-databases.com

Page 6: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 6/51

See you...?

● Weblog: http://www.usn-it.de (English)● Contact: [email protected]

Save the date: Nov 21st - 24th 20172000+ Oracle ProfessionalsCall for papers opens soon!

http://www.doag.org

Page 7: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 7/51

Tally-Ho!

Page 8: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 8/51

Are you heroes or wimps?

Wimps, but of the HARD kind!

Page 9: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 9/51

Basics

Page 10: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 10/51

Database

Database

Instance

Page 11: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 11/51

RAC

Page 12: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 12/51

RACReal Application Clusters= Product

Real Application Cluster= Grid Infrastructure + Database

w/ multiple (possible) instances

Database

Instance Instance Instance

Page 13: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 13/51

RACShared-Everything-Architektur(all nodes see all data)

Queries will return same result on all nodes

CacheFusion (US Patent US20060117074 A1 by Ahmet K. Ezzat)

creates virtual global cache(but on demand => Block Shipping)

Parsing singleton per node (Scalability!)Execution plans may differ - careful!

Page 14: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 14/51

Cache Fusion

Tablespace(s)

Process

SGA

SessionNode 1

GCS

RedoThread

1

RedoThread

2

Shared Storage

Process

SGA

SessionNode 2

GCS

Interconnect

Page 15: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 15/51

Cache FusionThree ways to access a block/buffer

Local Cache HitRemote Cache HitDisk Access

Global Ressource Directory(updated by lightweight msg. )

Page 16: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 16/51

Cache FusionDynamic Data Retrieval

Picture: Courtesy of Markus Michalewicz, Oracle

Page 17: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 17/51

Cache FusionDynamic Mastering / Remastering

Picture: Courtesy of Markus Michalewicz, Oracle

Page 18: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 18/51

Cache FusionDynamic Mastering / Remastering

Picture: Courtesy of Markus Michalewicz, Oracle

Page 19: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 22/51

ListenerSCAN Listener (11.2)(Single Cluster Access Name)

Standard: 3x SCANMy tip: 1x sufficient for 95%

Node-Listener

X

NL

Node 1 Node 2

NL

SCAN L

Page 20: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 23/51

RAC PerformanceRAC = Amplifier :)

Sequencesorderedcached

1 Query not „larger“ than 1 node

Avoid saturation (= latency degradation)of the interconnect

OP

Page 21: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 24/51

RAC Performance

Tablespace(s)

Session

SGA

SessionNode 1

RedoThread

1

RedoThread

2

Shared Storage

Session

SGA

SessionNode 2

Interconnect

Page 22: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 25/51

MiscellaneousAuto-DOP (Degree of Parallelism) in RAC=> Distribution PX worker over the nodesOf interest for InMemory w/o Exa!

InMemory IMCUs w/o Exa NOT redundantReason: Implemented with RDMA=> Only works w/ Infiniband(Mellanox requests defined environment)

Page 23: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 26/51

ASM

Page 24: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 27/51

ASMAutomatic Storage Management= Product

Logical Volume Manager

Software-RAID

Cluster - Filesystem

Cluster - LVM

Page 25: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 28/51

ASMCombines advantages of Raw Devices (small overhead)and file systems (simplified administration)

DB will access block device directly

Bad reputation: „I want my datafiles under my control!“

„Humane“ Administrationvia multiple UIs(SQL*plus, asmcmd -p, EM CloudControl)

Page 26: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 29/51

Classic I

Node

Storage

Node

Page 27: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 30/51

Classic II

Node

Storage Storage

Page 28: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 31/51

Classic III

Node

Storage

Node

Storage

Page 29: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 32/51

ASM++

Node

Storage

Node

Storage

Page 30: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 33/51

ASM Diskgroups

Disk 2

Disk 1

DiskgroupRedundancy „EXTERNAL“

Use SAME SIZE for Disk 1 and 2!

AllocationUnit

min. 1MB

Page 31: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 34/51

ASM Diskgroups

Disk 2Disk 1

DiskgroupRedundancy „NORMAL“

Page 32: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 35/51

Disk 4Disk 3

ASM Diskgroups

Disk 2Disk 1

DiskgroupRedundancy „NORMAL“

FailGroup0000

FailGroup0001

Page 33: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 36/51

ASMPreferred Read Node (11g)allows „local first“

Even Reads (12c)reads evenly from all mirrors of the DG

Instance 1

Disk 1

Instance 2

Disk 2

Red. NORMAL

Page 34: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 37/51

ASM ExperienceNo data loss in >12 years(But it was a close shave ...)

Biggest problem in 10.xShort-time missing LUNs not dropped + rebalanced

„Solution“ in 11.2Improved eventDISK_REPAIR_TIME (drop delay)

Biggest operational challengeReturning LUNs request manual intervention=> Monitoring!

Page 35: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 38/51

ASM 12.2Flex Diskgroup

allows grouping by database within DGallows Split-Mirror backup by database/tag within DG

ASM Filter DriverProtection against „hostile“ IO

ASM Servicecentalized management of Shared Disks for all RACsin a domainCentral Shared Disk Service („ASM via TNS“ = NAS)will come with a patch set

Page 36: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 39/51

Clusterware

Page 37: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 40/51

Oracle Clusterwarealias OCWGrid Infrastructure (GI)Cluster Ready Services (crs)

Clusterware, e.g. Failover of services

APIs and infrastructure for Database (CacheFusion, …)ASM (FlexASM, …)ACFS...

Instance Instance

Page 38: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 41/51

OCW ServicesOHASD - Oracle High Availability Service DeamonStarting and observing cluster processesInterface to service mgmt of the OS

EVM (evmd) - Event ManagerIn- and outgoing messages between nodes andservices, mostly used OCW-internal(think: Message Bus)

Page 39: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 42/51

OCW ServicesCSS (ocssd) - Cluster Synchronization ServiceManaging member nodes / availabilityDisk/Network HeartbeatLogging: ocssd.log / trc (12c)

CRS (crsd) - Cluster Ready ServiceControlling the „Userland“ Services(like DB, Listener)Logging: crsd.log / trc (12c)

OraRootAgent - privileged servicesOraAgent - other services(Danger: both names also used for OHASD comp.)Agent Logs show application output

Page 40: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 43/51

H.A.Heartbeat Node Fencing

Up to 12.1: Lowest node number(or larger cluster fragment)12.2: Lowest No. of services or Prio-Service

Split-Brain-Problem

Quorum: 1,3,5,7 …

Instance 1 Instance 2

Page 41: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 44/51

Page 42: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 45/51

Planning vs. TroubleInfrastructure for time sync MANDATORY(Node Evictions) - ntpd, chrony...

if NTP, use -x / Chrony has similar optionLast resort: CTSS not in observer mode=> (sync Node <-> Node)

Interconnect Redundant (no Bonding needed after 11.2.0.2)minimize and stabilize latencyuse Jumbo FramesMulticastIsolate RACs!!

Page 43: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 46/51

Cool & CLIsrvctl -h | grep xyz

Do it with …. srvctl !ALWAYS start utils from GRID-Home! ($OH+$PATH)No ADD for Diskgroup Services→ Auto create at first mount ;)Full support for Dataguard (w/ DG Broker)

crsctl status resource -t

crsctl -unsupported(e.g. to remove DB Service w/o $OH)

Page 44: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 47/51

OCW & SE*Use Grid Infrastructure as clusterware for FailoverCluster - DB Failover w/o RAC One Node!

So no need to link DB RAC Binaries=> Standard Edition, SE1, SE2=> 10-Day-Rule possible (max. 1 Storage)

Create 1-Node-Serverpools

Change DB Service for Single Instance toServer_pools=* (crsctl -unsupported, but works!)

Page 45: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 48/51

Non-RAC

Page 46: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 49/51

Did ya' know?$TWO_TASK concats itself with @ to sqlplus

Customized SQL prompt (glogin.sql) interferes withutilities (like dbca, asmca)+SQL+> -- this prompt did cost me dearly

Never end your ORACLE_HOME with /(hash compare fails)

No hyphen advised in DB_DOMAIN (various known DB Link problems)

Page 47: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 50/51

RMAN & ServiceRMAN restore from service name (12c)

RMAN> restore datafile 5 from service STANDBY;

Works well with Datafiles, Control Files

Does NOT with Archived Redo Logs :(

Page 48: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 51/51

EnvironmentOracle says, use different OS User Grid vs. DB

My tip: Aviod if possible.Write (or copy&understand) very good env scripts

Suggestion: /home/oracle/bin

Set environment and shell promptlike [oracle@myhost ~] (MYSID) $

Source with logon (.yourshell_profile)or manually: . db / . grid

Page 49: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 52/51

Page 50: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC

@MartinKlierDBA 42 Facts for Oracle RAC 53/51

Q & A

Download my Presentations and Whitepapershttp://www.performing-databases.com

Page 51: 42 (alternative?) Facts for Oracle Grid Infrastructure ... · PDF file@MartinKlierDBA 42 Facts for Oracle RAC 2/51 42 (alternative?) Facts for Oracle Grid Infrastructure, ASM and RAC