ASM Tipps and Tricks · •ASM Provides •Volume Manager •Cluster File System (ACFS) with Replication •Great for Oracle Database •One of the Best Pieces of Software 6 ASM is

ASM Tipps and Tricks

Ernst LeberDüsseldorf im Mai 2016

Facts & Figures

Technologie-orientiertBranchen-unabhängig

HauptsitzRatingen

240 Beschäftigte

Gründung1994

NiederlassungFrankfurt am Main

Ausbildungs-betrieb

Inhabergeführt

Zertifizierter Partner von

Oracle, Microsoftund SAP

24 Mio. Euro Umsatz

2

▪ What is ASM▪ Tools

▪ kfed

▪ kfod

▪ amdu

▪ asmcmd

▪ Adding a LUN causes a Desaster ?!▪ Diskgroup is full but no Files stored?

3

Overview and Contents


SQL> select open_mode from v$database; select open_mode from v$database * ERROR at line 1: ORA-01507: database not mounted

SQL> select table_name from dict order by 1; ORA-01219: database or pluggable database not open: queries allowed on fixed tables or views only 01219. 00000 - "database or pluggable database not open: queries allowed on fixed tables or views only“

SQL> select instance_role from v$instance;

INSTANCE_ROLE ------------------ UNKNOWN

show parameter instance_type instance_type string ASM

4

ASM is no Database


https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrl-state=rgt39jvq9_4&_afrLoop=542948295068059

5

ASM is no Database but has Database Processes


Process DecriptionASMB Communicates with the ASM instance, managing storage and providing statistics

RBAL ASM Rebalance Master Process

ARBn ASM Rebalance Process

DBWR Manages ASM buffer cache

SMON System monitor and communication to CSSD

CKPT Manages cross instance calls in cluster setup

PSP0 Spawns other ASM background processes after initial instance startup

GMON Disk Group Monitor Process monitors all the disk groups mounted in an ASM instance

MARK Marks ASM allocation units as stale following a missed write to an offline disk

VKTM Virtual Keeper of Time Process

LGWR Log Writer Process Copies logging information to an ASM diskgroupOnnn ASM Connection Pool Process

https://support.oracle.com/epmos/faces/SearchDocDisplay?_adf.ctrl-state=rgt39jvq9_4&_afrLoop=542948295068059

ASM has its own Databasewriter DBWR

Data are written by Databases only • No Access for ASM DBWR to buffer cache in SGA of Databases

• ASM Provides • Volume Manager • Cluster File System (ACFS) with Replication • Great for Oracle Database • One of the Best Pieces of Software

6

ASM is no Database


Where to store information about Diskgroups and LUNs?

• The only way to store is in LUN Header • Persistent across Servers • Reliable unless you use dd • Beginning with Oracle 11 header data is stored twice

• in LUN Header • At the End of the LUN

• Is there a chance to Read this information?

7

ASM is no Database


[oracle@server1 ~]$ asmcmd -p

ASMCMD> lsdg State Type Rebal Sector Block AU Total_MB Free_MB Req_mir_free_MB Usable_file_MB Offline_disks Voting_files Name MOUNTED EXTERN N 512 4096 4194304 5112 3480 0 3480 0 N DATA/ MOUNTED EXTERN N 512 4096 4194304 5112 5032 0 5032 0 N FRA/ MOUNTED EXTERN N 512 4096 4194304 1016 928 0 928 0 N SYSTEM/

startup / shutdown

8

asmcmd


lsdsk -G DATA lsdsk -k -G DATA

ASMCMD> lsdsk -k -G DG_ACFS Total_MB Free_MB OS_MB Name Failgroup Failgroup_Type Library Label UDID Product Redund Path 248 140 0 _DROPPED_0004_DG_ACFS DG_ACFS_0002 REGULAR System UNKNOWN 248 4 250 DG_ACFS_0000 DG_ACFS_0000 REGULAR System UNKNOWN /dev/acfs-disk1 248 4 250 DG_ACFS_0001 DG_ACFS_0001 REGULAR System UNKNOWN /dev/acfs-disk2 248 0 250 DG_ACFS_0003 DG_ACFS_0003 REGULAR System UNKNOWN /dev/acfs-disk4 ASMCMD> exit

lsdsk --candidate lsdsk -k --candidate

9

asmcmd


ASMCMD [+] > lsod [-H] [-G diskgroup] [--process process_name] [pattern]

lsod -G data --process LGWR Instance Process OSPID Path 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska1 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska2 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska3 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diskb1 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diskb2 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diskb3 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diskd1

The second example lists the open devices associated with the LGWR process for disks that match the diska pattern. ASMCMD [+] > lsod --process LGWR diska Instance Process OSPID Path 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska1 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska2 1 oracle@dadvmn0652 (LGWR) 26593 /devices/diska3

10

asmcmd


[oracle@server1 sf_DB11Gr2]$ kfod -------------------------------------------------------------------------------- Disk Size Path User Group ================================================================================ 1: 5114 Mb /dev/asm-disk4 oracle dba -------------------------------------------------------------------------------- ORACLE_SID ORACLE_HOME ================================================================================ +ASM /u01/app/oracle/product/11.2.0.4/grid [oracle@server1 sf_DB11Gr2]$

kfod help

kfod disks=raw,asm,badsize,all

kfod asm_diskstring=/dev/a* disk=all dscvgroup=TRUE status=TRUE verbose=TRUE

11

kfod Kernel Files OSM Disk


oracle@ exadata > kfed read /dev/asm-disk2 |more kfbh.endian: 1 ; 0x000: 0x01 kfbh.hard: 130 ; 0x001: 0x82 kfbh.type: 1 ; 0x002: KFBTYP_DISKHEAD kfbh.datfmt: 1 ; 0x003: 0x01 kfbh.block.blk: 0 ; 0x004: blk=0 kfbh.block.obj: 2147483648 ; 0x008: disk=0 kfbh.check: 3439698229 ; 0x00c: 0xcd05a135 … kfdhdb.driver.provstr: ORCLDISK ; 0x000: length=8 … kfdhdb.compat: 202375168 ; 0x020: 0x0c100000 kfdhdb.dsknum: 0 ; 0x024: 0x0000 kfdhdb.grptyp: 1 ; 0x026: KFDGTP_EXTERNAL kfdhdb.hdrsts: 3 ; 0x027: KFDHDR_MEMBER kfdhdb.dskname: DATA_0000 ; 0x028: length=9 kfdhdb.grpname: DATA ; 0x048: length=4 kfdhdb.fgname: DATA_0000 ; 0x068: length=9 kfdhdb.capname: ; 0x088: length=0

12

kfed Kernel Files metadata Editor


amdu -diskstring '/dev/asm-disk*'-*-amdu-*-

******************************* AMDU Settings ******************************** ORACLE_HOME = /u01/app/oracle/product/12.1.0.2/grid System name: Linux Node name: exadata Release: 2.6.39-400.17.1.el6uek.x86_64 Version: #1 SMP Fri Feb 22 18:16:18 PST 2013 Machine: x86_64 amdu run: 08-MAR-16 15:29:19 Endianess: 1

--------------------------------- Operations ---------------------------------

------------------------------- Disk Selection ------------------------------- -diskstring '/dev/asm-disk*'

------------------------------ Reading Control -------------------------------

------------------------------- Output Control -------------------------------

********************************* DISCOVERY **********************************

----------------------------- DISK REPORT N0001 ------------------------------ Disk Path: /dev/asm-disk1

13

amdu Report Disks


amdu -help

al/lides Dump indirect blks unconditionally -allides: AMDU ordinarily skips over empty indirect blocks. Specifying this option tells AMDU to dump those blocks unconditionally. Be warned that this can make the resulting AMDU dump quite large.

au/size AU size for corrupt disks -ausize <bytes>: This option must be set when -baddisks is set. It must be a power of 2. This size is required to scan a disk looking for metadata, and it is normally read from the disk header. The value applies to all disks that do not have a valid header. The value from the disk header will be used if a valid header is found. …

…

14

amdu ASM Metadata Dump Utility


ASMCMD> pwd +data/orcl/parameterfile ASMCMD> ls -ls

Type Redund Striped Time Sys Name PARAMETERFILE UNPROT COARSE MAR 08 14:00:00 Y none => spfile.257.904570929 ASMCMD>

oracle@ exadata > amdu -dis '/dev/asm-disk*' -extra DATA.257 -spfile amdu_2016_03_08_15_41_27/ AMDU-00204: Disk N0002 is in currently mounted diskgroup DATA. AMDU-00201: Disk N0002: '/dev/asm-disk2' oracle@ exadata > cd amdu_2016_03_08_15_41_27 oracle@ exadata > cd amdu_2016_03_08_15_41_27 oracle@ exadata > ls -al insgesamt 44 drwxr-xr-x 2 oracle oinstall 4096 8. Mär 15:41 . drwxr-xr-x 4 oracle oinstall 24576 8. Mär 15:41 .. -rw-r--r-- 1 oracle oinstall 3584 8. Mär 15:41 DATA_257.f -rw-r--r-- 1 oracle oinstall 5895 8. Mär 15:41 report.txt oracle@ exadata >

15

amdu Extract Files


16

Adding a LUN causes a Desaster ?!


/dev/asm-disk3

/dev/asm-disk4

•Create a LUN on a Database Server •Create Diskgroup DATA using this LUN •Perform Testing •Once you are finished simply stop the test machine •Hey, I can use this LUN in Production

•Make the LUN known to your production Server •Restart ASM •Diskgroup DATA will not mount anymore!

•Fix: Identify the last LUN you added •dd if=/dev/zero of=<the LUN> bs=1024 count=1 •alter disk group data mount •Fixed!!

•Always Drop a Diskgroup you do not need anymore •ASM Reads the LUN headers •IF more than one LUN with the same name in Header

•The last LUN Wins

17

Adding a LUN causes a Desaster ?!


[grid@oda-ww-01 ~]$ asmcmd -p ASMCMD [+] > lsdg State Type ... Total_MB Free_MB Req_mir_free_MB Usable_file_MB Name MOUNTED NORMAL... 52428736 22696212 3276796 9709708 DATA/ MOUNTED NORMAL... 1526208 40904 381552 -170324 FLASH/ MOUNTED NORMAL... 8618304 2911784 538644 1186570 RECO/ MOUNTED HIGH ... 763120 110824 381560 -90245 REDO/ ASMCMD [+] >

ASMCMD [+] > du REDO ASMCMD [+] > du DATA Used_MB Mirror_used_MB 6432 12896

18

Diskgroup is full but no Files in Diskgroup????


ASMCMD [+] > volinfo --all

Diskgroup Name: REDO

Volume Name: ACLDATSTORE Volume Device: /dev/asm/acldatstore-394 State: ENABLED Size (MB): 49152 Resize Unit (MB): 64 …… Usage: ACFS Mountpath: …… Volume Name: DATASTORE Volume Device: /dev/asm/datastore-394 State: ENABLED Size (MB): 163840 Resize Unit (MB): 64 Redundancy: HIGH Stripe Columns: 8 Stripe Width (K): 1024 Usage: ACFS Mountpath: /u01/app/oracle/oradata/datastore ASMCMD [+] >

19

Diskgroup is full but no Files in Diskgroup????


▪ ASM tools used by Support : KFOD, KFED, AMDU (Doc ID 1485597.1)▪ http://de.slideshare.net/AmitBhalla2/less05-asm-instance

▪ Server für diese Demos unter:▪ http://www.mt-ag.com/doagdb/

20

Links


www.mt-ag.com https://eleoracle.wordpress.com @eletwit

|

Telefon: 02102 30961 0 Telefax: 02102 30961 101

E-Mail: [email protected]

Vielen Dank Ernst Leber

Documents

ASM Tipps and Tricks · •ASM Provides •Volume Manager •Cluster File System (ACFS) with Replication •Great for Oracle Database •One of the Best Pieces of Software 6 ASM is