Tips and Tricks in Unix Shell scripting DB2 Database Administration

Speaker: James SobieskiFourth Millennium Technologies

Session Code: D14 -May 26, 2016 8:00 AM – 9 AM| Platform: DB2 for Linux, UNIX and Windows

Photo by Steve from Austin, TX, USA

Agenda

• Scripting Uses

• Template Scripts

• Building Shell Functions

• Common Unix Commands used in scripts

• Awk usage

• Examples

2

Shell Scripting Uses

• Automating DBA Activities

• Scheduling Complex activities via Cron

• Developing a library of “How I Did Something” scripts

• If you are going to do something 2 or more times, script it!

• General Advice• Ensure Scripts are Documented

• Be Consistent – this encourages reuse

• Write re-usable code snippets

• “Borrow generously” from others’ scripts

• This presentation will deal with mainly Korn Shell techniques

• We will review many powerful Unix commands used in shells

3

Templates Script

• Recommend that you create a template script

• Include• Heading Comments

• Name, Description, Usage, Author, Create Date, Mod Date + Description

• Standard Functions

• Help or Usage

• Command Line Parsing

• Exit Cleanup

• DB2 Command post processing

• Common Debugging Code

• Standard Variables containing names of directories for TEMP files, outputs, scripts, etc

• Common scripting structures

4

Temporary Files

• Many scripts use temporary files

• Recommend putting filenames in a variable

• Include the PROGRAM name in the filename

• Include some clue to its use in the filename

• Use “$$” to include the scripts’ Process Id in the file name

TEMP_SQL_OUT=“$TEMP_DIR/$PROGNAME.sql_outfile.$$”

db2 –tvf $MySqlFile >$TEMP_SQL_OUT

RC=$?

Run_SQL_Post_Processing_Function $TEMP_SQL_OUT

rm $TEMP_SQL_OUT

5

Common Shell Variables :

• $0 - the program name, fully qualified

• $* - all the arguments passed on the invoking scripts

• $# - the number of arguments provided on the command line

• $n - “n” is an integer – each successive arg on command line

• $$ - the variable containing the Process Id of the script

6

Common Shell Variables :

• When referencing variables, one can • Test if variable is null and assign a default value. Use the :- construct

DBNAME=${1:-”Sample”}

• Test if the variable is null and print an error message and exitDBNAME=${1?”$usage”}

• Invoking a script with a dot “.” in front of it, runs the script in the caller’s shell. Note that all variables and changes made in the script remain in effect when it returns control. ./.profile

7

Building a Shell Script Funtion:

• Functions have two declaration syntaxes:• function <funcname> {

stmts}

• <funcname> ( parameter_vars) {stmts}

• Both allow parameters passed in• Parameters are provided the same way a script is provided

parms• Variables are global – available to calling script• Local variables can be declared

• <xxx>

• Inside function, $ variables are local

8

A Usage Function:

• Functions need to be defined prior to invoking them

• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }

if [ $# != 2 ]

then Usage

fi

9

Typical Template Variables:

• Variables in many Template Scripts

# Get start time in db2 format

START_TIME=$( date '+%Y-%-m-%d-%:H:%M:%S )

# Scripts, Temp and Output directories: SCRIPT_DIR=/dbamaint/scripts

TEMP_DIR=/TMP/script_temp

OUTPUTS_DIR=$HOME/work/outputs

10

A Usage Function:

• Functions need to be defined prior to invoking them

• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }

if [ $# != 2 ]

then Usage

fi

11

Processing Command Line Options:

• GETOPTS is a function that will search the command line for a character preceded by a minus or plus and allow you to process the next argument following that character:

while getopts i: c

do

case $c in

i) instnme=$OPTARG;;

\?) Usage

exit;;

esac

done

shift `expr $OPTIND - 1`

12

DB2 CLP Command

• The db2 command, a.k.a, Command Line Processor is powerful

• Commonly used options

• adfsf

13

Option Descriptiont Terminators in SQL/Statements so they can be multi-linedf Filename follows it - reads SQL/Statements from the file

m Returns the number of rows modified by the last SQL stmta Returns the SQLCA areas Stops the execution if an Error or Severe Error occursp Toggles prompting for input +p turns off promptc Toggles Autocommit. -c Auto commits after each Stmt. +c does notv Verbose - echos statements to std out as they are executed

The DB2 SQLCA

• When –a is provided on the db2 command the SQLCA is displayed after every statement executed

• Fields in the SQLCA include

14

Option DescriptionSQLCAIDSQLCABCSQLCODE

SQLERRMLSQLERRMC

SQLERRP

SQLERRD1SQLERRD2SQLERRD3SQLERRD4SQLERRD5SQLERRD6SQLWARNSQLSTATE

Post Processing DB2 SQLCA example

15

Function to run a DB2 SQL Statement

• Run_db2 is a function which will run a db2 command and set standard variables for subsequent interrogation

16

AWK Programs and common uses

• AWK is a programming language well suited for parsing, math and command processing

• Common uses include piping output into an Awk program to:• Select the Nth token from the command line:

grep “$( hostname –s ) db2nodes.cfg | awk ‘{ print $1}’

• Find the line number of a string in a fileawk ‘/James Sobieski/ {print NR }’ my_contacts

• db2 get dbm cfg | grep ‘DIAGPATH’ | awk -F"=" '{print $2}'

17

Common Simple GREP usage

• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file

• Common grep options

18

Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched

Common SORT command usage

• SORT is powerful tool to post-process output from a command or a file and deduplicate output

• Common sort options

19

Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched

Common HEAD unix command usage

• Head is powerful tool to pull lines of data from a file or stdoutstarting at the beginning of the file

• Common head options

20

Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched

Common TAIL unix command usage

• Tail is powerful tool to pull lines of data from a file or stdoutreading backwards through the file

• Common tail options

21

Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched

Common XARGS unix command usage

• XARGS is powerful tool which will execute a command substituting variables in positions of the command

• XARGS options

22

Option Description

Common Simple GREP usage

• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file

• Common grep options

23

Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched

Example Snippets – Archiving the DB2 Diag log

• Archiving the db2 diag.log

24

Example Snippets – Post Processing db2pd -hadr

25

Example Snippets – Examining an Application Snapshot

• Snapu script:

26

Example Snippets – making list applications readable

• listu

27

Example Snippets – Displaying and Summarizing Tablespace status

• Show_ts.ksh

28

Example Snippets – Combining System Catalog Tables

• Show_tab_ix

29

Example Snippets – Counting SQL messages in a file

• show_all_sql_msgs.ksh

30

Example Snippets – using AWK to summarize vmstat

• vmstatt.ksh

• iostatt.ksh

31

Example Snippets – Capture Single SQL Execution Stats

• Stmt_execute.ksh

32

Example Snippets – Run an explain on SQL statement

• Jexpln.ksh

33

Example Snippets – Capture runaway Applheapsz

• Capture_applheapsz_too_large.ksh

34

Example Snippets – Add Partition to RP table

• Add_next_partition.ksh

35

Example Snippets – Convert Snapshot cmd output to script

• Gen_all_ss_variables.ksh

36

Example Snippets – Generate Views for Catalog Tables

• make_one_shortcat.ksh

37

Example Snippets – Generate Views for Catalog Tables

• make_all_shortcats.ksh

38

Example Snippets – Report on backups

• Report_all_backups_taken.ksh

39

Insert Name HereInsert Company Name HereInsert E-mail here

SessionTitle

Please fill out your session

evaluation before leaving!

Photo by Steve from Austin, TX, USA