1 ITEC 400 Shell (Part 2) George Vaughan Franklin University

1

ITEC 400Shell (Part 2)

George VaughanFranklin University

2

Topics

• VNC

• Root

• Special Characters in Shell

• Useful Unix Commands for Scripting

• computations in Shell Scripts

• Flow Control in Shell Scripts* Note: Your Sys Admin book has a nice description of shell

programming in the appendix

3

VNC

• VNC is a free, handy tool for remotely managing Windows, Linux or Unix machines

• Virtual Network Computing• URL: http://www.uk.research.att.com/vnc/• Allows one to administer the Desktops of many

computers on one computer.• Example: I can manage my Linux and Windows

desktops (hence the machines themselves) from my laptop.

http://www.uk.research.att.com/vnc/

4

ROOT

• ROOT is an overloaded term in Unix

• ROOT user is the super-user. Root can see and do anything. (You must be root to add new users to the system)

• ROOT directory is the parent of all UNIX directories (/tmp is under the root directory ‘/’)

5

Special Shell Characters

• The Dot “.” Command– The Dot Command allows you to run a script in the

current shell (i.e. a new shell is not created).– Note: using the dot command to run a script can

affect the current shell.– Example (using 2 telnet sessions):$ ps -u gvaughan PID TTY TIME CMD 1434 pts/0 00:00:00 bash <= Login shell from telnet #112594 pts/2 00:00:00 bash <= Login shell from telnet #212668 pts/2 00:00:00 sleep <= sleep from script on telnet #212669 pts/0 00:00:00 ps <= ps on telnet #1

6


• “\” Line Continuation Character– allows one logical line in shell to fit on more than one

physical line in a file– Example:

ls | \

wc –l

• will return a count of the files in the current directory.

• This is equivalent to: ls | wc -l• Useful for long source lines - improves readability

7


• “;” Command Separator– Used to separate multiple commands on same

physical line– Example:

cd bin; ls• This is equivalent to:

cd binls

– Commands separated by semi-colon are executed serially. How is this different from the pipe?

– 2 commands separated by a semi-colon are still executed in 2 distinct processes.

8


• “(…)” Sub Shell Construct– Allows one or more commands to be executed in a

sub shell.– Behavior is similar to putting commands in a script.– Useful to protect current shell form side effects.– Useful for redirecting output from many commands– Examples:

cd $HOME; (cd /bin; pwd); pwd

(ls; pwd; ls) > trash #send output of all 3 commands to trash

(ls; pwd; ls) & #run all cmds in background

9

Special Shell Characters• Example:

>date; psMon May 5 21:16:38 EDT 2003 PID TTY TIME CMD12325 pts/1 00:00:00 bash12367 pts/1 00:00:00 ps– Why don’t we see ‘date’ in the ‘ps’ report?

• Example:>(date; ps)Mon May 5 21:15:57 EDT 2003 PID TTY TIME CMD12325 pts/1 00:00:00 bash12361 pts/1 00:00:00 bash12363 pts/1 00:00:00 ps– Notice the 2 instances of bash

10


• “&” Run command in background– Will run command in background.– Example (with 1 telnet session):

>test01 &

[1] 12820 <= process ID of script in background

>ps

PID TTY TIME CMD

1434 pts/0 00:00:00 bash <= login shell

12820 pts/0 00:00:00 bash <= script shell

12821 pts/0 00:00:00 sleep

12822 pts/0 00:00:00 ps

11


• “|” Pipe– Take the output of command 1 and use it as

the input of command 2– Example:

ps -e | wc -l• the ‘ps’ and ‘wc’ commands are running

concurrently in 2 distinct processes.

– You can pipe multiple commands:who | awk '{print $1 }' | sort -u | wc -l

• This reports the number of unique active logins on the system

12


• “>” Redirect Output– Sends output of command to file.– Will overwrite file– Examples:

ls > trash

(date;ps) > trash

• “>>” Append Output– Appends output of command to end of file.– Example:

ls >> trash

13


• “<“ Redirect input– Useful for reading command input from a file.– Example:

• Use redirection to automatically edit a bunch of files.

• Assume we have a file named control.vi• Inside the file control.vi we have the following vi

commands (change bsh to ksh, write file, quit)::g/bsh/s//ksh

:wq

14


• Redirection Example (continued)– We can now use the control.vi file to

automatically edit another file such as myscript:

vi myscript < control.vi

– We can imagine how we might generalize this using a shell script with loops…

– The sed command is very useful for this purpose (we will see sed described later…)

15


– We can also use input redirection to generate a list as arguments for a command.

– Example:• Assume we have a file “trash” with the following

contents:/etc

/bin

• We could use the file “trash” as input to “ls”:ls $(<trash) #(equivalent to ls /bin /etc)

• ls will list the contents of /etc and /bin.

16


• “.” Current Directory (not the same as the dot command)– “.” is a shortcut for referencing the current directory– Example:

./myApp #execute the program myApp which exists

#in the current directory.

• “..” Parent Directory– “..” is a shortcut for referencing the parent directory– Example:

../myApp #execute the program myApp which exists #in the directory above.

17

Useful Unix Commands for Scripting

• Unix has many commands – some are especially useful for scripts.

• The following commands are feature rich – check man page.

18


• grep – prints any line in the input stream that contains a specified substring– When filtering output, think of grep as tool to make

horizontal cuts (line selection)– Supports regular expressions with –e option– Example: Show me all of my processes:

ps –ef | grep $LOGNAME

– Useful for filtering out items of interest– “grep -v string” will match every line that doesn’t

contain the string.– grep -i string (ignores case)

19


• awk, nawk – a filter like scripting language– When filtering output, think of ‘awk’ as tool to make

horizontal cuts (line selection) and vertical cuts (field selection).

– Quite powerful (it is a programming language)– Can be used to select and/or rearrange fields in input– Example:

• ps –ef shows the following (only subset is shown):> ps -ef | more

UID PID PPID C STIME TTY TIME CMD root 0 0 0 Dec 13 ? 0:01 sched root 1 0 0 Dec 13 ? 1:12 /etc/init - root 2 0 0 Dec 13 ? 0:00 pageout root 3 0 1 Dec 13 ? 529:56 fsflush

20


• awk, nawk (Cont.):– Assume we only want to see the command and the

owner, in that order:>ps –ef | awk '{ print $9, $1 }'sched root/etc/init rootpageout rootfsflush root/usr/lib/utmpd root/usr/sbin/keyserv root/usr/lib/saf/sac root/usr/lib/devfsadm/devfseventd root

– Notice the use of single quotes in AWK script (why?)

21


• awk, nawk (Cont.):• Another example:

– Each line in the password file (/etc/passwd) is variable length and delimited by a colon ‘:’

pangh:x:118726:100:Hanbin Pang:/export/home/pangh:/bin/ksh (subset of file)

– If I want to retrieve the user’s ID and name (field positions 1 and 5), I can use awk:

>cat /etc/passwd | awk '{ FS=":"; print ($1, $5) }'vaughang George Vaughan (subset of output)

– ‘FS’ in the example above is used to inform awk what character(s) are used as field separators (space is the default)

22


• cut- When filtering output, think of ‘cut’ as tool to make vertical cuts (column selection).

• When can make vertical cuts based on character position or by field (awk is better for this task).

23


• Example of ‘cut’ by character position:– The command ‘ls –l’ can be used to show everyone's permissions to a

file:>ls -ltotal 8-rwxr-xr-x 1 vaughang faculty 415 Sep 6 09:32 dirsize-rwxr-xr-x 1 vaughang faculty 151 Sep 9 21:27 ex0280drwxr-xr-x 2 vaughang faculty 512 Sep 6 00:06 junk-rw-r--r-- 1 vaughang faculty 20 Sep 6 00:07 trash2

– We can use cut based on character position to just display permissions for owner (info in columns 2 through 4):>ls -l | grep -v total | cut -c2-4rwxrwxrwxrw-

– In the example above, we used ‘grep –v’ to strip out the first line of ‘ls –l’ which contains information about the total number of disk blocks used by directory.

24


• Example of ‘cut’ by field position:– Each line in the password file (/etc/passwd) is variable

length and delimited by a colon ‘:’pangh:x:118726:100:Hanbin Pang:/export/home/pangh:/bin/ksh (subset of file)

– If I want to retrieve the user’s ID and name (field positions 1 and 5), I can use cut:>cat /etc/passwd | cut -f1,5 –d’:’vaughang:George Vaughan (subset of output)

– The ‘-d’ option is used to tell ‘cut’ which character (or characters) serve as the delimiter (TAB is the default).

– Note: awk is a more powerful tool for making vertical cuts based on fields.

25


• sort – sorts input.– can sort on any column (defaults to column

1). – the –u option will only show unique rows.– Example:

• Count the number of active processes:ps -ef | wc –l

• Count the number of unique process owners:ps -ef | awk '{ print $1 }' | sort -u | wc -l

26


• sed – stream editor– Used to edit input on the fly, powerful– Examples:

• Don’t list processes owned by root:ps -ef | sed '/root/d'

• Strip out blank lines:cat trash | sed '/^$/d'

• Change each comma to a blank, globallycat trash | sed 's/,/ /g'

27


• find – recursively search directories for files of interest– Book provides a good overview– Example:

• Count all files in the system that end in .htmlfind / –name *.html –print | wc –l

• date – print date and time– contains useful format control useful for creating file

suffices:– Example:

• Save a listing of my files as of todayFILE_NAME=file_list_`date '+%m%d%y'`

• Produces file name like: file_list_011203

28


• diff – show differences between 2 ASCII (text) files.– Output is cryptic, can be used to convert one file to

another, allows SCCS to store deltas.– If files are the same, no output.– Examplediff file_1 file_2 | wc –l #if zero, files are same

• cat – dump the contents of a file to standard out:– Example:

• Count the number of registered users on system by counting entries in password file:

cat /etc/passwd | wc -l

29


• touch – change file timestamps.– useful for ‘marking a file’.– If file doesn’t exist, it is created with zero

length.– ‘touched’ file can serve as a record when

something happened.– Example:

• touch the file ‘last_backup’ when backup completes:

touch last_backup

30


• basename – strip of the path from a fully qualified file name.– Commands like ‘find’ produce lists of files with

pre-appended paths. Use basename if you want just the filename.

– Example:echo `basename vaughang/itec400/lecture_notes/notes1.pp`

• produces the output ‘notes1.pp’

31


• which – tells you which copy of command you will execute if you execute the command.– Determined by your PATH variable.– Useful when you think you are using one copy

when in fact you are using another copy.– Example:

which ls

– produce the following result: /bin/ls

32


• set –x– Used to turn on tracing in shell script.– Useful for debugging

• set +x– Turns off shell tracing

• Example:set –xdateset +xdate

33


• exit – terminate script and return integer for return code

• Example:exit 2– Will cause the script to terminate with a return code of

2.– Useful for error handling by other scripts.– Which shell variable will contain the return code from

exit (see notes from previous lecture)?

34


• shift– The shift command will shift all command line

arguments to the left by 1.– So, after using shift, $1=$2, $2=$3, $3=$4,…– shift can be used in a loop.

35

How Script Can Change Parent Shell Variables (“.” Command)

• By default, shell variables are local to script.

• How can we make a shell script change contents of parent shell?

• With the “.” command:– example: . myScript

• myScript will change variables in parent shell• why?

36

How Parent Shell Can Initialize Shell Variables In Script (“export” command)

• When we use the export command on a shell variable, it becomes visible to subshells and scripts.

• Exported variable is something like a global variable.

• In the example below, $MY_VAR is set to 23 and then exported. $MY_VAR will be visible inside myScript

export MY_VAR=23

myScript

37

Using “.” and “export” commands together

• If we use a “.” command on a script and inside the script we “export” variables, then exported variables will not only be visible in parent shell but also in all future subshells.

• Example: PATH variable in .profile• Notice that we always execute .profile with

the “.” command (what would happen if we didn’t?)

38

A Note On Examples• Source for all examples in this and future lectures are on codd (einstein) at:

/export/home/vaughang/public_html/itec400/examples• and on the web at:

– http://cs.franklin.edu/~vaughang/itec400/examples/• All shell examples are written in Korn Shell (ksh) although they should be

quite compatible with Bash.• In all examples, the ‘greater-than’ symbol, ‘>’ will be used to indicate the

shell prompt. So, if we execute the date command,>dateThu Sep 2 20:43:58 EDT 2004

we only type ‘date’ and not ‘>date’.• All scripts should be able to execute on codd (einstein) which is a Solaris

machine, or any Linux distribution that supports Korn shell, located at /bin/ksh.

39

Flow Control in Shell Scripts

• We will explore the following flow control constructs:– for– while– case

40


• “for” flow construct– Format:

for VAR in list

do

statement-body

done– Loop for each element in list, assigning each

element to $VAR per loop iteration.

41

Example ex0240

0001 #!/bin/ksh00020003 ###########0004 # File: ex02400005 #0006 # Test each entry in arg0007 # to see if file or dir0008 ###########0009 for i in $1/*0010 do0011 file $i0012 done

• Line 9: For each file in dir specified by arg, determine file type.

• OUTPUT:>ex0240 $PWDex0010: executable /bin/ksh scriptex0020: executable /bin/ksh scriptex0030: executable /bin/ksh scriptex0040: executable /bin/ksh script

42

Example 0250

0001 #!/bin/ksh00020003 ###########0004 # File: ex02500005 #0006 # Test each entry in file0007 # to see if file or dir0008 ###########0009 for i in $(<$1)0010 do0011 file $i0012 done

• Line 9: Use the file specified by $1 as the list for the for loop

• OUPUT:>ls > trash>ex0250 trashex0010: executable /bin/ksh scriptex0020: executable /bin/ksh scriptex0030: executable /bin/ksh scriptex0040: executable /bin/ksh scriptex0050: executable /bin/ksh scripttrash: ascii text

43


• “while” flow construct– Format:

while [ expression ]

do

statement-body

done– Keep looping while expression is true.

44


0001 #!/bin/ksh00020003 ###########0004 # File: ex02600005 #0006 # Create a counter0007 ###########0008 COUNTER=00009 while [ "$COUNTER" -lt 5 ]0010 do0011 COUNTER=$(( $COUNTER + 1 ))0012 echo $COUNTER0013 done

• Line 9: Loop as long as COUNTER is less than 5.

• Line 11:Math computations are performed inside $(( ))

• Line 11:Notice space after “$((“ and before “))” – they are necessary.

• OUTPUT:>ex026012345

45


• “case” flow construct– Format:

case value in

pattern1) command1commandn;;

patternn) command1commandn;;

esac

46

Example ex02700001: #!/bin/ksh0002:0003: ###########0004: # File: ex02700005: #0006: # translate money0007: ###########0008: COIN=$10009: case "$COIN" in0010: 1) echo penny;;0011: 5) echo nickel;;0012: 10) echo dime;;0013: 25) echo quarter;;0014: *) echo not a coin!;;0015: esac

• Line 9: Use ‘case’ to translate coin value to coin name.

• Output:$ ex0270 25quarter$ ex0270 3not a coin!

47

More Than 9 Command Line Arguments

• As we have seen, a shell script can access command line arguments with shell variables such as $1, $2, $3, etc.

• Shell however, will only allow us to access up to 9 command line arguments ($10 will not work).

• The shift command is one way to address this, see:

http://www.shu.ac.uk/schools/cms/teaching/ps/unix/params.html

48


• Another technique is to use a special shell variable, $*

• $* contains all command line arguments.

• We can use $* to loop through an indeterminate number of command line arguments…

49


0001 #!/bin/ksh

0002 ###########

0003 # File: ex0280

0004 #

0005 # process more than 9 args

0006 ###########

0007 ARG_LIST="$*"

0008 for i in $ARG_LIST

0009 do

0010 printf "%03d " $i

0011 done

0012 printf "\n"

• Line 7: Store ‘n’ arguments in $ARG_LIST

• Line 8: Loop across all arguments• Line 10: print each argument

– without a new line– right justified– zero padded

• Line 12: print a “new line”• OUTPUT:ex0280 1 2 3 4 5 6 7 8 9 10 11001 002 003 004 005 006 007 008 009

010 011

50

Shell Arrays

• Shell supports arrays

• Setting Array Elements:MY_ARRAY[3]=34

MY_ARRAY[$COUNTER]=5

• Accessing Array Elements:echo ${MY_ARRAY[3]}

echo ${MY_ARRAY[$COUNTER]}

51

Shell Arrays0001: #!/bin/ksh0002: ###########0003: # File: ex02900004: #0005: # shell array example0006: ###########0007: COUNTER=10008: for i in blue green red yellow0009: do0010: MY_ARRAY[$COUNTER]=$i0011: COUNTER=$(( $COUNTER + 1 ))0012: done0013: printf "\n"0014:0015: echo ${MY_ARRAY[2]}

• Line 10: initialize each array element.

• Line 15: access 2nd array element• OUTPUT:ex0290

green

52

References

• “Essential System Administration”, by Aeleen Frisch, 2002

• “UNIX Shell Programming”, by Stephen G. Kochan and Patrick H. Wood, 1990

• “Learning the bash Shell”, by Cameron Newham and Bill Rosenblatt, 1998

Documents

1 ITEC 400 Shell (Part 2) George Vaughan Franklin University