271
09-Sep-2000 Version: 2.1 1 Unix Commands, Utilities System Programming And Overview of Internals By: Khaleel Ahmed

Unix Slides Vol1

Embed Size (px)

Citation preview

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 1/271

09-Sep-2000 Version: 2.1 1

Unix Commands, Utilities

System Programming

And Overview of Internals

By: Khaleel Ahmed

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 2/271

09-Sep-2000 Version: 2.1 2

Copyright © Notice

•Copyright © Information TechnologyConsultants reserves all rights to this

document. No part of this literature maybe stored in a retrieval system,transmitted, or reproduced in any way,including but not limited to photocopy,

photograph, magnetic or other record,without the prior agreement and writtenpermission of Information TechnologyConsultants

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 3/271

09-Sep-2000 Version: 2.1 3

Schedule – Day 1

• Overview of Unix• Learning the basic commands

• Customizing the shell environment

• Job Control

• Understanding file permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 4/271

09-Sep-2000 Version: 2.1 4

Schedule – Day 2

• Shell programming (Korn shell)• System Programming

• File Systems

• File Handling System Calls

• Process Management• Process Management System Calls

• Unix Architecture

• Process Management – Data structures and internals

• File System – Data structures and internals

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 5/271

09-Sep-2000 Version: 2.1 5

Schedule – Day 3

• System Programming• Overview of IPC facilities

• Using pipes

• Using named pipes

• Using shared memory

• Using message queues• Network Programming

• Overview of TCP/IP• IP, ARP, ICMP

• UDP• TCP

• Using socket system calls

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 6/271

09-Sep-2000 Version: 2.1 6

Unix Overview

• Developed at AT&T Bell Lab in 1969 by Ken Thompson on an idle PDP-7. Later moved to a

PDP-11.

• Ken Thompson also developed a new languagecalled ‘B’. Denis Ritchie developed ‘C’ languagefrom ‘B’.

• In 1973, Unix was re-written in ‘C’ language. This led to it’s popularity, the number of 

installations at Bell Lab grew to about 25.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 7/271

09-Sep-2000 Version: 2.1 7

Unix Overview

• As AT&T was not allowed to sell software at thattime, it was provided free-of-cost touniversities.

• By 1977, Unix installations grew to about 500,of which 125 were in universities. Licenses of Unix were provided to commercial andeducational institutions.

• Between 1977 and 1982, Bell Labs combinedseveral Unix variants into a single systemknown as Unix System III.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 8/271

09-Sep-2000 Version: 2.1 8

Unix Overview

• In 1983, Bell Lab announced their support for anew version, known as System V.

• By this time, University of California at Berkeley

had also developed a new variant of Unix,known as 4.3 BSD for VAX machines.

• The 4.3 BSD provided new features such as

reliable signals and TCP/IP.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 9/271

09-Sep-2000 Version: 2.1 9

Unix Overview

• Neither AT&T nor BSD is now active in Unixdevelopment– so most new developments arenow defined by a specific vendor or a body suchas IEEE (POSIX standard), FreeBSD organization

or the Linux community.

• Today, Linux is adding to Unix’s popularity.However, Linux is a Unix-clone and not Unix –

as the kernel was written from scratch andother tools have been largely taken from theFSF (Free Software Foundation).

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 10/271

09-Sep-2000 Version: 2.1 10

Unix Overview

Chronological evolution of Unix:

 Year From Bell Lab /AT&T From BSD

1969 First Edition

1973 Fifth Edition

1976 Sixth Edition1978 Seventh Edition 3 BSD

1982 System III 4.1/4.1a BSD

1983 System V

1984 System V, Release 2 4.2 BSD

1987 System V, Release 3 4.3 BSD1988 System V, Release 4 4.3 BSD Tahoe

1992 4.4 BSD

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 11/271

09-Sep-2000 Version: 2.1 11

Unix Overview

Various Unix Flavors

Product From Version

AIX IBM

HP Ux HP

Solaris Sun Tru64Unix Compaq

IRIX SGI

SCO Unix SCO

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 12/271

09-Sep-2000 Version: 2.1 12

Unix Overview

• Multi-user and multi-tasking environment

• Provides virtual memory support

• Devices are treated as files

• Provides more than one type of shells

(command interpreters)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 13/271

09-Sep-2000 Version: 2.1 13

Unix Overview

• Provides more than one mechanism for inter-process communication (IPC) .

• Excellent networking support through TCP/IP.

• The standard Unix File system is fast, but notvery robust – many vendors have introducedtheir own robust versions.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 14/271

09-Sep-2000 Version: 2.1 14

Unix Overview

• Most system operations depend upon theconfiguration files – such as /etc/passwd,/etc/group, /etc/hosts, /etc/services etc.

• Default system security can be enhancedthrough installation of additional security packs.

• Supports the X-Window system as a GUI.

• Supports POSIX standards to a large extent.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 15/271

09-Sep-2000 Version: 2.1 15

Unix Architecture

shell

Utility, command or shell script

File

System

I/O

System

Interrupt

and

Exception

Handlers

Memory

Manager 

Buffer 

Cache

Scheduler 

system calls

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 16/271

09-Sep-2000 Version: 2.1 16

Function of a shell

• Displays a prompt where the user can enter anew command.

• Validates the command entered by the user.

• Creates a new process using fork() or similarsystem call.

• Executes the command using one of exec()family of system calls.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 17/271

09-Sep-2000 Version: 2.1 17

Function of a shell

• Waits for a.out to complete using the wait() orsimilar system call, provided the command isnot being run as a background job.

  Note: Not all shells provide background jobfacility.

• Displays the prompt again, once the commandis complete, or immediately displays theprompt if it is a background job.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 18/271

09-Sep-2000 Version: 2.1 18

Function of a shell

• A shell also provides a programmingenvironment through a combination of :

• standard Unix utilities and commands

• built-in shell commands

• The programs written to work in environment of a shell are called shell scripts. A shell script is

a text file containing above commands.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 19/271

09-Sep-2000 Version: 2.1 19

 The process concept

• A process is a “program in execution”. It iscreated by Unix to run a “program”.

• Each process in Unix is allocated a time slice.

When a process consumes the time slice, Unixstops the process and resumes anotherprocess.

• In order to ensure that a process can bestopped and re-started as and when required,Unix stores a number of attributes for everyprocess.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 20/271

09-Sep-2000 Version: 2.1 20

Next Topic

Basic Unix Commands

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 21/271

09-Sep-2000 Version: 2.1 21

 Your Unix User Account

• Your Unix user account will have a login nameand most probably a password.

• In addition, a number of other attributes are

associated with a user account in Unix. The fourmost important attributes are:

• Home Directory Pathname

• Login Shell Pathname• User ID

• Group ID

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 22/271

09-Sep-2000 Version: 2.1 22

 Your Login Process

• Once you log in, Unix creates a process for you,and runs the login shell as a program in thatprocess.

• For the login shell process, Unix makes yourhome directory as the current directory.

• Therefore, after login, by default, you will be

working with your home directory.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 23/271

09-Sep-2000 Version: 2.1 23

 Your Login Process

• When you log into Unix, your login shell alsoexecutes a startup script for you from yourhome directory.

• The name of this script depends upon the loginshell. For Bourne shell, this file is “.profile” file. For C-shell, this file is “.cshrc” file.

• To customize your login environment, you canmodify these startup scripts.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 24/271

09-Sep-2000 Version: 2.1 24

Using Unix Commands

• Most Unix commands are individual programsthat perform various tasks. Unix has a large setof commands that are described relatedreference (man) pages.

• When you enter a command, the shellinterprets the command, and then gives anappropriate response -- that is, the systemeither runs the program or displays an error

message.

• If the command is valid, the shell directs theoperating system to do what is requested.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 25/271

09-Sep-2000 Version: 2.1 25

Using Unix Commands

• The shell acts as a command interpreter in thefollowing way:

• The shell displays a shell prompt and waits for you to

enter a command.• You enter a command, the shell analyzes it, andlocates the requested program.

• The shell asks the system to run the program, or itreturns an error message.

• When the program completes execution, controlreturns to the shell, which again displays the prompt.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 26/271

09-Sep-2000 Version: 2.1 26

Setting Your Password

• To set your password, use the passwd command. If your account does not have apassword, use the passwd command to setone.

• If your system is part of a networked system,you must use the yppasswd command toestablish or change your password.

• Your system administrator can advise you onthe specific procedures for your system.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 27/271

09-Sep-2000 Version: 2.1 27

Getting Help

• Many basic commands are covered in thiscourse. If you want to learn more about theseand other commands, see the man pages.

• When hard copy documents are not available,you can access online documentation by usingthe following commands:

• The man command displays online reference pages.• The apropos command displays a one line summary

of each command pertaining to a specified subject.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 28/271

09-Sep-2000 Version: 2.1 28

Getting Help

• The apropos command and man -k commandare useful tools if you forget a command name.

 These commands require access to the whatisdatabase. This database must be installed.

• Both these commands let you enter a commanddescription in the form of keywords. Thecommands then list all the reference pages that

contain any of the keywords.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 29/271

09-Sep-2000 Version: 2.1 29

Getting Help

• Assume that you cannot remember the name of the command that displays who is logged in tothe system.

• To display the names and descriptions of allman pages that are related to users who arelogged in, enter one of the following:

$ apropos "logged in"or

$ man -k 'logged in'

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 30/271

09-Sep-2000 Version: 2.1 30

Manipulating files and directories

• In Unix, each file is a series of bytes without anystructure (format). The directory is the only filethat has a structure imposed on it.

• Each file has one or more names. Each file isalso stored in a directory. The directories arestored in the file system.

• A file system is the useful arrangement of filesinto directories.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 31/271

09-Sep-2000 Version: 2.1 31

Listing files and directories

• In Unix, each file has a number of attributesassociated with it. Some of these are:

• File type

• File permissions• File owner user ID and group ID

• File size

• Date/time of creation, last update and last access

• Number of hard links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 32/271

09-Sep-2000 Version: 2.1 32

File names

• A file name can contain any character except thefollowing because these have special meaning to theshell:

• Slash ( / )

• Backslash ( \ )• Ampersand ( & )

• Left- and right-angle brackets (< and >)

• Question mark ( ? )

• Dollar sign ( $ )

• Left bracket ( [ )

• Asterisk ( * )• Tilde ( ~ )

• Vertical bar or pipe symbol ( | )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 33/271

09-Sep-2000 Version: 2.1 33

File names

• You may use a period or dot ( . ) in themiddle of a file name. If you use a dot at thebeginning of the file name it will be hiddenwhen doing a simple listing of files thru the “ls”

command.

• The maximum length of a file name dependsupon the file system used. Most new file system

allow a maximum length of 255 characters (thedefault). Older file systems allow a maximumfile name length of only 14 characters.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 34/271

09-Sep-2000 Version: 2.1 34

Name of current working directory

• The directory in which you are working at anygiven time is your current, or workingdirectory.

• If you are uncertain about the directory inwhich you are working, enter the pwd (printworking directory) command.

• You can use the cd command to move to yourhome directory from any location.

d di h

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 35/271

09-Sep-2000 Version: 2.1 35

Understanding pathnames

• A pathname specifies the location of adirectory or a file within the file system.

• A pathname consists of a series of directory

names separated by slashes ( / ) that endswith a directory name or a file name.

• A pathname that starts with a slash ( / ) is

called a full pathname or an absolutepathname.

U d di h

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 36/271

09-Sep-2000 Version: 2.1 36

Understanding pathnames

• You can also think of a full pathname as thecomplete name of a file or a directory.

• Regardless of where you are working in the file

system, you can always find a file or a directoryby specifying its full pathname.

• The file system also lets you use relativepathnames. Relative pathnames do not beginwith the / that represents the root directorybecause they are relative to the currentdirectory.

U d di h

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 37/271

09-Sep-2000 Version: 2.1 37

Understanding pathnames

• You can specify a relative pathname in one of several ways:

• As the name of a file in the current directory.• As a pathname that begins with the name of a

directory one level below your current directory.• As a pathname that begins with .. (dot dot, the

relative pathname for the parent directory).• As a pathname that begins with . (dot, which refers

to the current directory). This relative pathname

notation is useful when you want to run your ownversion of an operating system command in thecurrent directory (for example ./ls).

U d t di di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 38/271

09-Sep-2000 Version: 2.1 38

Understanding directories

• In Unix, a directory is a file that has a numberof entries. Each entry contains:

• A filename

• The I-node number of the file

• Every directory contains at least two entries:

• Filename of .. (dot dot)• Filename of. (dot)

U d t di di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 39/271

09-Sep-2000 Version: 2.1 39

Understanding directories

• In the C shell and the Korn or POSIX shell, youmay also use a tilde ( ~) at the beginning of relative pathnames.

• The tilde character used alone specifies yourhome directory. The tilde character followed bya user name specifies the home directory of that user. For example,

$ cd ~/mydir

$ cd ~micky/mouse

Li ti fil d di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 40/271

09-Sep-2000 Version: 2.1 40

Listing files and directories

• You can use the “ls” command to list files. Thesyntax of this command is:

ls [options] [directory_names]

• Some of the options are:• -a, for all file names

• -d, to list only directory names

• -l, for a long (detailed) listing• -R, for a recursive listing (traverse sub-directories)

• -t, for listing recently modified files

$ ls$ ls –l

$ ls -Rl

Li ti fil d di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 41/271

09-Sep-2000 Version: 2.1 41

Listing files and directories

1. Number of 512-byte blocks used by files in this directory.

2. Number hard of links to each file.

3. User name of the file's owner.

4. Group to which the file belongs.

5. Number of bytes in the file.6. Date and time the file was created or last modified.

7. Name of the file or directory.

Examples:$

$ ls -l total 4 [1] [2] [3] [4] [5] [6] [7]

-rw-r--r-- 1 larry system 101 Jun 5 10:03 file1

-rw-r--r-- 1 larry system 75 Jun 5 10:03 file2

-rw-r--r-- 1 larry system 75 Jun 5 10:03 file2

-rw-r--r-- 1 larry system 65 Jun 5 10:06 file3

drwxr-xr-x 2 larry system 32 Jun 5 10:07 project

Li ti fil d di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 42/271

09-Sep-2000 Version: 2.1 42

Listing files and directories

•  The “drwxr-xr-x” indicates file type andpermissions for each file.

• More about these permissions later…

Examples:$ ls -l 

total 2

-rw-r--r-- 1 larry system 101 Jun 5 10:03 file1

drwxr-xr-x 2 larry system 32 Jun 5 10:07 project

$

Li ti fil d di t i

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 43/271

09-Sep-2000 Version: 2.1 43

Listing files and directories

•Following are some of the file types:

- (hyphen) for ordinary files

b for block-special files

c for character-special files

d for directoriesl for symbolic links

p for pipe-special files (first in, first out)

s for local sockets

Finding the t pe of a file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 44/271

09-Sep-2000 Version: 2.1 44

Finding the type of a file

• Use the file command to see what kind of dataa file contains. The file command displayswhether the file is one of the following:

• A text file

• A directory• A FIFO (pipe) special file• A block special file• A character special file• Source code for the C or FORTRAN languages

• An executable (binary) file• An archive file in ar format

(continued…)

Finding the type of a file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 45/271

09-Sep-2000 Version: 2.1 45

Finding the type of a file

• An archive file in extended cpio or extended tarformat• An archive file in zip format• A compressed data file in gzip format• A file of commands text (shell script)

• An audio file in .voc, .iff, or .wav format• An image file in TIFF, GIF, MPEG, or JPEG format

• This command is especially useful when yoususpect that a file contains a compiledprogram, audio data, or image data. Displayingthe contents of these types of files can producestrange results on your screen.

Creating/Deleting Directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 46/271

09-Sep-2000 Version: 2.1 46

Creating/Deleting Directories

• You can use the “mkdir” command to createdirectory or sub-directories.

• You can use “rmdir” command to delete adirectory. The directory must be empty. Youcan also use the rm command with –r 

(recursive) option.$ rmdir c_progs$ rmdir c_progs/shared

$ mkdir c_progs$ mkdir c_progs/shared

Viewing files

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 47/271

09-Sep-2000 Version: 2.1 47

Viewing files

• You can use the “cat” command to view a fileon screen.

• You can also use the “cat” command to createa file.

$$ cat test.c$ cat *.c

$ cat >> file1This is line 1.

This is line 2.^D$

Viewing files

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 48/271

09-Sep-2000 Version: 2.1 48

Viewing files

• Following commands can also be used todisplay a file :

• pg

• more• page

• The pg command lets you view one or morefiles. When you display files that contain morelines than will fit on the screen, the pgcommand pauses as it displays each screen.

Printing files

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 49/271

09-Sep-2000 Version: 2.1 49

Printing files

• If your system has more than one printer, usefollowing format to specify where you want thefile to print:

lpr -P printername filename 

• The -P flag specifies a printer. Theprintername is the name of a printer. Printersoften have names that describe the location of 

the printer, the custodian or nearest monitor, orsome other descriptive name. Contact systemadministrator for info. on printer names.

Viewing Print Queues

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 50/271

09-Sep-2000 Version: 2.1 50

Viewing Print Queues

• To see the position of the request in the printqueue, use the lpq command.

• If there are no requests in the print queue, the

system responds with the following message:

no entries

• If there are entries in the queue, the systemlists them and indicates what is currently beingprinted.

Viewing Print Queues

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 51/271

09-Sep-2000 Version: 2.1 51

Viewing Print Queues

• Following is a typical lpq listing :

Rank Owner Job Files Total Size 

active marilyn 489 report 8470bytes

1st sue 135 letter 5444 bytes

2nd juan 360 (standard input) 969 bytes

3rd larry 490 travel 1492 bytes

• The lpq displays following for each print job:• Its position in the queue

• Its owner• Its job number• Name of the file• Size of the file in bytes

Copying files and directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 52/271

09-Sep-2000 Version: 2.1 52

Copying files and directories

• The cp (copy) command copies a file from onename to another name in your current directoryor copies the file from one directory to anotherdirectory.

$$ cp file2 file2x$$ mkdir reports$ cp file2 reports

Renaming/Deleting Files

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 53/271

09-Sep-2000 Version: 2.1 53

Renaming/Deleting Files

• You can use the “mv” command to rename ormove a file.

• You can use the “rm” command to delete a file.

$$ mv test.c newtest.c$ mv test.c c_progs

$ mv test.c c_progs/newtest.c

$

$ rm test.c$ rm c_progs/*.c$ rm c_progs/newtest.c

I/O Redirection

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 54/271

09-Sep-2000 Version: 2.1 54

I/O Redirection

• For every program, Unix automatically opensthree files. These files are:

• Standard input (file descriptor 0)

• Standard output (file descriptor 1)• Standard error (file descriptor 2)

• Normally, the standard output and standard

error file refer to the monitor whereas thestandard input file refers to the keyboard.

I/O Redirection

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 55/271

09-Sep-2000 Version: 2.1 55

I/O Redirection

• By using the I/O redirection facility, you canredirect output and error messages to a file.

 You can also take the input from a file.

• To redirect standard output, you can use the > symbol.

$$ ls -Rl / > allfiles$ cat test.c > copytest.c

I/O Redirection

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 56/271

09-Sep-2000 Version: 2.1 56

I/O Redirection

• To append standard output to an existing file,use >> symbol.

• To redirect standard error, use 2> symbol.

$$ ls –l *.c >> file_list

Note: The file will be created if it does not exist

$

$ cat test.c junk.c 2> errors

Note: Assumes that file does not exist.

I/O Redirection

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 57/271

09-Sep-2000 Version: 2.1 57

I/O Redirection

• To redirect standard input from an existing file,you can use the < symbol.

• In general, any command that expects input

from the keyboard can take the redirectedinput from a file.

$$ cat < file_list$ mail –s “this is a file” user1 < my_message

Piping

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 58/271

09-Sep-2000 Version: 2.1 58

Piping

• The piping facility allows you to send thestandard output from one program as inputto another program.

$$ ls –l | wc –l (   is sent to )$ who | wc –l (   is sent to )$ ls –l test.c junk.c | wc –l (   is sent to )

Note: The last example assumes that file junk.c does not exist.

Matching filenames

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 59/271

09-Sep-2000 Version: 2.1 59

Matching filenames

• If the file names have a common pattern, theshell can match that pattern, generate a list of those names, and automatically pass them tothe command as arguments.

•  The asterisk (*), sometimes referred to as awildcard , matches any string of characters.

ls file *

Matching filenames

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 60/271

09-Sep-2000 Version: 2.1 60

Matching filenames

• If the file names have a common pattern, theshell can match that pattern, generate a list of those names, and automatically pass them tothe command as arguments.

•  The asterisk (*), sometimes referred to as awildcard , matches any string of characters.For example, “file*” will match any filename

starting with word “file”.

Matching filenames

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 61/271

09-Sep-2000 Version: 2.1 61

Matching filenames

•  There is one exception to the general rules forpattern matching. When the first character of afile name is a period, you must match theperiod explicitly.

• For example, ls * displays the names of all filesin the current directory except those thatbegin with a period. The command ls -a 

displays all file names, including those thatbegin with a period.

Matching filenames

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 62/271

09-Sep-2000 Version: 2.1 62

Matching filenames

•  This restriction prevents the shell fromautomatically matching the relative directorynames. These are . (for the current directory)and .. (for the parent directory).

• In addition to the asterisk (*), shells provideother ways to match character patterns. Theseare summarized in the next slide.

Matching filenames

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 63/271

09-Sep-2000 Version: 2.1 63

Matching filenames

• The filename pattern-matching characters are:

* - matches any string, including NULL

? - matches any single character

[…] - matches any of the set characters[.-.] - matches any character within the

specified range

[!…] - matches any character except those

in the set

Hard Links and Soft Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 64/271

09-Sep-2000 Version: 2.1 64

Hard Links and Soft Links

• There are two kinds of links available for youruse:

• hard links• soft, or symbolic, links

 • When you create a hard link, you are providing

another name for the same file. Hard links letyou link only files in the same file system. All

the link names are on equal footing. It isincorrect to think of one file name as the realname, and another as only a link.

Hard Links and Soft Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 65/271

09-Sep-2000 Version: 2.1 65

Hard Links and Soft Links

• Soft links or symbolic links let you link both filesand directories.

• In addition, you may link both files anddirectories across different file systems.

• A symbolic link is actually a distinct file thatcontains a pointer to another file or directory.

 This pointer is the pathname to the destination

file or directory. Only the original file name isthe real name of the file or directory. Unlike ahard link, a soft link is actually only a link.

Creating Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 66/271

09-Sep-2000 Version: 2.1 66

Creating Links

• To link a hard link, use the following commandformat:

ln <existing_pathname> <new_pathname>

• If you want to link files and directories acrossfile systems, you can create symbolic links.

• To create a symbolic link, add an -s flag to theabove ln command and specify full pathnamesof both files.

Understanding Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 67/271

09-Sep-2000 Version: 2.1 67

Understanding Links

• Each file has a unique identification, called an I-node number. The I-node number refers tofile’s data stored at a particular location - ratherthan to the file name.

• A directory entry is a link between an I-nodenumber and a file name. This link enables youto link multiple file names to the same I-nodenumber.

• To display I-node number of files, use ls withthe -i flag.

Understanding Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 68/271

09-Sep-2000 Version: 2.1 68

Understanding Links

• Hard links to a file carry the same I-nodenumber. Since an I-node number represents afile within a particular file system, hard linkscannot exist between file systems.

• However, the soft link is a new file with its own,new I-node number.

• Since the soft link refers to the original file by

name, rather than by I-node number, they workacross file systems.

Deleting Links

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 69/271

09-Sep-2000 Version: 2.1 69

Deleting Links

• The rm (remove file) command does not alwaysremove a file.

• Assume that a file has several hard links (andtherefore more than one file name). In thiscase, the rm command removes the linkbetween the file I-node number and that filename, but leaves the physical file intact.

• The rm command physically removes a file onlyafter it has removed the last link between thatfile and it’s names.

Viewing File System Info

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 70/271

09-Sep-2000 Version: 2.1 70

e g e Sys e o

• The df command to displays the disk partition thatholds a particular directory. Examples show thatdirectories /u1/info and /etc are in different filesystems, but that /etc and /tmp are in the same filesystem:

$ df /u1/info Filesystem 512-blks used avail capacity Mounted on

/dev/rz2c 196990 163124 14166 92%/u1

$ df /etc 

Filesystem 512-blks used avail capacity Mounted

on/dev/rz3a 30686 19252 8364 70% /

$ df /tmp 

Filesystem 512-blks used avail capacityMounted on

/dev/rz3a 30686 19252 8364 70% / $

Understanding Unix Security

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 71/271

09-Sep-2000 Version: 2.1 71

g y

• Every valid user is known to the system througha user account. When a user account iscreated, information about the user is added tothe following two files:

• /etc/passwd - This file contains individual userinformation for all users of the system.

• /etc/group - This file contains group information forall groups on the system.

• These files define who can use the system andeach user's access rights.

 The /etc/passwd file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 72/271

09-Sep-2000 Version: 2.1 72

/ /p

• The /etc/passwd file contains records thatdefine login accounts and attributes for allsystem users. This file can be altered only by auser with super-user privileges.

• Each record in the /etc/passwd file defines alogin account for an individual user. The fieldsare separated by colons. The format of an/etc/passwd file entry is as follows:

username:password:UID:GID:gecos:login_directory:login_shell 

Fields in /etc/passwd entry

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 73/271

09-Sep-2000 Version: 2.1 73

/ /p y

• username - Your login name.• password - Your password stored in encrypted form.

• UID - A unique number identifying you to the system.

• GID - A number identifying your default group. You canbelong to one or more groups.

• gecos - This field usually contains general informationabout you, stored in some installation specific format.

• login_directory - Your current directory after logging into the system.

• login_shell - The program run by the login program afteryou successfully log in to the system.

 The /etc/group file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 74/271

09-Sep-2000 Version: 2.1 74

/ /g p

• The /etc/group file defines login accounts for allgroups using the system. This file can bealtered only by a user with super-userprivileges.

• Each entry in this file defines the login accountof one group. Groups provide a convenient wayto share files among users who are working onthe same project.

• The format of each entry is :

groupname:password:GID:user1[,user2,...,userN ]

Format of /etc/group file entry

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 75/271

09-Sep-2000 Version: 2.1 75

g p y

• groupname• A unique character string that identifies the group to

the system.

• password

• This field is always empty. Entries in this field are

ignored.• GID

• (Group ID) A unique number that identifies the groupto the system.

• usernames

• A list of users who belong to the group.

Protecting Files & Directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 76/271

09-Sep-2000 Version: 2.1 76

g

• Unix has a number of commands that enableyou to control access to your files anddirectories.

• You can protect a file or directory by setting orchanging its permissions, which are codesthat determine the way in which anyoneworking on your system can use the storeddata.

Protecting Files & Directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 77/271

09-Sep-2000 Version: 2.1 77

g

• Each file and directory has nine permissionsassociated with it, from following three types :• r (read)

• w (write)

• x (execute)

• Above three permissions occur for each of thefollowing three classes of users:• u (file owner)

• g (group members of file owner)

• o (all others; also known as world)

Protecting Files & Directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 78/271

09-Sep-2000 Version: 2.1 78

g

• The “r” permission lets users view or print thefile. The “ w” permission lets users write to(modify) the file. The “x” permission lets usersexecute (run) the file or search directories.

• The user/owner of a file or directory is generallythe person who created it. If you are the ownerof a file, you can change the file permissionswith the chmod  command.

Protecting Files & Directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 79/271

09-Sep-2000 Version: 2.1 79

g

• The “r” permission for a directory means thatit’s contents can be read, but not searched.Usually r and x are used together.

• The “ w” permission for a directory means thatfiles can be created or deleted.

• The “x” permission for a directory means that

it can be searched.

Protecting files and directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 80/271

09-Sep-2000 Version: 2.1 80

•  The “drwxr-xr-x” indicates file type and permissions for

each file.

•  The first character indicates file type. The remainingcharacters are interpreted as three groups of threecharacters each that indicate what read (r), write (w),and execute (x) permissions are set for the owner,group, and others. If a hyphen (-) appears, thecorresponding permission is not set.

Examples:$ ls -l 

total 2

-rw-r--r-- 1 larry system 101 Jun 5 10:03 file1

drwxr-xr-x 2 larry system 32 Jun 5 10:07 project

$

Protecting files and directories

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 81/271

09-Sep-2000 Version: 2.1 81

• The entry drwxr-xr-x indicates the following:

• This is a directory (the d letter)

• The owner can view it, write in it, and search it (therwx sequence)

• The group can view it and search it, but not write init (the first r-x sequence)

• All others can view it and search it, but not write in it(the second r-x sequence)

Default File Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 82/271

09-Sep-2000 Version: 2.1 82

• When you create a file or directory, the systemautomatically supplies a default permission.

 The following is a typical file permission :

-rw-r--r--

• This permission specifies that the owner hasread and write permissions while the group and

all others have only read permission.

Default File Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 83/271

09-Sep-2000 Version: 2.1 83

• When you create a file or directory, the systemautomatically supplies a default permission.

 The following is typical directory permission :

drwxr-xr-x

• This permission specifies that the owner hasread and write permissions while the group and

all others have read and execute (search)permission.

Default File Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 84/271

09-Sep-2000 Version: 2.1 84

• The default permission codes that your systemprovides relieve you from the task of specifyingthem explicitly every time you create a file ordirectory.

• If you want to create your own defaultpermission codes, you must change your usermask with the umask command.

Changing File Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 85/271

09-Sep-2000 Version: 2.1 85

• Use the chmod  (change mode) command to setor change the permissions for your files anddirectories. Please remember that whateverrestrictions you impose, the super-user can

always override them.

• There are two ways to specify the permissionsfor chmod :

• You can specify permissions with letters andsymbols.

• You can specify permissions with octal numbers.

Changing Permissions Using Letters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 86/271

09-Sep-2000 Version: 2.1 86

• The following is the format of the chmod command when using letters and symbols:

chmod userclass-operation-permission

filename

• The userclass-operation-permission entryrepresents three codes that specify the user

class code, operation, and permission code thatyou want to activate. The filename entry is thename of the file or files whose permissions youwant to change.

Changing Permissions Using Letters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 87/271

09-Sep-2000 Version: 2.1 87

• Use one or more of the following to define auser class:

u User (owner)

g Groupo All others (besides owner and group)

a All (user, group, and all others)

Changing Permissions Using Letters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 88/271

09-Sep-2000 Version: 2.1 88

• Use one or more of the following to define anoperation:

+ Add permission

- Remove permission= Assign permission regardless of previous

setting

Changing Permissions Using Letters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 89/271

09-Sep-2000 Version: 2.1 89

• Use one or more of the following to define apermission:

r Read

s Set user or group IDw Write

x Execute

Changing Permissions Using Letters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 90/271

09-Sep-2000 Version: 2.1 90

• Assume a file “file1” that has “-rw-r--r—” set of permissions.

• Now, enter the chmod command with the flags

go+w.

• This command expands the permissions forboth the group (g) and for others (o) by giving

them write access (+w) to file1 in addition tothe read access they already have.

Changing Permissions Using Octals

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 91/271

09-Sep-2000 Version: 2.1 91

• You can also use octal numbers to changepermissions. To use octal permission codes withthe chmod command, enter the command inthe following form:

chmod octal-number filename

• The octal-number entry is a 3-digit octalnumber that specifies the permissions for

owner, group, and others. The filename entry isthe name of the file whose permissions youwant to change.

Changing Permissions Using Octals

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 92/271

09-Sep-2000 Version: 2.1 92

• An octal number corresponds to each type of permission:

4 = read 2 = write 1 = execute

• To specify a group of permissions, add togetherthe appropriate octal numbers :

• 3 = -wx (2 + 1)

• 6 = rw- (4 + 2)• 7 = rwx (4 + 2 + 1)• 0 = --- (no permissions)

Setting Default Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 93/271

09-Sep-2000 Version: 2.1 93

• Every time you create a file or a directory,default permissions are established for it.

• These default permissions are initially

established either by the operating system orthe program you are running.

 

• The operating system assigns the default

permission values of 777 for executable filesand 666 for all other files.

Setting Default Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 94/271

09-Sep-2000 Version: 2.1 94

• If you want to further restrict the permissionsestablished by a program when it creates a fileor directory, you must specify a user mask withthe umask command.

• The user mask is a value that determines theaccess permissions when a file or directory iscreated.

Setting Default Permissions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 95/271

09-Sep-2000 Version: 2.1 95

• The umask command has following format:

umask octal-number

• The octal-number is a 3-digit octal number thatspecifies the permissions to be subtracted fromdefault permissions (777 or 666).

• Please note that through umask you actuallyspecify permissions that are not to begranted.

Becoming another user

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 96/271

09-Sep-2000 Version: 2.1 96

• The su command lets you alter your identityduring a login session.

• A reason for altering your identity is to be able

to access files that you do not own.

• To protect system security, you should notassume another identity without the owner's or

the system administrator's permission.

Becoming another user

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 97/271

09-Sep-2000 Version: 2.1 97

• The su command lets you log in to anotheruser's account only if you know that user'spassword.

• The su command authenticates you and thenresets both the process's user ID and theeffective user ID to the value of the newlyspecified user ID.

• The effective user ID is the user ID currently ineffect for the process

Becoming another user

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 98/271

09-Sep-2000 Version: 2.1 98

• The format of the su command is:

su username

• The username entry is the user name whoseidentity you want to assume.

• To confirm what identity you have assumed,use the whoami command. This commanddisplays the user name of the identity you haveassumed.

Becoming another user

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 99/271

09-Sep-2000 Version: 2.1 99

• After completing your work under a newidentity, you should return to your own loginidentity.

• To do so, press Ctrl/D or enter the exit command.

 The super-user concept

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 100/271

09-Sep-2000 Version: 2.1 100

• Every Unix system has a super-user who haspermissions that supersede those of ordinaryusers. This super-user is often referred to asroot.

• The root user has absolute power over therunning of the system. This user has access toall files and all devices and can make anychanges to the system. The root user is said tohave super-user privileges.

 The super-user concept

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 101/271

09-Sep-2000 Version: 2.1 101

• The following is a list of tasks ordinarilyperformed by root users:

• Edit files not usually changeable by ordinary users(for example, /etc/passwd )

• Change ownership and permissions of all files• Execute restricted commands like mount or reboot • Kill any process running on your system• Add and remove user accounts• Boot and shut down the system

• Back up the system• Think properly before doing any of the above…

 The super-user concept

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 102/271

09-Sep-2000 Version: 2.1 102

• To become a root user, use the su command. You mustknow the password for the root user. The format is:

su 

• The following example shows to become a root user :

$ su

Password: ...

#

• The # prompt typically indicates the root user.

Changing owners and groups

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 103/271

09-Sep-2000 Version: 2.1 103

• In addition to setting permissions, you cancontrol how a file or directory is used bychanging its owner or group.

• Use the chown command to change the ownerof a file. Enter the command in the followingform:

chown owner filename

Changing owners and groups

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 104/271

09-Sep-2000 Version: 2.1 104

• Use the chgrp command to change the groupof a file. Enter the command in the followingform:

chgrp group filename

• To change the group ownership of a file, youmust be a member of the group to which youare changing the file.

Using Processes

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 105/271

09-Sep-2000 Version: 2.1 105

• A program is a set of instructions that acomputer can interpret and run.

• While a program is running, it is called aprocess. Unix assigns a process identifier (PID) to every process.

• When a process begins executing, Unix opensthree files for the process: stdin (standard

input), stdout (standard output), and stderr(standard error).

Using Processes

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 106/271

09-Sep-2000 Version: 2.1 106

• Unix allows you to run a number of differentprocesses at the same time. These differentprocesses can be from one or multiple users.

• Therefore, you do not have to enter commandsone at a time at the shell prompt. Instead, youcan run both foreground and background 

 jobs simultaneously.

Using Processes

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 107/271

09-Sep-2000 Version: 2.1 107

• Usually, when you enter a command on thecommand line, you wait for the results todisplay on your screen. Such commands arecalled foreground processes or jobs.

• However, some commands require long time. If a long-duration command runs as a foreground

 job, you cannot execute other commands untilthe current one finishes. As a result, you maywant to run such command as a background job.

Using Background Jobs

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 108/271

09-Sep-2000 Version: 2.1 108

• To run a background job, you end the commandwith an ampersand (&).

• Once a job is running in the background, you

can perform additional tasks by entering othercommands at your workstation.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 109/271

New Topic

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 110/271

09-Sep-2000 Version: 2.1 110

Using Regular Expressions

Regular Expressions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 111/271

09-Sep-2000 Version: 2.1 111

• A regular expression specifies a set of strings tobe matched. It contains ordinary textcharacters and operator characters. Ordinarycharacters match the corresponding charactersin the strings being compared. Operatorcharacters specify repetitions, choices, andother features.

• Regular expressions fall into two groups:• Basic regular expressions

• Extended regular expressions

Basic Regular Expressions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 112/271

09-Sep-2000 Version: 2.1 112

• Basic expressions are built by concatenatingsimpler basic regular expressions.

• Ordinary characters and operator characters

together make up the set of simple basicregular expressions. You can concatenate anynumber or combination of simple expressionsto create a compound expression that willmatch any sequence of characters thatcorresponds to the concatenated simpleexpressions.

Basic Regular Expressions - Rules

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 113/271

09-Sep-2000 Version: 2.1 113

• An ordinary character matches itself.

• A period (dot) matches any single characterexcept the newline character.

• An * (asterisk) matches any number of occurrences of the preceding simpleexpression, including none.

Basic Regular Expressions - Rules

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 114/271

09-Sep-2000 Version: 2.1 114

• The [chars] (Bracket expression) matches asingle instance of any one of the characterswithin the brackets. Ranges of characters canbe abbreviated by using a hyphen. Forexample, [0-9a-z] matches any single digit orlowercase letter.

• Within brackets, all characters are ordinarycharacters except the hyphen (when used in arange abbreviation) and the circumflex (whenused as the first character inside the brackets).

Basic Regular Expressions - Rules

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 115/271

09-Sep-2000 Version: 2.1 115

• The ^ (Circumflex) - when used at thebeginning of a regular expression (or a sub-expression), matches the beginning of a line(`anchors' the expression to the beginning of the line). When used as the first character

inside brackets, excludes the bracketedcharacters from being matched. Otherwise, hasno special properties.

• $Dollar sign - when used at the end of a regularexpression, matches the end of a line(`anchors' the expression to the end of theline). Otherwise, has no special properties.

Basic Regular Expressions - Rules

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 116/271

09-Sep-2000 Version: 2.1 116

• The \char ( Backslash) - except within a bracketexpression, escapes the next character topermit matching on explicit instances of characters that are usually basic regularexpression operators.

• The expr  expr ... (concatenation) - matches anystring that matches all of the concatenatedexpressions in sequence.

 The grep Command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 117/271

09-Sep-2000 Version: 2.1 117

• The name of the grep command is an acronymfor global regular expression printer.

• We have already used the basic form of grep

command during discussion of regularexpressions. Therefore, we will not discuss thiscommand now.

• See “man grep” for more details….

Introduction to Shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 118/271

09-Sep-2000 Version: 2.1 118

Part I

Introduction to Shell

Introduction to shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 119/271

09-Sep-2000 Version: 2.1 119

• The shell is a program that acts as an interfacebetween Unix kernel and the user.

• Since the shell is just a program, there are

many types of shells available in a typical Unixsystem. Some of these are:

• Bourne Shell

• C-shell

• Korn-shell

• Bash etc.

Introduction to shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 120/271

09-Sep-2000 Version: 2.1 120

• When a user logs into Unix, the Unix systemautomatically executes a shell for the user – asspecified in the /etc/passwd file.

• The shell displays a prompt on the terminal,and reads a command entered by the user.Once the user enters a command, the shellinterprets the command and if the command isa valid command, executes the command itself of requests the kernel to execute thecommand.

Basic functions of a shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 121/271

09-Sep-2000 Version: 2.1 121

• Command interpretation• Command execution

• Filename substitution

• I/O re-direction

• Piping• Environmental control

• An interpretive programming language

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 122/271

Command Interpretation

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 123/271

09-Sep-2000 Version: 2.1 123

• If the command is not a built-in shell command,the shell looks up the list of directories specifiedby the PATH environment variable, and finds aprogram (having the same name as thecommand entered or looking up at an aliastable) that will perform that command.

• If the shell cannot find the command, it displaysan error message. Otherwise it executes thecommand.

Command Execution

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 124/271

09-Sep-2000 Version: 2.1 124

• If the command is a built-in shell command, theshell itself performs the command.

• If the command is not a built-in shell command,

the shell passes on the command to the kernelby:

• Creating a new (child shell) process

• Requesting the kernel to execute the command inthe new (child shell) process

Filename substitution

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 125/271

09-Sep-2000 Version: 2.1 125

• When the user enters a command string, theshell parses the string into followingcomponents:

•Command(the first part of the string)• Command arguments (the subsequent parts of thestring)

• For example, if you enter a command-string

such as “ls –l *.c”, this string contains the “ls”command and two arguments “-l” and “*.c”.

Filename substitution

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 126/271

09-Sep-2000 Version: 2.1 126

• In arguments of a command, the shellrecognizes certain characters – such as *, ?, [, ],and . - as special characters and expands thesecharacters into a filename list before executingthe command.

• To see how this works, enter followingcommands:

•echo “this is a message”• echo this is a message

• echo *

Filename substitution

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 127/271

09-Sep-2000 Version: 2.1 127

• To see how this works, enter followingcommands:

• echo \.*

•echo \.????????????• echo s*

• echo s??????.sh

• echo s[12345].sh

• echo s[1-5].sh

• echo s??????.sh

I/O Re-direction

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 128/271

09-Sep-2000 Version: 2.1 128

• The shell provides facilities to re-direct input,output and error for a command throughspecial characters:

• Use > character to redirect outout of a command to

a file.• Use < character to redirect input to a command from

a file.• Use >> characters to append outout of a command

to a file.

• The > and 2>&1 characters (Bourne Shell only)redirect outout and error of a command to a file.

I/O Redirection-Examples

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 129/271

09-Sep-2000 Version: 2.1 129

• cat *.c > outfile• cat *.c >> outfile

• cat nosuchfile 2> outfile

• cat * nosuchfile > outfile 2>&1

• mail $user –s “testing IO redirection” < outfile• cat * nosuchfile >> outfile 2>&1

Input Redirection-Special Case

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 130/271

09-Sep-2000 Version: 2.1 130

• The shell supports a special case of inputredirection that allows you to provide theinput to a command from within a shellscript.

• For example, the cat command expects it’sinput from the standard input (keyboard) or afile. Therefore, if you use the cat command asfollows in a shell script, it will expect the inputto come from the keyboard:

cat > outfile 

Input Redirection-Special Case

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 131/271

09-Sep-2000 Version: 2.1 131

• In the previous example, if you wanted toprovide input to the cat command from withinthe shell script itself, you can use <<characters with a label. Consider followingshell script:

$ cat > outfile << _END_OF_DATA_

 This data comes from a shell script

 This data comes from a shell script_END_OF_DATA_

$ cat outfile

Using the tee command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 132/271

09-Sep-2000 Version: 2.1 132

• The “tee” command allows you to send theoutput of a command to a file as well as tothe standard output.

• Example:

$ ls –l | tee outfile

$ cat outfile

Environment Control

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 133/271

09-Sep-2000 Version: 2.1 133

• The shell recognizes a number of environmentvariables, that allow you to customize the shell

environment.

  Note: Shell also supports normal variables.

• These variables include: PATH, PS1, PS2 etc.

• However, some environment variables cannot

be changed. These include: HOME, USER, PWDetc.

Programming Language

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 134/271

09-Sep-2000 Version: 2.1 134

• The shell allows you to assign values tovariables. This is the most basic featurerequired of a programming language.

• The shell also provides other programmingconstructs including:

• Condition checking

•Loops• Input and output statements

Different types of shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 135/271

09-Sep-2000 Version: 2.1 135

• Bourne Shell

• Developed in 1979 by Stephen Bourne.

• Program name is sh.

• The Bourne shell is distributed as the standard shellon most Unix systems.

• Was the first shell developed for Unix.

Different types of shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 136/271

09-Sep-2000 Version: 2.1 136

• “C” Shell

• Developed by Bill Joy at University of California,Berkeley as part of BSD Release of Unix.

• Program name is csh.

• It is called “C” shell as the programming language itprovides is similar to the “C” programming language.

• Provides additional features such as running aprogram in background, command aliasing etc.

Different types of shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 137/271

09-Sep-2000 Version: 2.1 137

• Korn Shell

• Developed by David Korn at Bell Labs around 1988.

• Program name is ksh.

• Incorporates most features of Bourn shell as well asC shell. Bourne shell scripts can run withoutmodification in this shell – and therefore mightbecome a replacement for the Bourne shell.

Different types of shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 138/271

09-Sep-2000 Version: 2.1 138

• Bourne Again Shell

• This is a freeware from the FSF - Free SoftwareFoundation.

• Developed by Brian Fox and Chet Ramey.

• Program name is bash.

• Incorporates most features of Bourn shell as well asC shell. Provides a number of enhancements such ascommand line editing.

• This is the default shell provided in Linux.

Using the shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 139/271

09-Sep-2000 Version: 2.1 139

• As discussed earlier, Unix starts a shell for youwhen you log in.

• The shell displays a prompt (such as a $

prompt) and waits for you to enter a command.

• You can enter following types of commands:• Interactive command (such as cat, vi, ls etc.)

• Non-interactive command (such as a shell script)• Built-in command (such as set, echo, cd, pwd)

Using the shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 140/271

09-Sep-2000 Version: 2.1 140

• To find out which shell you are using, enter

echo $SHELL

• To change your login shell to the Bourne Shell,enter:

chsh /bin/sh

Using the shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 141/271

09-Sep-2000 Version: 2.1 141

• To temporarily use the C Shell, enter:

csh

• If C shell is available on your system, it will bestarted. You can now use C shell. To return tothe original shell, enter:

exit OR press CTRL-D

Using the shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 142/271

09-Sep-2000 Version: 2.1 142

• To find out whether you have Korn Shellavailable on your system, enter:

find / -name ‘ksh’ -print

• If Korn shell is available on your system, find will show you the directory where it is stored.

 The output may be something like this:

  /sbin/ksh

Using the shell

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 143/271

09-Sep-2000 Version: 2.1 143

• To add the current directory in PATH, enter:

export PATH=$PATH:. (C/Korn/bash shells)

PATH=$PATH:. ; export PATH (Bourne shell)

• To set your terminal type to vt100, enter:

export TERM=vt100 (C/Korn/bash shells)

TERM=vt100 ; export TERM (Bourne shell)

Using sub-shells

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 144/271

09-Sep-2000 Version: 2.1 144

• A sub-shell (or a child shell) is a new shellprogram that you start from the current shell.

• The sub-shell has it’s own set of environment

variables – and does not have access to theparent shell’s environment.

• You can run a command in a sub-shell by using

special characters – ( ) and { ; }

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 145/271

Next Topic

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 146/271

09-Sep-2000 Version: 2.1 146

Introduction to Shell Scripting

What is a shell script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 147/271

09-Sep-2000 Version: 2.1 147

• A shell script is a text file containing acombination of following:

• Unix commands (such as ls, cat).

• Shell commands (such as variable assignment) and

programming constructs (such as if statement andloops)

• Since a shell script is a program, it is normally

assigned execute permission using the chmod command.

What is a shell script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 148/271

09-Sep-2000 Version: 2.1 148

• To write effective shell scripts, you need tohave working knowledge of basic commandsand utilities in Unix. These include:

• regular expressions

• grep, cut, awk and sed utilities

• This course does not permit time to cover theabove utilities. Where relevant, we might

demonstrate the use of these utilities.

Executing a shell script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 149/271

09-Sep-2000 Version: 2.1 149

• There are many ways of executing a shellscript:

• By passing the shell script name as an argument tothe shell. For example:

sh script1.sh

• If the shell script is assigned execute permission, it

can be executed using it’s name. For example:

./script1.sh

Executing a shell script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 150/271

09-Sep-2000 Version: 2.1 150

• There are many ways of executing a shellscript:

• If the shell script has execute permission and is stored in adirectory listed in PATH, it can be executed using it’s name.

For example:

script1.sh

• If the shell script has execute permission and is stored in adirectory listed in PATH, it can be executed using the dot

command. For example:

. script1.sh

Sample Shell Script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 151/271

09-Sep-2000 Version: 2.1 151

#! /bin/sh

## The above line has a special meaning. It must be the

# first line of the script. It says that the commands in

# this shell script should be executed by the Bourne

# shell (/bin/sh).

# ---------------------------------------------------------------

echo “Hello $USER….”

echo “I hope you like working with shell scripts..”

# ---------------------------------------------------------------

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 152/271

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 153/271

Using Normal Variables

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 154/271

09-Sep-2000 Version: 2.1 154

• Having defined above variables, you can usethe echo command to display the value of eachvariable:

• $ echo $x

• $ echo $textline_1

• $ echo $textline_2

• $ echo $allusers

• $ echo $countusers

Using Normal Variables

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 155/271

09-Sep-2000 Version: 2.1 155

• To reset a variable, use

• $ var_name=

• OR

• $ var_name=#

• OR• $ var_name=“”

• All the three commands are equivalent.

Using Environment Variables

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 156/271

09-Sep-2000 Version: 2.1 156

• To define an environment variable, usefollowing syntax:

variable_name=value

export variable_name

• Examples:

$ x=10; export x

$ allusers=`who` ; export allusers

Built-in Environment Variables

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 157/271

09-Sep-2000 Version: 2.1 157

•Korn shell provides a number of built-inenvironment variables. These include:

• PATH

• HOME

• PWD• SHELL

• TERM

• MAIL

• USER

• LOGNAME

Built-in Environment Variables

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 158/271

09-Sep-2000 Version: 2.1 158

•Each of the built-in environment variablescontains information assigned by the shell.

• Some of these environment variables (such as

PATH, TERM, MAIL etc.) can be modified.

• The other environment variables (such as USER,SHELL, LOGNAME, PWD etc.) can not be or

should not be modified.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 159/271

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 160/271

Shell meta-characters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 161/271

09-Sep-2000 Version: 2.1 161

•Examples:

• $ x=* ; echo $x

• $ x=?????? ; echo $x

• $ x=[a-p]????? ; echo $x

• $ x=‘$USER sold the book for $10’ ; echo $x• $ x=“$USER sold the book for $10” ; echo $x

• $ x=“$USER sold the book for \$10” ; echo $x

• $ x=`who` ; echo “Current users = “ $x

Shell meta-characters

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 162/271

09-Sep-2000 Version: 2.1 162

•Examples:

• $ x=* ; echo $x

• $ x=?????? ; echo $x

• $ x=[a-p]????? ; echo $x

• $ x=‘$USER sold the book for $10’ ; echo $x• $ x=“$USER sold the book for $10” ; echo $x

• $ x=“$USER sold the book for \$10” ; echo $x

• $ x=`who` ; echo “Current users = “ $x

Passing parameter to script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 163/271

09-Sep-2000 Version: 2.1 163

• You can pass parameter to a shell script byentering the parameters after the name of the

shell script.

•Within the shell script, you can refer to theparameters through variables $1 thru $9.

• If you need to use more than 9 parameters, you

can use the shift command within the shellscript to refer to additional parameters.

Passing parameter to script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 164/271

09-Sep-2000 Version: 2.1 164

• Consider following shell script:

----------------------script2.sh--------------------------

echo “Total parameters entered: $#”

echo “First parameter is : $1”

echo “Last parameter is : $$#”

------------------------------------------------------------

• Execute the above script using the“script2.sh these are the parameters” command.

Passing parameter to script

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 165/271

09-Sep-2000 Version: 2.1 165

• The shell parameters are passed as strings.

• If you want to pass a string containing multiplewords as a single parameter, you must enclosethe string within quotes. For example,

$ script2.sh “P1” “P2” “P3”

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 166/271

Doing arithmetic operations

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 167/271

09-Sep-2000 Version: 2.1 167

• You can use normal operators with the expr command. For example,

$number_1 = `expr $1 + $2`

$number_2 = `expr $1 - $2`

$number_3 = `expr $1 / $2`

$number_4 = `expr $1 * $2`

$number_5 = `expr $1 % $2`

Condition Checking In Scripts

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 168/271

09-Sep-2000 Version: 2.1 168

• Korn shell provides the if command to test if acondition. The general format of this commandis:

if condition

thencommand

fi

• The condition is typically formed using thetest command.

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 169/271

09-Sep-2000 Version: 2.1 169

• The general syntax of test command is:

test <expression>

• The expression can be formed using acombination of shell variables and theoperators supported by the test command.

 These operators provide facility to comparenumbers, string and logical values, file types

and file access modes.

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 170/271

09-Sep-2000 Version: 2.1 170

• To compare two integers using test followingoperators are available:

-eq (equal to)

-ne (not equal to)

-lt (less than)

-le (less than or equal to)

-gt (greater than)

-ge (greater than or equal to)

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 171/271

09-Sep-2000 Version: 2.1 171

• Please note that you can only compare twointegers using the test command.

• The syntax for this comparison is:

test integer1 operator integer2OR

[ integer1 operator integer2 ]

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 172/271

09-Sep-2000 Version: 2.1 172

• To compare two strings using the test command, following operators are available:

• string1 == string2 (equal to)

• string1 != string2 (not equal to)

• string1 (string is not NULL)• -n string1 (string is not NULL and exists)

• -z string1 (string is NULL and exists)

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 173/271

09-Sep-2000 Version: 2.1 173

• The syntax for this string comparison is:

test string1 operator string2

OR

[ string1 operator string2 ]

OR

test operator string

OR

[ operator string ]

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 174/271

09-Sep-2000 Version: 2.1 174

• To check a file type/access permissions using

the test command, following operators areavailable:

• -s file (file is not empty)

• -f file (file is not a directory)• -d file (file is a directory)

• -r file (file is readable)

• -w file (file is write-able)

• -x file (file is executable)

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 175/271

09-Sep-2000 Version: 2.1 175

• To check a file type/access permissions using

the test command, following operators areavailable:

• -b file (file is a block device)

• -c file (file is a character device)• -p file (file is a named pipe)

• -g file (file has sticky bit set)

• -u file (file has setuid bit set)

• -t file_des (file descriptor is standard output)

Using the “test” command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 176/271

09-Sep-2000 Version: 2.1 176

• The syntax for this file checks is:

test operator filename

OR

[ operator filename ]

Combining conditions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 177/271

09-Sep-2000 Version: 2.1 177

• It is possible to combine conditions by using

following operators:

• -a (logical AND operator)

• -o (logical OR operator)

• ! (logical NOT operator)

 

Combining conditions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 178/271

09-Sep-2000 Version: 2.1 178

• The syntax for this is:

test expression_1 –a expression _2, OR

[expression _1 –a expression _2 ]

test expression_1 –o expression _2, OR

[expression_1 –o expression_2 ]

test ! expression _1, OR [ ! expression_1]

 

Checking multiple conditions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 179/271

09-Sep-2000 Version: 2.1 179

• The complex form of if statement is as follows:

if condition_1

then

command

elif condition_2

then

command

elsecommand

fi 

Using the “case” statement

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 180/271

09-Sep-2000 Version: 2.1 180

• Korn shell also provides a case statement for

this purpose. The syntax is:

case value in

pattern_1) command

command ;;…

pattern_n) command

command ;;

*) commandcommand ;;

esac

Using the “case” statement

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 181/271

09-Sep-2000 Version: 2.1 181

• The value can be a shell variable.

• The patterns pattern_1 thru pattern_n can besimple string values or regular expressions.

Using the “for” loop

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 182/271

09-Sep-2000 Version: 2.1 182

• The Korn shell provides a for loop. The syntax

of this loop is:

for variable in list

do

  command…

  command

done

Another example

i t4 h

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 183/271

09-Sep-2000 Version: 2.1 183

----------------------script4.sh--------------------------

#! /bin/shusernames=`who | awk {print $1}`

echo “Total users logged in = $#usernames”

#

for user in ${usernames}

do

echo $user

done------------------------------------------------------------

Using the “while” loop

h h ll id hil l h

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 184/271

09-Sep-2000 Version: 2.1 184

• The Korn shell provides a while loop. The

syntax of this loop is:

while condition

do

  command…

  command

done

Using shell functions

• Shell functions are one of the most useful

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 185/271

09-Sep-2000 Version: 2.1 185

• Shell functions are one of the most useful

features of the Korn shell.

• Once a function is defined to the shell, it ismaintained in memory, so disk i/o is reduced

and execution is faster than a traditional shellscript.

• A function can also be called several times inyour script, so total lines of code are reduced. Ashell function can be defined in the .profile file,or in a separate script file.

Using shell functions

Sh ll f ti t d

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 186/271

09-Sep-2000 Version: 2.1 186

• Shell functions are a way to group commands

for later execution using a single name for thegroup. They are executed just like a "regular"command.

• Shell functions are executed in the current shellcontext; no new process is created to interpretthem. Functions are declared using this syntax:

[ function ] name () { command-list; }

Using shell functions

[ function ] func name () { command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 187/271

09-Sep-2000 Version: 2.1 187

[ function ] func_name () { command-

list; }

• The above defines a shell function namedname. The reserved word function is optional. If 

the function reserved word is supplied, theparentheses are optional.

• The body of the function is the command-listbetween { and }. This list is executed wheneverfunc_name is specified as the name of acommand.

Using shell functions

[ function ] func name () { command

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 188/271

09-Sep-2000 Version: 2.1 188

[ function ] func_name () { command-

list; }

• The exit status of a function is the exit status of the last command executed in the body.

• When a function is executed, the argumentsto the function become the positionalparameters during its execution. Thespecial parameter `#' that expands to thenumber of positional parameters is updated toreflect the change. Positional parameter 0 isunchanged.

Using shell functions

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 189/271

09-Sep-2000 Version: 2.1 189

[ function ] func_name () { command-list; }

• If the built-in command return is executed in afunction, the function completes and executionresumes with the next command after thefunction call.

• You can also provide an integer argument tothe return statement.

Using shell functions

[ function ] func name () { command-

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 190/271

09-Sep-2000 Version: 2.1 190

[ function ] func_name () { command-

list; }

• When a function completes, the values of thepositional parameters and the special

parameter `#' are restored to the values theyhad prior to the function's execution.

• If a numeric argument is given to return, that isthe function's return status; otherwise thefunctions's return status is the exit status of thelast command executed before the return.

Using shell functions

[ f i ] f () { d

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 191/271

09-Sep-2000 Version: 2.1 191

[ function ] func_name () { command-list; }

• Variables local to the function may be declaredwith the local built-in variable syntax. Thesevariables are visible only to the function andthe commands it invokes.

• Functions may be recursive. No limit is placedon the number of recursive calls.

Shell functions - Example

# Function net on returns 0 if SQL*Net processes are

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 192/271

09-Sep-2000 Version: 2.1 192

# Function net_on returns 0 if SQL Net processes are

found# and 1 if they are not found. No arguments required.

#

net_on () {

sleep 10

if ps -f -u oracle | grep -v grep | grep tns > /dev/nullthen

return 0

else

return 1

fi}

Shell functions - Example

# Function dbs on takes one argument (SID) and returns 0 if

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 193/271

09-Sep-2000 Version: 2.1 193

# Function dbs_on takes one argument (SID) and returns 0 if 

# related database processes are found and 1 if they are not.#

dbs_on () {

sleep 10

if ps -f -u oracle | grep -v grep | grep ora_ | grep "$1" > /dev/null

thenreturn 0

else

return 1

fi

}

Shell functions - Example

#!/bin/sh

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 194/271

09-Sep-2000 Version: 2.1 194

#!/bin/sh

## Initialize shell functions

. $HOME/bin/shell_functions

# Make sure that user specified an SID value.

if [ -z "$1" ]then

echo "dbs_start: `date '+%H:%M:%S`: Error: No databasespecified."

exit 2

fi(…continued...)

Shell functions - Example

(…continued from previous slide...)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 195/271

09-Sep-2000 Version: 2.1 195

# Check to see if the specified database is alreadyrunning.

if dbs_on "$1"

then

echo "dbs_start: `date '+%H:%M:%S`: $1 Databasealready started."

exit

fi

(…continued...)

Shell functions - Example

(…continued from previous slide...)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 196/271

09-Sep-2000 Version: 2.1 196

# Check to see if SQL*Net Listener is runningif net_on

then

echo "dbs_start: `date '+%H:%M:%S`: SQL*Net Listeneralready on."

elseecho "dbs_start: `date '+%H:%M:%S`: SQL*Net Listener

not running."

lsnrctl start

(…continued...)

Shell functions - Example

(…continued from previous slide...)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 197/271

09-Sep-2000 Version: 2.1 197

if net_onthen

echo "dbs_start: `date '+%H:%M:%S`: SQL*Net Listenerstarted."

else

echo "dbs_start: `date '+%H:%M:%S`: SQL*Net Listenernot started."

exit 2

fi

fi

(…continued ...)

Shell functions - Example

(…continued from previous slide...)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 198/271

09-Sep-2000 Version: 2.1 198

# Set environment variablesORACLE_SID=$1

export ORACLE_SID

# Start the specified database using Server Manager in

line mode.svrmgrl << ENDMGR

connect internal

startup exclusive open

ENDMGR

(…continued ...)

Shell functions - Example

(…continued from previous slide...)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 199/271

09-Sep-2000 Version: 2.1 199

# Check to make sure database starts correctly.if dbs_on "$1"

then

echo "dbs_start: `date '+%H:%M:%S`: $1 Databasestarted successfully."

elseecho "dbs_start: `date '+%H:%M:%S`: Error: $1 Database

could not start."

exit 2

fi

(…end of script...)

Debugging Shell Scripts

• At times you may need to debug a program to

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 200/271

09-Sep-2000 Version: 2.1 200

• At times you may need to debug a program tofind and correct errors. Two options to the shcommand can help you debug a program:

sh -v shell_script_name

• The “-v” (verbose) option prints the shell inputlines as they are read by the system.

Debugging Shell Scripts

sh x shell script name

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 201/271

09-Sep-2000 Version: 2.1 201

sh -x shell_script_name

• The “-x” (execute) option prints the commandsand their arguments as they are executed.

• To try these two options, create a shell programthat has an error in it.

Debugging Shell Scripts

• For example, create a file called bug that contains the

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 202/271

09-Sep-2000 Version: 2.1 202

For example, create a file called bug that contains the

following list of commands:

$ cat bug

today=`date`

echo enter person

read personmail $1

Hello $person ,

The system will not be available between 3 and 5 PM today.

$today.

From System Administrator

$

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 203/271

New Topic

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 204/271

09-Sep-2000 Version: 2.1 204

System Programming

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 205/271

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 206/271

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 207/271

 The Role Of Standard “C” Library

• The standard “C” library contains:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 208/271

09-Sep-2000 Version: 2.1 208

 The standard C library contains:

• Object code of all library routines such as printf,scanf, getchar etc.

• Object code of system call wrapper functions.Each wrapper function invokes a system call. Theactual system call is a part of Unix kernel.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 209/271

File Handling

• Topics

Ashwin:

Start from here

2marrow

Ashwin:

Start from here

2marrow

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 210/271

09-Sep-2000 Version: 2.1 210

• Creating files• Reading and writing files

• Getting file statistics

• Sharing and locking files

• Reading directories

Unix Files

• A file is a stream (series) of bytes. A file does

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 211/271

09-Sep-2000 Version: 2.1 211

y

not have any structure (such as “fixed-lengthrecords”, “indexed”etc.).

• There is no concept of a “record” as far as Unix

is concerned. If you want to write an applicationthat uses records, you need to implement thisfeature yourself.

• There is no system call to delete a part of a file

(a record from a file). If you need this feature,you need to implement it yourself.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 212/271

Creating/Opening a file

• The open( ) system call has following prototype:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 213/271

09-Sep-2000 Version: 2.1 213

p ( ) y g p yp

int open( const char *pathname, // the pathname andint flags, // the mode in which

mode_t mode // access permissions

);

• The open( ) system, if successful, returns adescriptor (which is actually a number) to referto the open file. The process then uses this

descriptor to read/write etc. from the file.

Creating/Opening a file

• The flags parameter can be:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 214/271

09-Sep-2000 Version: 2.1 214

g p

• O_CREAT

• O_RDONLY

• O_WRONLY

• O_RDWR

• O_APPEND

• O_EXCL ( used with O_CREAT) - create a file only if it does notexist

• The mode parameter indicates octal

permissions to be assigned to the file. Onlyused this parameter when the flags containsO_CREAT.

Creating a file

• Try the followingTry:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 215/271

09-Sep-2000 Version: 2.1 215

y g

int fd;

fd = creat( “myfile.txt”, 0644);

if ( -1 == fd ) {

perror (“Cannot create file”);}

else

{

printf(“File created”);

}

Change “myfile.txt” to“/myfile.txt” and run the

 program

Opening an existing file

• Try the following (include another header file:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 216/271

09-Sep-2000 Version: 2.1 216

y g

<fcntl.h>)

int fd;

fd = open( “myfile.txt”, O_APPEND);

if ( -1 == fd ) {

perror (“Cannot open file”);

}

else

{

printf(“File opened…descriptor is: %d \n”, fd );

}

Try:

Change “myfile.txt” to

“notmyfile.txt” and run the

 program

Creating a file using open

• Try the following (include another header file:

Try:

Change O RDWR to

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 217/271

09-Sep-2000 Version: 2.1 217

y g

<fcntl.h>)

int fd;

fd = open( “newfile.txt”, O_CREAT | O_RDWR, 0644 );

if ( -1 == fd ) {

perror (“Cannot create file”);

}

else

{

printf(“File created…descriptor is: %d \n”, fd );

}

g _

O_RDWR | O_EXCL andrun the program

Reading from a file

• To read from a file, a process uses the read( )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 218/271

09-Sep-2000 Version: 2.1 218

system call, which has following prototype:

#include <unistd.h>

ssize_t read( int fd, // file descriptor

void *buf, // address of the buffer to store data

size_t count // length of the buffer

);

• The read( ) system call, if successful, returns

the number of bytes read from the file. If itencounters end-of-file, it returns 0.

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 219/271

Writing to a file

• To write to a file, a process uses the write( )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 220/271

09-Sep-2000 Version: 2.1 220

system call, which has following prototype:

#include <unistd.h>ssize_t write( int fd, // file

descriptorvoid *buf, // address of the

buffer size_t count //length of the buffer

);

• The write( ) system call, if successful, returnsthe number of bytes written to the file. If itencounters error, it returns -1.

Writing to a file

• Add following to last program:

Try:

Change sizeof to strlen 

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 221/271

09-Sep-2000 Version: 2.1 221

int bytes;char buffer[64];/* the existing code goes here… */

printf(“Enter data:”);gets(bufffer);bytes = write ( fd, buffer, sizeof(buffer) );if ( -1 == bytes ) {

perror(“Cannot write to file…”);}printf(“Written %d bytes to file..\n”, bytes);

g

and run the program

Try-2:

Change fd to 1 and run the

 program

Try-3:

Change fd to 0 and run the

 program

Closing a file

• To close a file, a process uses the close( )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 222/271

09-Sep-2000 Version: 2.1 222

system call, which has following prototype:

#include <unistd.h>

int close( int fd, // file descriptor

);

Lab 1 – Reading the file

• Write a C-program to

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 223/271

09-Sep-2000 Version: 2.1 223

• Accept a file name from command line parameter(use argc and argv parameter to main - argv[1] willbe the filename)

• Open the file in “readonly” (O_RDONLY) mode

• Read 80 characters at a time from the file, until theend of the file is detected (the read system callreturns 0 when end of file is detected).

• Display each line as it is read

• Close the file when end-of-file is reached

Additional system calls

• Unix provides following additional calls for files:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 224/271

09-Sep-2000 Version: 2.1 224

• lseek(), to move within a file

• unlink(), to delete a file

• stat(), to obtain information about a file

• fcntl(), to change attributes of a file (including lockand unlock)

• flock()/lockf() to lock a file

• dup()/dup2() to duplicate a file descriptor

Moving within a file

• The lseek() call has following prototype

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 225/271

09-Sep-2000 Version: 2.1 225

#include <unistd.h>

off_t lseek( int fildes, // file descriptor

off_t offset, // the byte offset where to move

int whence // how to interpret the offset

// SEEK_SET - w.r.t start of file// SEEK_CUR - w.r.t current position

// SEEK_END - w.r.t end of file

);

Moving within a file

• The lseek() call’s “offset” parameter can be a

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 226/271

09-Sep-2000 Version: 2.1 226

positive or a negative value.

• If successful, lseek( ) returns the new position inthe file (where file pointer is pointing to).

• It is also possible to move beyond end of file !!!,but not possible to move before start of file-)))))

Obtaining information of a file

• To obtain information about a file, you can use

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 227/271

09-Sep-2000 Version: 2.1 227

stat() or fstat() system calls

int stat( const char *file_name, // pathname of the file

struct stat *buf // pointer to a variable of 

// type struct stat);

int fstat( int filedes, // file descriptor

struct stat *buf );

Changing/getting file attributes

• To change/get attributes of a file, you can use

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 228/271

09-Sep-2000 Version: 2.1 228

the fcntl() system call

int fcntl(int filedes, // file descriptor

int cmd // command such as F_GETOWN

);

int fcntl(int filedes, // file descriptor

int cmd // command such as F_SETOWN

long arg // argument to command

);

Deleting a file

• To remove a file name, you can use unlink()

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 229/271

09-Sep-2000 Version: 2.1 229

system call. Please note that this call removes aname and decrements the link count. If all thenames are removed (link count becomes zero),then the file is physically deleted.

int unlink(const char *pathname

// pathname of file to be deleted

);

Locking a file

• To lock/unlock a file, you can use the flock() or

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 230/271

09-Sep-2000 Version: 2.1 230

fcntl() system calls:

#include <sys/file.h>

int flock( int fd, // file descriptor

int operation // type of lock)

 The lock types (operation) are:

Shared (LOCK_SH) or Exclusive ( LOCK_EX)

Locking a file

• To lock/unlock a file, you can use flock() or

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 231/271

09-Sep-2000 Version: 2.1 231

fcntl() system calls:

#include <sys/file.h>

int flock( int fd, // file

descriptorint operation // type of lock

)

 The lock types (operation) are:Unlock (LOCK_UN) or Donot wait/block for lock ( LOCK_NB)

Locking a file

• Example of using flock()

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 232/271

09-Sep-2000 Version: 2.1 232

int main(){

int fd, status;

fd = open( “sharedfile.txt”, O_RDWR);status = flock( fd, LOCK_EX | LOCK_NB );if ( -1 == status ){

perror(“Cannot lock file”);…

}

Locking a file

• Example of using fcntl() to lock a file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 233/271

09-Sep-2000 Version: 2.1 233

int fd, status;

struct flock lockinfo;

fd = open( “sharedfile.txt”, O_RDWR);

lockinfo.l_type = F_WRLCK; // write lock

lockinfo.l_whence =SEEK_SET ; // fromBOF

lockinfo.l_start = 0; // start from BOFlockinfo.l_len = 100; // lock 100 bytes

status = fcntl( fd, F_SETLKW , &lockinfo );

Duplicating a file descriptor

• You can use the dup() or dup2( ) system call to

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 234/271

09-Sep-2000 Version: 2.1 234

duplicate an existing file descriptor.

• If you duplicate an existing file descriptor, youwill have two descriptors referring to the same

open file.

• And then you can do magic...

Duplicating a file descriptor

• Try this in a program where a file is open for

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 235/271

09-Sep-2000 Version: 2.1 235

writing:

/* after existing code */

close(1);

dup(fd);

printf(“This should appear on screen….\n”);

Duplicating a file descriptor

• If you duplicate an existing file descriptor, you

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 236/271

09-Sep-2000 Version: 2.1 236

will make that existing file descriptor to alsorefer to another file.

• For example, if you have file descriptor “4”

referring to “use_dup.txt” and if you duplicatethat descriptor, another descriptor (such as filedescriptor “1”) may also refer to the same file(“use_dup.txt”).

Duplicating a file descriptor

• This happens because of the “close” system

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 237/271

09-Sep-2000 Version: 2.1 237

call that is used before the call to dup, to closedescriptor 1.

• The dup system call returns the lowest

available file descriptor.

• Since descriptor “1” (which refers to standardoutput, or screen) was closed, it is the lowest

available file descriptor and is returned by dup.

Duplicating a file descriptor

• Therefore, fd as well as “1” refer to the same

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 238/271

09-Sep-2000 Version: 2.1 238

file.

• When you use printf in your program, it writesit’s data to standard output (descriptor 1).

Since descriptor 1 also refers to the same file,the output of printf goes to the same file.

• This means that the standard output refers to

another file. This is what the shell does whenyou use a command such as “ls -l > outfile”.

Duplicating a file descriptor

• The shell uses the dup/dup2 calls to implement:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 239/271

09-Sep-2000 Version: 2.1 239

• input/output redirection

• piping (in combination with the pipe() system call)

Standard File Descriptors

• Every Unix process gets three standard file

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 240/271

09-Sep-2000 Version: 2.1 240

descriptors, that are always open:

• descriptor 0, means standard input (stdin) and refersto the keyboard

• descriptor 1, means standard output (stdout) andrefers to the monitor screen

• descriptor 2, means standard error (stderr) andrefers to the monitor screen

Reading Directories

• To read directories, following calls are available:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 241/271

09-Sep-2000 Version: 2.1 241

• opendir

• readdir or scandir

• rewinddir/seekdir

• closedir

Reading Directories

• To read directories, following calls are available:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 242/271

09-Sep-2000 Version: 2.1 242

• Include <dirent.h> header file.

• Declare a variable DIR *d;

• Declare a variable struct dirent *dp;

•Use opendir() function to open a directory.• In a while (1) loop, use readdir() function to readeach directory entry. Break from the loop and closethe directory when readdir returns NULL.

• Use printf to display following from each entry (the

dp variable):• d_ino (print using %d)

• d_name (print using %s)

Lab 2 – Using Files

• Write a C program that maintains a database

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 243/271

09-Sep-2000 Version: 2.1 243

(which is a file) of your training schedule. Thisdatabase contains following information:

• course name, upto 40 characters• start date , upto 12 characters

• end date , upto 12 characters• faculty , upto 16 characters

• The program will use following system calls:

• open, read, write, lseek, close

Lab 2 – Using Files

• The training schedule data is stored in a

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 244/271

09-Sep-2000 Version: 2.1 244

structure that contains following members

• schedule ID (a number, starting from 1)

• course name, upto 40 characters

• start date , upto 12 characters• end date , upto 12 characters

• faculty , upto 16 characters

• delete flag (0-not deleted or 1-means deleted)

Lab 2 – Using Files

• The program will provide options to :

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 245/271

09-Sep-2000 Version: 2.1 245

• add a new schedule record

• query an existing schedule record, by schedule ID

• view all schedule records

• update an existing schedule record , by schedule ID• delete an existing schedule record , by schedule ID

Using debugger

• Each Unix system may provide a different

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 246/271

09-Sep-2000 Version: 2.1 246

debugger.

• Most Unix systems support “dbx”. Linuxprovides the “gdb” debugger.

• To use the debugger:• compile your program with “-g” option to the “cc”

command.

• Invoke the debugger using gdb <program_name>

New Topic

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 247/271

09-Sep-2000 Version: 2.1 247

Process Management

System Calls

Process Management

• Topics

C ti

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 248/271

09-Sep-2000 Version: 2.1 248

• Creating a new process• Running a new program in a process

• Waiting for child process to exit

• Collecting exit status from child process

• Accessing process environment

• Changing process environment

• Process relationships (Process Groups)

• Redirecting output of a process to a file

• Redirecting input to a process from a file

 The Process Concept

• A “process” (also called a task) is used by Unix

t

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 249/271

09-Sep-2000 Version: 2.1 249

to run a program.

• You can say that a process describes theattributes of a “running” program.

• Since a running program has a number of dynamic attributes, Unix continuously storesthe attributes of a running program for every

process.

 The Process Concept

• A number of attributes of a running program

d i ll h S f th

Ashwin:

Satrt 4m here 24rd 

april 2002

Ashwin:

Satrt 4m here 24rd april 2002

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 250/271

09-Sep-2000 Version: 2.1 250

dynamically change. Some of these are:

• Program Counter

• Stack Pointer

• CPU Registers

• Memory locations referenced by the program

• And more…. (more about this in Unix Internalstraining program).

 The Process Concept

• A number of attributes of a running program

DO NOT NORMALLY h S f th

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 251/271

09-Sep-2000 Version: 2.1 251

DO NOT NORMALLY change. Some of these are:

• Process ID (when a program runs, it is given a ID)

• Parent Process ID (the Process ID of the process thatstarted this program)

• Real User ID (the User ID of the user that started thisprogram)

• Real Group ID (the Group ID of the user that startedthis program)

Reading Process Attributes

• Some of the process attributes (such as Process

ID P t P ID R l U ID R l G

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 252/271

09-Sep-2000 Version: 2.1 252

ID, Parent Process ID, Real User ID, Real GroupID) can be read using Unix system calls.

• Some of the process attributes (such as Real

User ID, Real Group ID, Effective User ID,Effective Group ID) can be changed using Unixsystem calls.

Creating a new process

• A process uses the fork() system call to

t Thi th t t t

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 253/271

09-Sep-2000 Version: 2.1 253

create a new process. This means that to createa new process you always need a process.

• In Unix, all processes are created using the

fork() system call, with the exception of the“init” process (which has a PID of 1), which is“hand-crafted “ (meaning it is not created usingthe fork() system call, but by another method).

Creating a new process

• When you use the fork() system call, one more

ill b t d if f k() i f l

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 254/271

09-Sep-2000 Version: 2.1 254

process will be created if fork() is successful.

• All statements after the fork() system call inyour program are executed by two processes -

the original process that used fork(), plus thenew process that was created by fork( ).

Creating a new process

• To change this behavior, the fork system call

h b d i d t t diff t l th

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 255/271

09-Sep-2000 Version: 2.1 255

has been designed to return a different value toeach process.

• The fork() returns “0” in the new (child) process

• The fork() returns >0 (child process’s PID) in theparent process, meaning the process that used thefork( ) system call

Note: fork( ) also returns -1 if it cannot createa process

Creating a new process

• When you use the fork() system call in a C-

program you check the return value of the

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 256/271

09-Sep-2000 Version: 2.1 256

program, you check the return value of thefork() and then force the child process andparent process to execute a different set of statements

• if it “0”, then it is child process

• if it is >0 then it is parent process

Running a new program in a process

• To run a new program in a process, you use one

of the “exec” family of calls (such as “execl”)

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 257/271

09-Sep-2000 Version: 2.1 257

of the “exec” family of calls (such as “execl”)and specify following:

• the pathname of the program to run

• the name of the program• each parameter to the program

• (char *)0 as the last parameter to specify end of parameter list

Running a new program in a process

• Therefore, to run a new program in the child

process:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 258/271

09-Sep-2000 Version: 2.1 258

process:

• check that fork() returns 0 (it is a child process).

• use one of the “exec” calls (such as execl( ) )to run anew program in the child process.

Waiting for child process to exit

• A parent process can use the wait( ) system

call to wait for the exit of ANY child process

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 259/271

09-Sep-2000 Version: 2.1 259

call to wait for the exit of ANY child process.

• When the parent process uses the wait( ) system call, it will be suspended until one of 

it’s child processes (not a specific childprocess) exits. Once a child process exits, theparent process comes out of the wait( ) systemcall.

Waiting for child process to exit

• A parent process can also use the wait( ) 

system call to collect the exit status of the child

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 260/271

09-Sep-2000 Version: 2.1 260

system call to collect the exit status of the childprocess.

• A child process can return an exist status to the

parent by using the int parameter to theexit( ) system call.

• To wait for a specific child process to exit, a

parent process can use waitpid( ) system call.

Waiting for child process to exit

• The waitpid() call has following prototype:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 261/271

09-Sep-2000 Version: 2.1 261

pid_t waitpid(pid_t pid, int *status, int options);

• The parent will pass the PID of the child in the

“pid” parameter to indicate a specific childprocess.

Lab 3 – Getting Process ID’s

• Write a C program that displays the following

information about the current process:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 262/271

09-Sep-2000 Version: 2.1 262

information about the current process:

• Process ID (use getpid() system call)

• Parent process ID ( use getppid() system call)

• Real user ID and group ID ( use getuid()/getgid()system call)

• Effective user ID and group ID ( usegeteuid()/getegid() system call)

Lab 4 – Creating Processes

• Write a C program that does the following :

• Creates a process ( use fork() system call )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 263/271

09-Sep-2000 Version: 2.1 263

• Creates a process ( use fork() system call )

• The child process displays following message and runs “ps-l” command (use execl() system call):

“Child <PID> : Trying to run the ps command”

• The parent process waits for child process to finish the “ps-l” command (use wait() system call) and displays followingmessage ( and the exit status ? )

“Parent <PID> : Finished running the ps command”

Lab 5 – Accessing Environment

• Write a C program that displays a list of allenvironment variables in the following format:

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 264/271

09-Sep-2000 Version: 2.1 264

Environment Variable Value

----------------------------------------------------------

HOME /home/user1

SHELL /bin/bash

… …

• Use extern variable “environ” to access the abovelist and display. See “man environ” for details.

Process Relationships

• When a child process (the program running in

the child process) exits it returns a status to

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 265/271

09-Sep-2000 Version: 2.1 265

the child process) exits, it returns a status tothe parent process.

• Therefore, when a child process is exiting, the

parent process should be ready to collect theexit status. If the parent process is not ready tocollect the exit status (if it is busy orsuspended), the child process will not be able toexit. Such a child process is known as a“zombie” process.

Process Relationships

• Unix always ensures that a process has a

parent process

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 266/271

09-Sep-2000 Version: 2.1 266

parent process.

• If the parent process exits (while the childprocess is alive), Unix makes another process

(the “init” process) as the parent process of thechild that is alive. However, since the originalparent process terminated, such a child processis called an “orphan”.

Lab 6 – Zombie Process

• Write a C program that creates a zombie process. The zombie process should be destroyed after 15

seconds

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 267/271

09-Sep-2000 Version: 2.1 267

seconds.

• Use the fork() and sleep() routines in this program.

• Use the ps -l command to check that the processhas become a zombie process.

Process Groups

• Each Unix process belongs to a process

group Unix uses process groups to distributei l ll f j b l

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 268/271

09-Sep-2000 Version: 2.1 268

group. Unix uses process groups to distributesignals as well as for job control.

• By default, a process inherits it’s process group

from the parent process.

• However, a process can change it’s processgroup, or create a new process group at

anytime by using the setprgp() system call.

Redirecting output

• Unix opens three files for every Unix process.

These files are identified by file descriptors 0( di ) 1 ( d ) d 2 ( d )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 269/271

09-Sep-2000 Version: 2.1 269

 These files are identified by file descriptors 0(stdin) , 1 (stdout) and 2 (stderr).

• A process can close the standard output using

the close(1) system call, and use the dup()system call to redirect output to a file. The filemust have been opened earlier (in write mode)using the open() system call.

Redirecting input

• Unix opens three files for every Unix process.

These files are identified by file descriptors 0( tdi ) 1 ( td t) d 2 ( td )

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 270/271

09-Sep-2000 Version: 2.1 270

 These files are identified by file descriptors 0(stdin) , 1 (stdout) and 2 (stderr).

• A process can close the standard input using

the close(0) system call, and use the dup()system call to redirect output from a file. Thefile must have been opened earlier (in readmode) using the open() system call.

Lab 7 – Output Redirection

• Write a C program that :

• Use the open system call to create a new file. Thefil i d t t th

7/31/2019 Unix Slides Vol1

http://slidepdf.com/reader/full/unix-slides-vol1 271/271

Use the open system call to create a new file. Thefilename is passed as an argument to the program.

• Closes standard output and duplicates the filedescriptor returned in previous step.

• In a for loop that has 10 iterations, displays a

suitable message.• Closes the file