BASIC UNIX

Version 1.0

UNIX Operating System

Features :

Multiuser & Multiprocessor System

Time Sharing Operating System

Written in “C” (HLL)

has all advantages of HLL

performance low compared with HLL

Simple User Interface

Hierarchical File System

Consistent File Format - the Byte Stream

Supports Languages - FORTRAN, BASIC, PASCAL, Ada, COBOL, LISP,

PROLOG, C, C++

System V - about 64 system callsUnix/Intro/01

Architecture of UNIX System

Hardware

Kernel

Cpp Sh who

A.cut

date

wc

greped

vi

os

comp

nroff

CC

Oth

er a

pplic

atio

n pr

ogra

m

Unix/Intro/02

Unix O.S.

Services Provided :

Process Management - Creation, Termination or

Suspension and

Communication

CPU Scheduling

Memory Management - Swapping & Paging

File Management

Peripheral Device controlled access to process - terminal, printer, tape drives,

disk drives and network devices

Unix/Intro/03

File System

A Hierarchical Structure

Consistent Treatment of Data

The Ability to Create and Delete Files

Dynamic Growth of Files

The Protection of File Data

The Treatment of Peripheral Devices as Files

Unix/Intro/04

File System

bin etc usr unix dev fs

sh date who passwd src bin tty01tty00 maurymjb

cmd

Date.c Who.c

Unix/Intro/05

/ (Root)

File System

File Types

Directory File Regular File Device File

Classes of User

Owner Group Others

Access Permission

Read Write Execute

Unix/Intro/06

File System

Consists of Sequence of Logical Blocks

(Multiple of 512 k bytes)

An Installation may have

several physical disk units

each containing one or more file system

File System Treated as Logical Device (Logical Device Number)

Data from another File System can be Accessed by Mounting that File System

Block - unit of data transfer

- can be assigned to only one file

Unix/Intro/07

File System Layout

Boot Block

SuperBlock

Inode List

DataBlocks

Unix/Intro/08

File System

Boot Block

Beginning of File System (1st Sector)

Contains Bootstrap Code

Every File System has (possibly empty) Boot Block

Super Block

Size

# of Files that can be Stored

Free Space Information

Inode Information

Inode List : Information about each inode e.g. access permissions, data/time,

phy. Addr. Etc.

Data Block : File Data & Administrative Data

Unix/Intro/09

File System

Inode Table Fields :

File Owner Identifier - Owner & Group id File Type - Regular, Directory, Character / Block /Pipe File Access Permissions

r w x r w x r w x

user group others

Date & Time - Last File/Inode Modified/Accessed No. of Links Disk Addresses File Size

Unix/Intro/10

File System

Directories

Directory Entry Size is 16 bytes Two Fields

Inode Number File Names

832

179817992114

0

:

...

Initmount

passwd:

crash

:

Current DirectoryParent Directory

Unix/Intro/11

Building Block Primitives

1. Each Process has Access to 3 Files

standard input (“<“)

standard output (“>”)

standard error (“2>”)

e.g.

$ls

$ls>temp

$mail xyz <letter

$nroff - mm <file1> outfile 2> errfile

$cat >test

$cat ->test

$cat >>test

$catt ->>testUnix/Intro/12

Building Block Primitives

2. Pipe - allows stream of data to be passed between reader &

writer process

The data that the first processor writes into the

pipe is input to second process.

E.g. a) $grep main a.c. b.c. c.c. | wc-l

- counts the number of lines in the files

that contain the string “main”

b) $who| wc-l

c) $ls|wc-l

d) $ls|wc-l>fcount

Unix/Intro/13

Building Block Primitives

3. Tee - standard output can be saved in a file as well as displayed

- can be placed anywhere in pipeline

e.g. $who | tee user.lst

kumar tty01 May 18 09 : 32

sharma tty02 May 18 11 : 18

tiwary tty03 May 18 13 : 21

e.g. $who | tee user.lst|wc-l

3

$_

Unix/Intro/14

File SystemThe Structure of File System

Root (/)

bin dev etc mnt tmp unix usr

<Files> <Files> <Files> <Files> bin Kumar Sharma

<files> prog safe

<files> <files>

Unix/Intro/15

File System

/ bin commonly used UNIX Command like

/usr/bin who, ls, cat, wc etc. are stored here

/dev - contains device files of all hardware devices

/etc - contains those utilities mostly used by system administrator

e.g. passwd, chmod, chown

/tmp - used by some UNIX utilities especially vi and by user to store

temporary files

/usr - contains all the files created by user, including login directory

/unix - kernel

Release V - does not contain / bin

- contains / home instead of /usr

Unix/Intro/16

Basic Commands - pwd, cd

1. Checking your directory - pwd command

$pwd

/usr/Kumar

2. Changing directories - cd commands

e.g. $pwd

/usr/Kumar

$cd Prog

$pwd

/usr/Kumar/Prog

e.g. $cd / bin

e.g. $cd .. / tmp

e.g. $cd

Unix/cmd/01

Basic Commands - logname, cat

3. Prompting login directory - logname command

$cd /usr/Sharma

$logname

Kumar

$_

4. Displaying and Creating Files - cat command

e.g. $cat dept.lst

01|accounts|6213

02|admin|5423

:

:

06|training|1006

$_

Unix/cmd/02

cat Command

Scrolling of large files can be freezed by pressing <ctrl s> and resumed by <ctrl q>

e.g. $cat chap1 chap2-display contents of chap2 immediately after displaying chap1

e.g. $cat>testType here the text contentsAfter typing the entire textPress <ctrl d> to save and exit<ctrl d>$_

e.g. $cat pgm1 pgm2 > pgm3

e.g. $cat test test test

Unix/cmd/03

ls Command

5. Listing current directory contents - ls command

e.g. $ls

a.out

chap1

chap2

test

test.c

$_

Unix/cmd/04

ls Command

Options available in ls

Option Description

-x Displays multicolumnar output (prior to Release 4)-F Marks executables with *and directories with /-r Sorts files in reverse order (ASCII collating sequence

by default)-l The long listing showing seven attributes of a file-d Forces listing of a directory-a Shows all files including ., .. And those beginning with

a dot-t Sorts files by modification time-R Recursive listing of all files in sub-directories-u Sorts files by access time (when used with the -t

option)-i Shows i-node number of a file-c Sorts by time of change of the i-node-s Displays number of blocks used by a file-1 One file per line (Release 4)

Unix/cmd/05

ls CommandE.g. $ls - ltotal 8_rw_rw_rw_1 Kumar group 44 May 9 09:08 dept.h_rw_rw_rw_1 Kumar group 212 May 9 09:08 dept.q_rw_rw_rw_1 Kumar group 154 May 9 09:08 emp.h_rw_rw_rw_1 Kumar group 742 May 9 09:08 emp.q$_

Consider first entry -Field1 --> mode

- r w x r w x r w x

File type

- --> regular filed --> directory filec --> character - readb --> block read

--> user permissions --> group permissions --> others permissions

Unix/cmd/06

ls Command

Field2 - indicates number of links

Field3 - File owner id

Field4 - Group id

Field5 - File size in bytes

Field6 - Date/time last altered

Field7 - Filename

The total field at the beginning indicates total no. of block occupied by these

files

Unix/cmd/07

mkdir Command

Creates a directory

e.g. $mkdir doc

e.g. $mkdir doc doc/example doc/data

e.g. $mkdir doc/example doc

- will give error - Order important

Unix/cmd/08

rmdir Command

Used to remove directory

Only empty dir can be deleted

More than one dir can be deleted in single command

Command should be executed from at least one level above in the

hierarchy

e.g. $rmdir doc

$rmdir doc/example doc

$rmdir doc doc/example

- will give error

Unix/cmd/09

chmod Command

Used to alter file permissions

Format :

chmod <category> <operation> <permission> <filenames>

Category Operations Attribute

u-user +assigns permission r-read

g-group -remove permission w-write

o-others =assigns absolute x-execute

permission

a-all

Unix/cmd/10

chmod Command

E.g. $chmod u+x note

$ls - l note

-rwx r-- r -- 1 …… note

e.g. $chmod ugo+x note

$ls - l note

-rwxr-xr-x …… note

e.g. $chmod u-x, go+r note

$chmod u+x note note1 note2

$chmod o+wx note

$chmod ugo=r note

Unix/cmd/11

chmod Command

Octal notation describes both category and permission similar to =operator (absolute assignment) read permission - assigned value is 4

write permission - assigned value is 2

Execute permission - assigned value is 1

e.g. $chmod 666 note

will assign read & write permission to all

e.g. $chmodd 777 note

will assign all permissions to all

e.g. $chmod 753 note

?

Unix/cmd/12

cp Command

Copies a file or group of files

e.g. $cp chap1 temp/chap2

Option - i (interactive)

$cp - i chap1 chap2

cp: overwrite chap2 ? y

Option -r (recursive) to copy entire directory

$cp - r temp newtemp

Unix/cmd/13

rm Command

To delete filese.g. $rm chap1 chap2 chap3

$rm chap*$rm *Are you sure? y

Option - i (interactive delete)$rm - i chap1 chap2chap1 : ? ychap2: ? n

Option - r (recursive delete)$rm -r *(Warning: Pl. do not use this option)

Option -f (deletion of write protected files)$rm -f chap1$rm -r -f*

Unix/cmd/14

mv Command

Used to rename file or group of files

e.g. $mv chap1 man1

the destination file if existing gets overwritten

dir can also be renamed

e.g. $mv temp doc

e.g. $mv chap1 chap2 chap3 man1

will move chap1, chap2 & chap3 to man1

directory

Unix/cmd/15

more Command

Allows user to view a file one screen at a time

Syntax :more <options <+linenumber> <+/pattern> <filename>

: appears after displaying screen

<spacebar> or <enter> - next screen

q - quit

n - switch to next file on command

p - switch to previous file

f - displays filename and line no.

/pattern - to locate line with pattern

! - to run command

Unix/cmd/16

more Command

Command Description

k:n Skips to kth next file that has beenspecified in the command line

k:p Skips to kth previous file that has beenspecified in the command line

:f Displays current filename and linenumber

= Displays current line numberspacebar Displays next screenks Skips k lines forwardkf Skips k screens forward, Reverts to beginning of file if no search

made, or to point where last search wasmade

/pattern Searches for pattern forwardn Repeats last search forwardv Starts up the vi editor. Repeats previous command! cmd Executes the Unix command cmdq Exits from more

Unix/cmd/17

wc Command

Counts lines, words and character depending on option

takes one or more filename as arguments

e.g. $wc infile

3 20 103 infile

$wc infile test

3 20 103 infile

10 100 180 test

13 120 283 total

$wc - l infile

3 infile

$wc - w infile

20 infile

$wc - c infile

103 infile

$wc - wl infile

20 3 infile

Unix/cmd/18

od Command

Lists the octal value (ASCII) of any file content

used to display non-printing character

options

b to display 16 byte of data on each line

c to display character above octal value

e.g. $od - bc odfile

Unix/cmd/19

cmp Command

To compare two files & echo first mismatch

option -l gives detailed list of byte no. and different byte in octal

e.g. $cmp note1 note2

note1 note2 differ : char3, line1

$cmp - l note1 note2

3 143 145

6 170 167

7 171 170

Unix/cmd/20

comm Command

To compare two sorted files

displays 3 - columnar output

first column - lines unique to first file

second column - lines unique to second file

third column - line common to both files

option -1, -2, -3 can be used to display specific column only

Unix/cmd/21

diff Command

Display file differences

display which lines in one file have to be changed to make both files identical

option -e generates a script when applied to first file makes it identical to

second

option -b consecutive blanks considered as one blank and trailing blanks

are ignored

Unix/cmd/22

banner Command

Creates poster by blowing up its arguments

maximum ten character in a line

e.g. $banner UNIX

$banner ‘UNIX’ ‘IS HERE’

Unix/cmd/23

cal Command

Displays calendar of a particular month/ year

any calendar from year 1 to 9999

e.g. $cal 1991

$cal 01 1991

Unix/cmd/24

date Command Displays system date number of options available

option m - monthh - month named - day of monthy - last two digit of the yearH - hourM - minuteS - secondT - time in hh:mm:ssa - day of week (Sun to Sat)r - time in AM/PM

e.g. $date Fri Dec 7 15:00:21 EST 1990$date +%m 12$date +”%h%m” Dec 12$date “+The day of week is %a and time y.r” The day of week is Fri and time 03:23:42 PM

Unix/cmd/25

who Command

Displays current users of the system

displays logname, terminal no., login date and time

e.g. $wo

$who - H displays Header for all

columns

$who - Hu displays additional info. As

IDLE, PID, comments

$who am i logname of the user

Unix/cmd/26

Terminal Related Commands

tty - displays device name of the terminal

stty -a - displays current settings

stty can also be used to change the setting

e.g. to use <ctrl c> as interrupt key instead

<Del> key

$stty intr \^c

Unix/cmd/27

head Command Displays the beginning of a file also works with multiple filese.g. $head -3 emp.lst

will display first 3 lines if not specified (default) 10 lines$head -2 emp.lst dept.lst

tail command displays the end of a file default - display last 10 lines $tail -3 emp.lst

displays last 3 lines $tail +50 emp.lst

displays till end of file starting from line no. 50 max. 4096 char segment can be displayed if size specified is greater than 4096 then last 4096 char are displayed

Unix/cmd/28

tail Command

$tail -15c emp.lst display last 15 characters

$tail -1b emp.lst block is used as unit of measurement

Unix/cmd/29

Cut & Paste CommandsCut :

Allows you to slice the file vertically identifies both column and fields

e.g.$cut -c6-22, 24-32 emp.lst cuts the file emp.lst vertically from column 6 to 22 and from column 24 to 32

$cut -c-5, 10-15, 25 - emp.lst

implies 1-5 implies 25 - end of file d option to specify delimiter can be used with -f option which specifies field no. default delimiter is tab cut selects entire line if not properly delimited

$cut -d “|” -f 2,3 emp.lst

$cut -d “|” -f 1,5- empl.lst | tee temp

Unix/cmd/30

Cut & Paste CommandsPaste :

paste more than one file laterally

$paste file1 file2

<…file1> <file2> delimiter can be inserted between two files

$paste -d “|” file1 file2 Inserting field delimiter with cut and paste if file does not have delimiter

$temp =/tmp/Kumar

$cut -c1-4 emp.lst > $temp.01

$cut -c5-10 emp.lst > $temp.02

:

:

$cut -c73-80 emp.lst > $temp.06

$paste -d “|” $temp??

Unix/cmd/31

sort Command $sort <file> …. Sorts entire line reorders a line in a ASCII collating sequence sorting can be done on a field delimiter can be specified, -t option

$sort -t “|” +1 emp.lst

+1indicates sort after skipping first field i.e.on 2nd field

sorting in reverse order -r option$sort -r+1 emp.lst

output filename can be specified as argument$sort -0 sortedf emp.lst

-c (check) option to check if sort successful. No message indicates sort successful char position can be specified

$sort -t “|” +4.6 - 4.9 emp.lst will sort 7th & 8th column of field 5

sorting on multiple fields$sort -t “|” +2-3 +1 file1

Unix/cmd/32

$sort -m file1 file2 file3

merges sorted files

Numeric Sort

$sort -n file1

Option - b ignores leading blank spaces

- d Alphanuemric sort

- f folds uppercase into lowercase

Uniq Command

removes duplicate records (-d option)

$uniq dept.lst

$sort dept.lst | unique - uniqlst

-u option if only unique records to be listed

-d if only one copy of repeated records to be selected

sort Command

Unix/cmd/33

nl Commandnl - line numbering command

$nl empl.lst

1 1001 …..

2 1002 …..

:

: width of line number can be specified

-w option -s (separator) can be specified -n leading zeros in line number

$nl -w2 -s “|” -nrz emplst

01 | 1001 .. Right justified

02 | 1002 ..

:

: -v option to specify starting line number -i option specifies increment

Unix/cmd/34

grep Command - Advanced Filter Searches for a given pattern in the files specified

grep <options> <pattern> <filename(s)>

Options

-c displays count of occurrences

-l displays list of filenames only

-n displays line number along with lines

-v displays all but the line matching pattern

-i ignores case

-h omits filename

-f fname reads expression/pattern from file

(egrep & fgrep option)

-x displays lines match in entirety

(fgrep option)

Unix/cmd/35

grep Command

e.g. $grep sales emp.lst

- lists records with sales as dept

$grep ‘sales’ emp.lst

$str = ‘sales’

$grep -c “$str” emp.lst

will count no. of directors in all emp?.lst

display will be for each file separate

e.g. Find out designation of youngest person

who is not a director

Unix/cmd/36

Regular Expressions Used to specify pattern specified in single quotesExpressionch* - 0 or more occurrence of char chch \ {m\} - matches m occurrence of char chch \ {m,n\} - matches min. m & max. n occurrence

of char chch \ {m,\} - matches min. m occurrence[pqr] - matches single char p, q or r[c1-c2] - match single char within ASCII range

specified[^pqr] - matches single char which is not p, q

or r\(ptn\) - matches pattern ptn enclosed in \(,\)

with the tag \n (n ranges 1 to 9) e.g. ` \ (.. \) \1`

Unix/cmd/37

Expression^ptn - matches pattern ptn at beginning of lineptn$ - matches pattern ptn at end of linee.g. Consider following R.E.(i) ‘[cC]ho [wu]dh*ury’(ii) ‘[7-9] …$’ char [7-9] followed by 3 more

chars rep by … at end of line(iii) ‘director.*[7-9]…$’(iv) ‘^[\t]*$’(v) ‘^ ……… 03’

.\{10\}(vi) [0-9] [0-9] [0-9] [0-9]

Q. Write R.E. to locate Sengupta & Dasgupta

Q. Write R.E. to locate people having the same month of birth as current month

Unix/cmd/38

egrep Command

egrep - Extending grep

extends grep pattern matching capabilities

Extending R.E. features

ch+ matches 1 or more char ch

ch? Matches or 1 char ch

exp1/exp2 matches exp1 or exp2 expression

(x1/x2)x3 matches expression x1x3 or x2x3

x1/x2 matches expression x1 or x2

e.g. egrep ‘(sen/das)gupta’ emp.lst

egrep ‘[aA]gg?[ar]+wal’ emp.lst

egrep -f pat.lst emp.lst

Unix/cmd/39

fgrep Command

fgrep - multiple string searching

only simple (pattern) string or group of strings can be searched

faster than grep and egrep

e.g. $fgrep -f pat.lst emp.lst

$cat pat.lst

sales

admin

Unix/cmd/40

Process Status $ps - give pid, tty, time, command options

-f - full from (additional - user-id parents pid,CPU time consumed, start timecumulative process time)

-u - details of only users processes-a - all processes details-l - detailed listing-e - system processes

Background Process $sort -0 emp.lst emp.lst &

$a.out & nohup (no hangup) - permits execution of process even if user has logged

off$nohup sort emp.lst &

sends output to nohup.out

Unix/cmd/41

Process Related Commands Wait

shell command waits for child process to complete $wait 138 - waits for background job with pid 138

Kill$kill 1005 (default signal 15)

kills job with pid 1005$kill -9 1005

sure killing of job$kill 0 - kills all background process

nice - job execution with low priority prefix the command with nice

$nice wc -l xxx$nice -15 wc -l xxx

reduce priority by 15 units

Unix/cmd/42

Editors

Three Types

Line Editor - ed, ex

Screen Editor - vi

Stream Editor - sed

Unix/vi/01

vi Editor

Three Modules :

:set showmode displays message when in input mode

CommandMode

exMode

Input command

Mode

i, I, a, A,r, R, o, O,s, S

<Esc>

<Enter>

sh

:

<ctrl d>or exit

Unix/vi/02

vi Editor - Save & Quit

From input mode ZZ

From input mode to command mode press <Esc> and from command mode

To Save :w

To Quit :q

To save & quit :wq

or :x

Unix/vi/03

vi Editor - Command Mode

Deletion

x - single char

4x - deletes 4 characters from cursor to right

dd - line delete

4dd - deletes 4 lines

Unix/vi/04

vi Editor - Navigation

Moving between lines

G command is its line address

e.g. 40G - move to 40th line

1G - move to beginning of file

G - move to end of file

Unix/vi/05

vi Editor - Other Features

Joining line

J - to join current line with next line

4J - to join 4 lines from current line

Marking Text

m <single char label> - to mark line

<single char label> - to locate marked

line

Repeating last Instruction - . (dot key)

Undo last Instruction - u

Reverse all changes made to current line - U

Unix/vi/06

Input Mode Commands

Command Function

i Insert text to left of cursorI Inserts text at beginning of linea Appends text to right of cursorA Appends text at the end of lineo Opens line belowO Opens line aboverch Replaces single character under

cursor with character ch (no<Esc>)R Replaces text from cursor to rights Replaces single character under

cursor with any number ofcharacters

S Replaces entire line

Unix/vi/07

Cursor Movement Commands

Command Function

h or backspace Moves cursor left

j Moves cursor down

k Moves cursor up

I or spacebar Moves cursor right

^ Moves cursor to beginning of first word

of line (no repeat factor)

O or l Moves cursor to beginning of line (no

repeat factor with O)

$ Moves cursor to end of line

b Moves cursor backwards to beginning

of word

e Moves cursor forward to end of word

w Moves cursor forward to beginning of

word

Unix/vi/08

Paging Functions

Command Function

<Control-f> Full page forward

<Control-b> Full page backward

<Control-d> Half page forward

<Control-u> Half page backward

<Control-l> Redraw page screen (no repeat

factor)

Unix/vi/09

Search and Repeat Commands

Commands Functions

/pat Searches forward for pat?pat Searches backward for pattern patn Repeats search in the same direction along

which the previous search was made (norepeat factor)

N Repeats search in a direction opposite tothat which the previous search was made(no repeat factor)

fch Moves cursor forward to first occurrence ofcharacter ch in the current line

tch Moves cursor forward onto but before firstoccurrence of character ch in the currentline

; Repeats search in the same direction alongwhich the previous search was made withf or t

, Repeats search in a direction opposite tothat along which the previous search wasmade with f or t

Unix/vi/10

A Few Specimen Operator-Command Combinations

Command Function

d$ or D Deletes from cursor to end of line5dd Delete five linesd/endif Deletes from cursor up to the first occurrence

of the string endif in the forward directiond30G Deletes from cursor up to line number 30df. Deletes from cursor to first occurrence of a

dotc0 Changes from cursor to beginning of linec$ or C Changes from cursor to end of line3cw or c3w Changes three wordscc Changes current linecG Changes from cursor to end of filey$ or Y Yanks from cursor to end of line3yw or y3w Yanks three words from cursor position5yy Yanks five linesy?case Yanks from cursor up to the first occurrence

of the string case in the reverse!Gsort Sorts all lines from current position to end of

file

Unix/vi/11

Command Function

:r note Reads file note into present cursor location:w saves buffer and remains in editing mode:w note1 Creates file note1 and writes buffer contents

to it:w! note1 Overwrites contents of file note1 with

contents of buffer:w>> note1 Appends buffer contents to file note1:x Saves buffer and quits editing mode:wq Saves buffer and quits editing mode:q Quits editing mode without saving:q! As above but after abandoning changes:e note1 Stops editing current file, and edits file

note1“e! note1 As above, but after abandoning changes

made to current file:e! Loads last saved edition of current file:e # Returns to editing the most recently edited

file:n Edits next file (on the vi command line):rew Rewinds file list to start editing from first file

specified in the command line100 Cursor at start of 100th line

Unix/vi/12

Set Options Used by ex/vi

Option Abbreviation Significance

autoindent ai Next line starts at the previous indented

level

autowrite aw Writes current file automatically when-

ever switching files with :n and escaping

to the shell with :sh

ignorecase ic Ignores case while searching for patterns

magic Treats the characters of the regular

expression set as special while searching

for patterns

number nu Displays line numbers on screen

showmatch sm Shows momentarily the mach to a ) & }

showmode Displays a message when vi is in input

mode

tabstop ts Sets tab for display (default : 8 spaces)

wrapscan ws Continues pattern search by moving to

the other end of a file

Unix/vi/13

Advanced Filter - awk Report writer named after Aho

WeinbergerKernigham

As powerful programming language awk pgm are slow It can access, transform and format individual fields in a record Syntaxawk <options ‘line specifier {action}’ <files>

I. Simple awk filteringawk ‘/director/ {print}’ emp.lst

line specifier Action

(selects line (as has an effect of for processing) printing all selected lines)

$0 variable stores entire line print print $0

Unix/awk/01

Advanced Filter - awk Line specifier and action option are optional, either of them needs to be

specified

line specifier not specified indicates all lines to be selected

{action} omitted indicates print (default)

fields are identified by special variable

$1, $2, ….; default delimiter is contiguous string of spaces

Explicit delimiter can be specified using -F option

e.g. awk -F “|” ‘/sales/{print $3, $4}’ emp.lst

Regular expression of egrep can be used to specify pattern

Line numbers can be selected using NR built-in variable

$awk -F “|” ‘NR ==3, NR ==6 {print NR, $0}’ emp.lst

Unix/awk/02

awk Command Output can be formatted using printf

similar to printf in ‘c’ but not a function

e.g. $awk -F “|” ‘/[aA]gg?[ar]+wal/{

>printf “%3d%-20s%-12s%d\n”, NR, $2,$3,$6}’

>emp.lst

Record no./

Line number

will be printed

, (comma) to be used as delimiter in printf

Logical and Relational operators

Logical operator &&, ||

$awk - F “|” ‘$3 ==“director” || $3 == “chairman”{

>printf “%-20s”,$2}’ emp.lst

== tries to find perfect match but string may have trailing spaces to overcome this ~

and !~ (match & negate of match) can be used with R.E.

Unix/awk/03

awk Commande.g.$awk -F “|” ‘$2~/director/||$~/g.m/{

>printf $0}’ emp.lste.g.$3 ~/^g.m/

Relational Operators : <, <=, ==, !=, >=, >, ~, !~e.g.$awk -F “|” ‘$6>7500 {printf “%20s”, $2}’

emp.lste.g.$awk -F “|” ‘$3 == “director” || $6>7500 {

>print $0}’ emp.lstNumber Processing

Computation on numbers can be done+, -, *, /, % operator available

no type declaration initialized to zero

e.g.$awk -F “|” ‘$3 == director” || $6>7500 {>kount = kount+1>printf “%3d%-20s\n”, kount,$2}’ emp.lst

e.g.valid expressionkount=kount+1kount++; ++kount

Unix/awk/04

awk Command -f option

awk program can be written in a separate file and used in awke.g. $awk -F “|” -f emp.awk emp.lst

single quoted contents are written in this file without quotes

BEGIN and END SectionFormat: (i) BEGIN {action}

(ii) END {action}(iii)

awk<options> ‘BEGIN {action} line specifier {action} END {action}’ <files>

Unix/awk/05

awk Command

e.g. $cat emp.awk

BEGIN {

printf “\n\t Employee details \n\n”

}

$6>7500{

# increment sr. no. and sum salary

kount++; tot+=$6

printf “%d%-20s%d\n”, kount, $2, $6

}

END {

printf “\n The Avg. Sal. Is %6d\n”, tot/kount

}

$_

&awk -F “|” -f emp.awk emp.lst

Unix/awk/06

awk Command Positional parameters and shell variable

(i) Method I

requires entire awk command should be in the shell script

to differentiate positional parameter and field identifier, the positional

parameter should be single quoted in awk program

e.g. $3 > ‘$1’

(ii) Method II

store entire command sequence in separate script file

shell can pass values to awk

e.g. awk pgm

$6 > mpay { … }

$awk - F “|” -f emp.awk mpay = 7800 emp.lst

assignment should be immediately before filename

Unix/awk/07

awk Command

e.g. List the details of employee drawing salary

greater than avg. salary

$awk - F “|” ‘$6>x’ x = `awk - F “|”’ ‘{x+=$6}

END {print x/NR}’ emp.lst` emp.lst

Unix/awk/08

nawk - getline Statement

awk uses i/p from standard input or file

nawk - i/p from standard input device & file possible (at beginning)

using getline statement

when getline is execute the contents of next line gets stored in

var.

e.g. getline x

getline x < “Emp.lst”

getline x < “/dev/tty”

Unix/awk/09

Built-in Variable

NR - Cumulative no. of records read

FS - The i/p field separator

OFS - The output field separator

NF - No. of fields in current record

FILENAME - The current i/p file

ARGC - No. of arguments in command

line

ARGV - List of organization

Unix/awk/10

Arrays Can handle single dimension array index can be anything even string no declaration required, initialized to zero automatically

e.g. totl[1] +=$6; tot[2]+=da; tot[3]+=HRA; temp[$3]

Functions

int(x) - Returns integer value of x

sqrt(x) - Returns square root of x

index(s1,s2)- Returns the position of string s2 in s1

length() - Returns length of the argument

substr(s1,s2,s3) - Returns portion of string of length - s3,

starting position s2 in string s1

split(s,a) - Split string s into array a optionally

returns no. of fields

Unix/awk/11

Interface with Shell Redirection > , piping | possible restriction - filenames, commands that follows these symbol must be in double

quotes unix commands can be run within nawk using system function

e.g. BEGIN {system (“date”)}

Control FLOW :

(I) if … then …if (conditions is true)

<statements>else

<statements>

(II) For statementfor (i=1; i <= 9; i++)

{

}

Unix/awk/12

Interface with Shell

(iii) for loop for arrays

for (k in array)

<commands>

(iv)While loop

while (condition is true)

{

}

(v) break and continue

Unix/awk/13

Shell

Shell - The agency that sits between user and

UNIX System

- much more than command processor

Bourne Shell - named after its founder Steve Bourne

- widely used

C Shell - product from Univ. of California, Berkeley

- advanced user interface and enhanced

features

Korn Shell - by David Korn of Bell Lab.

Executables in /bin directory

sh indicates - Bourne Shell

csh if present indicates - C Shell

ksh if present indicated - Korn Shell

Unix/sh/01

echo Command

Displays argument

e.g. $echo GOOD Morning

GOOD Morning

$echo \*

*

$echo ‘*’

* escape sequence to manipulate cursor position

\t : tab

\f : formfeed

\n : newline

\c : cursor immediately after argument

e.g. $echo ‘Enter Your Name : \c’

Enter Your Name : $_ accepts ASCII Octal values

e.g. $echo ‘\007’

<… beep hear…:

Unix/sh/02

Command Substitution

Command enclosed in backquotes (`), the shell executes the command first,

and the enclosed command text is replaced by the output of the command

e.g. $echo The date today is `date`

The date today is Fri 27 00:12:55 EST 1990

$echo The date today is; date

- issues command echo and date

sequentially

e.g. $echo “There are `ls | wc - l` files in current

directory”

$cal `date “+%m 19%y”`

Unix/sh/03

Shell Variables

User variables can be defined

no type declaration or initialization

Format : variable=value

Value is of string type

No space preceding and following =operator

Evaluated by prefixing variable name with $

multi word assignment must be quoted

e.g.1 $x = 37

$echo $ x

37

e.g.2 $msg = You have mail

have not found

e.g.3 $msg = ‘You have mail’

$echo $msg

You have mail

Unix/sh/04

Shell Variables

4. $msg = You\ have\ mail

$echo $msg

You have mail

5. $x = aa

$y = bb

$echo $x$y

aabb

6. $echo ‘Amount = $1000’

$echo Amount = \$1000

- both will display : Amount =$1000

$echo “Amount = $1000”

Amount=000

- here shell evaluated variable $1 as double

quoted

Unix/sh/05

Shell Variables

7. $p = `pwd`

$echo $p

8. $echo ${fname}

emp.sh

$echo ${fname}x

emp.shx

Unix/sh/06

Shell Variables System Variables

Set during - booting

- login

script executed during login time - profile

profile alters operating environment of a user

to display list of system variable $set

list of variable

PATH - list of dir. In order of precedence that have to be scanned for hunting

command

Home - dir. Named after Login name

IFS - string of char which are used as word separator in the command line

MAIL - mail directory

PS1 - Two prompts - primary and secondary

PS2 - (used by sed, awk)

TERM - Terminal type

Unix/sh/07

Shell Variables

DATA in the same file - The here document

< redirection symbol, redirects the standard input to come from a file

<< here document symbol signifies data is here rather than in a Separate file

e.g. $grep director <<mark - delimiter to

indicate end of

document

>1001 | Sanjay | g.m. | 5

>5001 | Rahul | director | 7

:

:

>mark

Unix/sh/08

Read Statement

Takes input from standard i/p can be used to read one or more variables

e.g. echo “\n Enter the Pattern : \c”

read patname

echo “\n Enter file name : \c”

read fname

grep “$patname” $fname

OR

echo “\n Enter pattern & file name : \c”

read patname fname

Unix/sh/09

Command Line Arguments

Shell procedures arguments in command Arguments are specified along with the name of shell procedure Arguments are assigned to special variables (positional parameters) $1 -

First parameters

$2 - Second parameters

:

: Special parameters also available

$0 - Name of executed command

$* - Complete set of positional parameters

$# - No. of arguments

$$ - PID of current shell

$! - PID of last background job

$? - Exit status of last command

$@ - Same as $* except when enclosed in double quotes

Unix/sh/10

E.g.1 if [$# -nc 3]

then

:

:

fi

2 $kill - 9 $$

3 grep “$1” $2

4 grep director emp.lst >/dev/null;echo$?

Exit Status indicates successful

1 indicates failure

2 indicates unreadable file

Command Line Arguments

Unix/sh/11

Conditional Execution Logical Operators && and ||

&& operator delimits two commands, the second command is executed only if first succeeds

|| operator delimits two commands, the second command is executed only if first fails

e.g. $grep `director` emp.lst && echo “patern found”

$grep `manager` emp.lst || echo “pattern not found”

Script Termination - exit statement

premature termination of program

control returned to the calling program

e.g. grep “$1” $2 || exit 2

echo “pattern found”

Return value

default zero

Unix/sh/12

if Statement Format :(i) if <condition is true>

then<execute commands>

else(execute commands>

fi(ii) if <condition is true>

then<execute commands>

fi(iii) if <condition is true>

then<execute commands>

elif <condition is true>then

<execute commands><…>

<…>else

<execute commands>fi

Unix/sh/13

if Statement Condition can be specified either using test or [cond] Relational Operator used

eq Equal to nc Not equal to gt Greater than gc Greater than or equal to lt Less than lc Less than or equal to

File related tests with test

-a <file> Trace if <file> exists

-f <file> if exists & regular file

-r <file> if exists & readable

-w <file> if exists & writable

-x <file> if exists & executable

-d <file> if exists & is dir file

-s <file> if exists & size > 0

Unix/sh/14

Test - String Comparison String tests used by test

n str True, if str not a null string z str True, if str is a null string S1 = S2 True, if S1 = S2 S1! = S2 True, if S1 S2 str True, if str is assigned and not null

Logical Operators a .AND. O .OR. ! Not

e.g. If test $x -eq $y

if [$x -eq $y]

e.g. If [! -f fname]

then

echo “file does not exists”

fi

Unix/sh/15

The Case Statement Format :

case <expression> in

<pattern 1> ) <execute commands> ;;

<pattern 2> ) <execute commands> ;;

<…>

<…>

esac

e.g.echo “\n Enter Option : \c”

read choice

case $choice in

1) ls -l ;;

2) ps -f ;;

3) date ;;

4) who ;;

5) exit ;;

esac

Unix/sh/16

The Case Statement

E.g.

Case `date | cat -d “ “ -fi ` inMon > <commands> ;;Tue > <commands> ;;

::

esac

E.g.

Case “$ans” in[yY] [eE] [sS] ;;

[nN] [oO] exit ;;*) “invalid option” ;;

Unix/sh/17

While Statement

Format :while <condition is true>do

<execute statements>done

e.g. while [$x -gt 3]do

ps -fsleep 5

done

e.g. while truedo

ps -fsleep 5

done

Unix/sh/18

While Statement

E.g. Script to edit, compile & execute a program

while truedo

cc $1case $? In

o) echo “Compilation Successful” echo “Executing a.out” a.out ; exit ;;

*) echo “Compilation Error” echo “Press <Enter> to edit” read pause vi $1 ;;esacdone

Unix/sh/19

Break & Continue Statement Continue : Suspends execution of all statements

following it, and switches control to top of loop for next iteration

Break : causes control to break out of the loop

e.g. Validation of designation codefname = desig.lstwhile echo “Designation code : \c”do

read desigcase “$desig” in[0-9] [0_9] if grep “^$desig” $fname

>/dev/nullthen

echo “code exists”continue

fi ;;*) echo “invalid code”

continue ;;esac

done

Unix/sh/20

Until Statement

Complement of while stm loop body is executed repeatedly as long as the condition remains

false

e.g. until false

do

ps -f

sleep 5

done

Unix/sh/21

for Statement

Format :for variable in listdo

<execute commands>done

e.g. for x in 1 2 3do

echo “The value of x is $x”done

e.g.for var in $PATH $HOME $MAILdo

echo “$var”done

e.g.for file in *.cdo

cc $filedone

Unix/sh/22