Embed Size (px)
Citation preview
UNIX Basics
1
Contents1.Introduction to Unix 03 - 11
2. Logging In and Basic Commands 12 - 26
3File system 27 - 45
4.managing files 46 - 55
5.File Access Permissions 56 - 68
6.vi Editor 69 - 78
7.Shell Basics 79 – 94
8. Standard I/O Redirection 95 – 103
9. Filters & some utilities 104 – 123
10. Controlling Processes 124 – 132
11. Shell Programming concepts 133 - 158
12. Labs 159 - 161
2
Introduction to Unix
Module 1
3
Objectives
Upon completing this module you should be able to understand the following:
what is an Operating System History of unix Operating system Unix Architecture More features of unix Unix Flavors Linux Flavors
4
What Is an Operating System?
• Interface between Users and the Hardware
• take care of Storage Management
• take care of I/O device management
5
1969 AT&T Bell Labs UNICS system -designed by Ken Thompson & Dennis Ritchie
1970 UNICS finally became UNIX
1973 UNIX rewritten in C, making it portable to different hardware
Mid 1970s University of California at Berkeley (BSD) contributed many important features like vi, C shell etc.
1982 AT & T came back and started commercial production- Editions ->Systems
Late 1980s AT & T released SVR4 unification of SV3.2,BSD,SunOs & XENIX
1991 Linux from Linus Torvalds
1990s POSIX Standard, MIT introduced X-Windows
History of the UNIX Operating System
6
UNIX Architecture – Kernel & Shell
Shell Kernel HardwareUser
7
UNIX Architecture – System Calls
Though there are over a thousand commands on the system, they all use a handful of functions, called system calls, to communicate with the kernel.
All unix flavors use the same system calls and are described in the POSIX specification.
8
• Hierarchical file system• Multi tasking• Multi user• The building block approach• Pattern matching (wildcard characters)• Toolkit(Applications, RDBMSs, Languages etc..)• Programming facility• Documentation
More Features of UNIX9
Flavors of UNIX
• Digital Unix
• IRIX
• SCO Open Server
• SCO UnixWare
• IBM -AIX
• HP - UX
• Sun - Solaris
10
Clone of UNIX (LINUX)
• Red Hat
• Calders
• SuSE
• Mandrake
• Debian
11
Logging In and Basic Commands
Module 2
12
Objectives
Upon completing this module you should be able to understand the following:
Logging In and Out Command Line Format The Secondary Prompt Online Manual Pages id Command who Command date Command cal, clear Commands passwd Command finger command
13
• Log in to identify yourself and gain access.
• Execute commands to do work.
• Log off to terminate your connection.
A Typical Terminal Session14
Logging In and Out
telnet 192.168.10.1
User:
Password:
exit or <ctrl>+<d> will terminate the login session
15
Syntax:
$ command [-options] [arguments]
1. Separation : mail –f newmail not mail – f newmail
2. Order: mail –f newmail not mal newmail –f
3. Multiple options: who –m –u or who –mu not who –m u
4. Multiple arguments: mail team1 team2 not mail team1team2
Command Line Format16
$ echo 'hi
>Good Morning‘
> is the default secondary prompt
The Secondary Prompt17
Content of the Manual Pages
NAME WARNINGSSYNOPSIS EXAMPLESDESCRIPTION AUTHORRETURN VALUE FILESERRORS SEE ALSOBUGS etc.
18
Syntax:
man –[k|X] keyword
in which X is the number of one of the manual sections
Examples: $ man ls Display the “ls" man page. $ man -k cat Display entries with keyword "cat". $ man passwd Display the "passwd" man page-Section 1. $ man 4 passwd Display the "passwd" man page-Section 4.
The Online Manual19
Syntax: id Displays effective user and group identification for session
Example: $id uid =303 (user3) gid=300 (class)
The id Command20
Syntax: who Reports information about users who are
currently logged on to a systemExamples: $ who root tty1p5 Jul 01 08:01 user11 tty1p4 Jul 01 09:59 user12 tty0p3 Jul 01 10:01
$ who am i user12 tty0p3 Jul 01 10:01
$ whoami user12
The who Command21
Syntax: date Reports the date and time
Example: $ date Fri Jul 1 11:15:55 EDT 2005
The date Command
22
Syntax: cal Reports the calendar of 2011 September(Current month)
Example: $ cal 8 2011 for Aug 2011 $ cal 2011 for the full calendar of year 2011
The cal Command23
Syntax: passwd Assigns a login password
Example: $ passwd
Changing password for user1 Old password: New password: Re-enter new password:
The passwd Command24
The clear Command
Syntax:
clear Clears terminal screen
25
fingerFinger: Displays information about the users currently logged on
Eg:
$ finger user1
Login name: user1 Directory: /export/home/user1 shell:/usr/bin/sh On since Sep 05 09:10:12 on tty1 No plan
26
File System
Module 3
27
Objectives
Upon completing this module you should be able to understand the following:
What Is a File System? File System structure Hard Link File Types Ordinary Files Directory Files Symbolic Links Device files The File System Hierarchy Basic Commands pwd, cd, ls, mkdir, rmdir, touch
28
What Is a File System?
collection of control structures and Data blocks that occupy the space defined by a partition and allow for the storage and management of data.
Data + Meta data
29
a
File System structure
Primary Superblock
Backup Superblock
Cylinder Group Block
Inode Table
Data Blocks
30
Hard Link
Allows a file to have more than one name and yet maintain a single copy on the disk.
All the links have the same inode number
Eg: $ ls –i f1
1113 f1
$ ln f1 f1.lnk
$ ls –i f1.lnk
1113 f1.lnk
31
a
File Types
Common File Types:
Ordinary files: regular files
Directory files: table of contents, that stores a list of files/directories within that directory
Device files: For every device there is a device file used by kernel to interact with the device.
Symbolic Link: Its link to other files
32
a
Ordinary Files
A regular file simply holds data.
Regular files can hold ASCII text, binary data, image data, databases, application-related data, and more.
33
a
Directory Files
Directory Inode Table
name inum # type mode links user group date size locd1 4 4 dir 755 2 user1 group1 Sep 5 9:30 512f1 10 10 file 644 1 user1 group1 Sep 5 9:45 12
34
a
Symbolic Links
contains the path of the file to which it links
Exists even after the source file is removed and is exactly similar to Windows shortcut
Syntax: ln –s soucefile linkfile
Eg: ln -s /f1 f1.lnk
Overcomes 2 limitations of Hard Link:1. possible across filesystems2. can link to a directory
35
a
Device files
A device file provides access to a device.
# cd /devices/pci@1f,0/pci@1,1/ide@3# ls -lbrw------- 1 root sys 136, 0 Apr 3 11:11 dad@0,0:a
Two types of device files:block device filescharacter device file
36
The File System Hierarchy
/
usr etc devices dev var tmp
bin sbin
ls who
adm sadmdsk rdsk
37
a
system directories
/ :root directory, top level directory
/bin : user commands like cp, cron, cmp etc..
/usr/sbin: admin commands
/dev : logical device files of all hardware devices
/devices: physical device files
/etc : System configuration files and user database
/tmp : to store temporary files
/usr : the binaries, shared libraries, shared documentation etc.
/var: stores the log files and dynamic files
38
a
File Path nameA sequence of file names, separated by slashes (/), that
describes the path, the system must follow to locate a
file in the file system
Absolute pathname (start from the /-directory):
Eg: /export/home/user1/file1
Relative pathname (start from the current directory)
./test1 (. = current directory)
../team03/.profile (.. = parent directory)
39
pwd - Present Working Directory
pwd prints the Current Directory
40
6
Syntax: ls [-adlFR] [pathname(s)]
Example: $ ls f1 f2 memo $ ls -F f1 f2* memo/ $ ls -aF profile f1 f2* memo/ $ ls memo f1 f2
ls - List Contents of a Directory41
cd Change Directory
Example:$ pwd/home/user3$ cd memo; pwd/home/user3/memo$ cd ../..; pwd/home$ cd /tmp; pwd/tmp
syntax:
cd [dir_name]
42
mkdir, rmdir – create & remove Directories
Syntax: mkdir directory
eg:
$ mkdir d1
syntax: rmdir dir
eg:
$ rmdir /export/home/user1
43
working with Multiple Directories
Create multiple directories simultaneously: $ mkdir -p dir1/dir2/dir3
Remove a directory and all its subdirectories
$ rmdir -p dir1/dir2/dir3
44
The touch CommandUpdates the atime and mtime of a file if it exists.
Else create an empty file
Example:
$ ls –l
-rw-r—r-- 1 user1 group1 100 Sep 5 09:30 test
$date
Mon, Sep 5 10:00 2005
$ touch test file1
$ ls -l
-rw-r—r-- 1 user1 group1 100 Sep 5 10:00 test
-rw-r—r-- 1 user1 group1 100 Sep 5 10:00 file1
45
Managing Files
Module 4
46
Objectives
Upon completing this module you should be able to understand the following:
File Characteristics cat more tail wc cp mv rm
47
File Characteristics
FileType
Permissions
Links
Owner
Group
SizeTimestamp
Name
$ ls -l -rw-r--r-- 1 user3 class 37 Jul 24 11:06 f1-rwxr-xr-x 1 user3 class 52 Jul 24 11:08 f2drwxr-xr-x 2 user3 class 1024 Jul 24 12:03 memo
48
Syntax: cat [file...] Concatenate and display the contents of file(s)
Examples: $ cat remind Your mother's birthday is November 29. $ cat note remind The meeting is scheduled for July 29. Your mother's birthday is November 29.
$ cat abc abc 1234 1234
cat - Display the Contents of a File
CTRL + D
49
Syntax:more [filename]... Display files one screen at a time
Example: $ more funfile . . . --funfile (20%)-- Q or q Quit moreReturn One more line Space bar One more page
more - Display the Contents of a File
50
Syntax:
tail [-n] [filename]... Display the end of file(s)
Example:
$ tail -1 test soon as it is available.
tail - Display the End of a File51
wc command
The wc command counts the number of lines, words and bytes in a named file:
syntax:
wc [-c] [-l] [-w] filename
Options:
-c counts the number of bytes
-l counts lines
-w counts words
Example:
$wc testfile
10 98 1000 testfile
52
Syntax:cp [-i] file1 new_file Copy a filecp [-i] file [file...] dest_dir Copy files to a directorycp -r [-i] dir [dir...] dest_dir Copy directories
Example: cp file1 d1 copies file1 to d1 directory cp file2 file3 create a copy of file2 as file3 in the same directory
cp - Copy Files53
Syntax:
mv [-i] file new_file Rename a filemv [-i] file [file...] dest_dir Move files to a directorymv [-i] dir [dir...] dest_dir Rename or move directories
Example:
$ ls -F $ mv note remind memof1 f2* memo/ note remind $ ls -F$ mv f1 file1 file1 memo/$ ls -F $ls -F memofile1 f2* memo/ note remind file2* note remind$ mv f2 memo/file2 $ mv memo letters$ ls -F $ ls -F file1 memo/ note remind file1 letters/$ ls -F memo file2*
mv - Move or Rename Files54
Syntax: rm [-if] filename [filename...] Remove files rm -r[if] dirname [filename...] Remove directories
Examples:
rm f1 removes the file f1 rm –r d1 remove the directory.
rm - Remove Files55
File Access Permissions
Module 5
56
Objectives
Access to files is dependent on a user's identification and the permissions associated with a file. This module will show how to Understand the read, write, and execute access to a file Permissions
ls (11, ls -l) chmod umask chown chgrp su setuid, setgid, sticky bit
57
File Permissions and Access
Access to files is dependent on a user's identification and the permissions associated with a file. This module will show how to Understand the read, write, and execute access to a filePermissions
ls (11, ls -l)chmodumaskchownchgrpsu
Determine what access is granted on a fileChange the file access Change default file access Change the owner of a file Change the group of a file Switch your user identifier
58
Who can access a File?
The UNIX system incorporates a three-tier structure to define who has access to each file and directory:
user The owner of the file group A group that may have access to the file other Everyone else
• The ls -l command displays the owner and group who has access to the file.$ ls -l-rw-r--r-- 1 user3 class 37 Jul 24 11:06 f1-rwxr-xr-x 1 user3 class 37 Jul 24 11:08 f2drwxr-xr-x 2 user3 class 1024 Jul 24 12:03 memo | | owner group
59
contents can be examined. contents can be examined.
contents can be changed. contents can be changed.
file can be used as a command.can become current working directory.
Types of Access
There are three types of access for each file and directory:
Read files: directories:
Write files: directories:
Execute files: directories:
60
Change Permissions (Symbolic Notation)
Syntax: chmod mode_list file... Change permissions of file(s)
mode_list [who[operator]permission] [ ,... ]
who user, group, other or all(u/g/a) operator + (add), - (subtract), = (set equal to) permission read, write, execute
Example:Original permissions: mode user group other rw-r--r-- rw- r-- r--
$ chmod u+x,g+x,o+x file or $ chmod +x fileFinal permissions: mode user group other rwxr-xr-x rwx r-x r-x
61
Change Permissions(Octal Notation) File and directory permissions can also be specified as an octal number:
Read Permission :4
Write Permission :2
Execute Permission:1
We can just add the numbers to specify the permission for each category
Example: 6 means read and write, 7 means read, write and execute
eg:
$ chmod 664 f1 will give read and write permissions for owner and group while only read for others
62
Default Permissions
The default protections for newly created files and directories are:
File -rw-r—r-- 644
Directory drwxr-xr-x 755
These default settings may be modified by changing the umask value.
63
umask - Permission Mask
Umask specifies what permission bits will be set on a new file or directory when created.
New Directory: 777 – 022 755 rwxr-xr-x
New File : 666 – 022 644 rw-r—r—
The default value of umask is set in /etc/profile. This can be changed for all the users or a particular user
64
chown - Change File Ownership
Syntax: chown owner [:group] filename ...
Example: $ id uid=101 (user1), gid=101 (group1) $ cp f1 /tmp/user2/f1 $ ls -l /tmp/user2/f1 -rw-r----- 1 user1 group1 3967 Jan 24 13:13 f1 $ chown user2 /tmp/user2/f1 $ ls -l /tmp/user2/f1 -rw-r----- 1 user2 class 3967 Jan 24 13:13 f1
Only the owner of a file (or root) can change the ownership of the file.
Changes owner of a file(s) and, optionally, the group ID
65
The chgrp CommandSyntax: chgrp newgroup filename ...
Example: $ id uid=303 (user3), gid=300 (class) $ ls -l f3 -rw-r----- 1 user3 class 3967 Jan 24 13:13 f3 $ chgrp class2 f3 $ ls -l f3 -rw-r----- 1 user3 class2 3967 Jan 24 13:13 f3 $ chown user2 f3 $ ls -l f3 -rw-r----- 1 user2 class2 3967 Jan 24 13:13 f3
66
su - Switch User IdSyntax: su [user_name]
Example: $ ls -l f1 -rwxr-x--- 1 user1 group1 3967 Jan 24 23:13 class_setup $ id uid=303 (user1), gid=300 (group1)
$ su – user2 Password: $ id uid=400 (user2), gid=300 (group1)
Change your effective user ID and group ID
67
Special File Permissions – setuid, setgid, sticky bit
setuid – changes the effective user id of the user to the owner of the program chmod u+s f1 or – chmod 4744 f1
setgid – changes the effective group id of the user to the group of the program chmod g+s f1 or chmod 2744 f1
sticky bit – ensures the deletion of files by only file owner in a public writable directory chmod +t f1 or Chmod 1744 f1
68
vi Editor
Module 6
69
Objectives
Upon completing this module you should be able to understand the following:
What Is vi? Starting and Ending a vi Session Cursor Control Commands Input Mode: i, a, O, o Deleting Text: x, dw, dd, dG Copying, moving and changing Text Searching for Text: /, n, N Find and Replace
70
• A screen-oriented text editor• Included with most UNIX system distributions• Command driven• Categories of commands include
– General administration– Cursor movement– Insert text– Delete text– Paste text– Modify text
What Is vi?71
Starting and Ending a vi Session
vi [filename] Start a vi edit session of file
Example:
$ vi testfile - If the file doesn’t exist, it will be created - Otherwise vi will open the existing file
All modifications are made to the copy of the file brought into memory.
:wq or :x or <shift-zz> write and quit
:w write
:q quit
:q! Quit without saving
72
Cursor Control Commands
$w,Wl<right-arrow>
<down-arrow>
j
L
<ctrl-f>
G
1G
<ctrl-b>
H
k
<up-arrow>
0 b,B h <left-arrow>
73
Input Mode: i, a, O, o
i will insert character at the present cursor positionI will insert character at the beginning of the linea will append character at the present cursor position A will append character at the end of the line
o will insert a blank line belowO will insert a blank line above
74
Deleting Text: x, dw, dd, dG
x deletes the current character
dw deletes the current word
dd deletes the current line
dG delete all lines to end of file, including current line.
With any of these you can prefix a number
Example: 3dd will delete 3 lines
d$ to delete to the end of the line
d0 to delete to the start of the line
75
Copying, moving and changing Text
yy copy the line
yw copy the word p will paste the yanked lines below
dw cut the word P will paste the yanked lines above
dd cut the line
r will overwrite the current character
R will replace all text on the right of the cursor position
cw will replace only the current word
76
Searching for Text: /, n, N
/director will locate for the first occurrence of the pattern ‘director’
n to locate the next occurrence
N to locate the previous occurence
77
Find and Replace
To find and replace the word director with member :
:1,$s/director/member/g
1,$ represents all lines in the file
g makes it truly global. g will ensure that all occuences in each line is replaced. Without g only the first occurrence of each line will be replaced.
78
Shell Features
Module 7
79
Objectives
Upon completing this module you should be able to understand the following:
Shell functionalities Commonly Used Shells BASH Shell Features Aliasing, Command History Re-entering Commands The User Environment Setting Shell Variables Variable Substitution Command Substitution Transferring Local Variables to the Environment Functions of a Shell before Command execution.
80
Shell functionalities
command execution environment settings variable assignment variable substitution command substitution filename generation I/O redirection Interpretive programming language
81
/bin/bash/bin/ksh/bin/sh/bin/csh/bin/rksh/bin/rsh/bin/tcsh/bin/zsh
Bash shellKorn shellBourne shellC ShellRestricted Korn shellRestricted Bourne shellAn amended version of the C shellThe Z shell
Commonly Used Shells82
• A shell user interface with some advanced features:
– Command aliasing– File name completion– Command history mechanism– Command line recall and editing– Job control– Enhanced cd capabilities– Advanced programming capabilities
BASH Shell Features83
Syntax: alias [name[=string]]
Examples: $ alias dir=ls $ alias mroe=more $ alias mstat=/home/tricia/projects/micron/status $ alias laser="lp -dlaser" $ laser fileX request id is laser-234 (1 file) $ alias displays aliases currently defined $ alias mroe displays value of alias mroe mroe=more
Aliasing84
• The shell keeps a history file of commands that you enter.• The history command displays the last 16 commands.• You can recall, edit, and re-enter commands previously entered.
Syntax: history [-n| a z] Display the command history.Example: $ history -2 list the last two commands
15 cd 16 more .profile
$ history 3 5 list command numbers 3 through 5
3 date 4 pwd 5 ls
Command History 85
Example: $ history 3 5 list command numbers 3 through 5 3 date 4 pwd 5 ls $ r 4 run command number 4 pwd /home/kelley
Re-entering Commands
• You type r c to re-enter command number c.
86
Syntax: envExample: $ env HOME=/home/gerry PWD=/home/gerry/develop/basics ... PATH=/usr/bin:/usr/contrib/bin:/usr/local/bin:\ /home/gerry/bin
The User Environment
• Your environment describes your session to the programs you run.
87
The PATH variable– A list of directories where the shell will search for the
commands you type The TERM variable
– Describes your terminal type and screen size to the programs you run
$ env ... PATH=/usr/bin:/usr/contrib/bin:/usr/local/bin $ TERM=70092
Two Important Variables88
.
Setting Shell Variables
.
.
.
Syntax: name=value
Example:
color=blue PATH=$PATH:/usr/ucb
89
$ echo $PATH/usr/bin:/usr/contrib/bin:/usr/local/bin$ PATH=$PATH:$HOME:.$ echo $PATH/usr/bin:/usr/contrib/bin:/usr/local/bin:/home/user3:.$ echo $HOME/home/user3$ file_name=$HOME/file1$ more $file_name<contents of /home/user3/file1>
Variable Substitution90
Syntax: $(command)
Example:
$ dt=date This will store the string date in the variable dt not the value
$ dt=$(date) This will do command substitution and store the result of date command in the variable dt
note: instead of $(date) we can also use `date`
Command Substitution91
.
$ echo $HOME/home/user3
$ envHOME=/home/user3….SHELL=/usr/bin/sh
$ setHOME=/home/user3PATH=/usr/bin:/usr/contrib/bin:/usr/local/bin….color=lavendercount=3dir_name=/home/user3/tree
$ unset dir_name
Displaying Variable Values92
Transferring Local Variables to the Environment
Syntax: export variable Transfer variable to environment
93
Functions of a Shell before Command execution.
Searches for a command Substitutes Shell variable values. Command substitution, I/O redirection Interpreted programming interface
94
Standard I/O Redirection
Module 8
95
Objectives
Upon completing this module you should be able to understand the following:
The Standard Files Input Redirection Output Redirection Creating a file with cat Error Redirection Combined Redirection Split Outputs
96
The Standard Files
Standard Input File (0) Keyboard
Standard Output File (1) Monitor
Standard Error File (2) Monitor
operators:
standard input redirection 0< or <
standard output redirection 1> or >
standard Error redirection 2>
97
Input Redirection
Default standard input
$ mail user1
subject: Hi
Test mail
<ctrl-d>
Redirect Input from a file: <
$ mail user1 < letter
98
Output Redirection
Default standard output:$ ls f1 f2 f3
Redirect output from a file: >$ ls > ls.out
Redirecting and appending output to a file: >>$ who >> who.out
99
Creating a file with catWhile normally used to list the contents of files, using cat with redirection can be used to create a file
$ lsfile1 file2 file3
Using redirection
$ cat > newfilefile created by using the output redirection<ctrl-d>
$ lsfile1 file2 file3 newfile
100
Error Redirection
Default standard error:
$ cat file1 file3
This is file1
cat: cannot open file2
Redirecting error output to a file: 2> (To append: 2>>)
$ cat file1 file3 2>err
This is file1
$ cat err
cat: cannot open file2
$ cat file1 file3 2> /dev/null
101
Combined Redirection
Combined redirects: $ command > outfile 2>errfile <infile $ command >> appendfile 2 >> errorfile <infile
Association Examples: $ command > outfile 2 >&1
note: This is NOT the same as above $ command 2>&1 >outfile
102
Split Outputs
The tee command reads standard input and sends the data to both standard output and a file.
Example:
$ ls | tee ls.save | wc –l
103
Module 9
Filters & other commands
104
Objectives
Upon completing this module you should be able to understand the following:
grep, sort, find ,cut , tr ftp paste command split command uniq command diff(Differential file comparator) Command cmp command file command tar (tape archive program) Restoring Files
105
grep
Search for lines matching specified pattern
syntax:
grep [options] pattern [file1 file2 …..]
Example:
emp.dat
Robin Bangalore
John Chennai
Rina Bangalore
$ grep Bangalore emp.dat
Robin Bangalore
Rina Bangalore
106
grep with Regular Expressions
grep ‘regular_expression’ file
Valid metacharacters
. Any single character
* Zero or more occurences of the preceding character
[aA] Enumeration: a or A
[a-f] Any one of the characters in the range of a through f
^a Any lines that start with a
z$ Any lines that end with z
107
grep options
-v print lines that do not match
-c print only a count of matching lines
-l print only the names of the files with matching lines
-n number the matching lines
-i ignore the case of letters when making comparisons
-w do a whole word search
108
sort command
The sort command sorts lines and writes the result to standard output:
$ sort [ -t delimiter] [+field[.column]] [options]
Options:
-d sorts in dictionary order. Only letters, digits and spaces are considered in comarisons
-r reverses the order of the specified sort
-n sorts numeric fields in arithmetic value
109
sort examples
$ cat animalsdog.2cat.4elephant.10rabbit.7
$ sort animalscat.4dog.2elephant.10rabbit.7
$ cat animals | sort +0.1rabbit.7cat.4elephant.10dog.2
$ cat animals | sort -t. -n +1dog.2cat.4rabbit.7elephant.10
110
find command
Search one or more directory structures for files that meet certain specified criteria
Display the names of matching files or execute commands against those files
find path expression
Examples:$ find / -name file1 ---search for files in whole system with name file1$ find / -user user1 –exec rm { } \; ---search for the
files owned by user1 and delete them$ find / -user user1 –ok rm { } \; ---search for the
files owned by user1 and delete them interactively
111
find options
-type f ordinary filed directory
-size +n larger than "n" blocks-n smaller than "n" blocks n equal to "n" blocks
-mtime +x modified more than "x" days ago-x modified less than "x" days ago
-perm onum access permissions match "onum"mode access permissin match "mode"
-user user1 finds files owned by "user1"
-o logical "or"
-newer ref searches for files that are newer than the reference file.
112
cut command
Used to cut fields from each line of a file or columns of a table
cut –d: -f1,5 /etc/passwdcut –c2 test second character of each line cut –c-2 test first 2 characters of each line
113
tr command
Used to substitute the values
tr [a-z] [A-Z] will convert each small letter to caps
cat test | tr [a-z] [A-Z] will convert the small letters to caps in display
tr –s “ “ will squeeze multiple space occurences of space into one
114
ftpThe ftp Command Syntax :
$ ftp hostname
get Gets a file from the remote computer.
put Sends a local file to the remote system
mget get multiple files from the remote system
mput Sends multiple files to the remote system
cd changes the directory in the remote system
lcd changes the directory in the local system
ls Lists files on the remote computer.
? Lists all ftp commands
help command Display a very brief help message for command.
bye Exits ftp.
115
The paste commandUsed to paste files vertically
eg. file1 file2 paste file1 file2
name address age name address age
ram mvm 29 ram mvm 29
raghu rjnagar 25 raghu rjnagar 25
(To rearrange f1 f2 f3 to f3 f2 f1, cut each field,put into file then paste)
By default paste uses the tab character for pasting files, but –d to specify one
eg. paste –d \| file1 file2
name address|age
ram mvm|29
raghu rjnagar|25
116
The split command
Used to split up the files:
This command breaks up the input into several equi line segments.
by defalult, split breaks a file into 1000 line pieces(or whatever is left)
split creates files like xaa,xab,xac …. xaz then xba, xbb …. xbz and … xzz . So there can be 676(26*26) files
split –5 chap (here the chap file is broken into files of 5 lines size)
split chap small (the file names will be smallaa,smallab …. smallzz)
117
The uniq command• used in EDP environment
• to clear the duplicate entries by mistakes
eg. dept.list 01|accounts|6213
01|accounts|6213
02|admin|5423
03|marketing|6521
03|marketing|6521
uniq dept.list 01|accounts|6213
02|admin|5423
03|marketing|6521
The input to uniq must be ordered data. So sort and send the data
sort dept.list | uniq -uniqlist (uniqlist is the output file)
118
The diff(Differential file comparator) Command
Analyzes text files
Reports the differences between files
diff [-options] file1 file2
119
The cmp command
$ cmp names names.oldnames names.old differ: byte 6, line 1
$cmp -l names names.old6 12 1517 102 1568 157 145.........cmp: EOF on names
120
The file commandThe file command tells the type of file
Example:$ file /usr/bun/vi/usr/bin/vi:executable (RISC system/6000) or object module
$ file c1c1: ascii text
$ file /usr/bin/usr/bin: directory
121
create a disk archive that contains a group of files or an entire directory structure
-c copy-x extract-t list [only one can be present at a time]-f to specify the device name
tar [–]cvf etc.tar /etc/*
tar (tape archive program)122
tar –xvf tarfile
Example: tar –xvf etc.tar
selective extractiontar –xvf etc.tar /etc/passwd
tar –tvf etc.tar will display the archive
Restoring Files123
Module 10
Controlling Processes
124
Objectives
Upon completing this module you should be able to understand the following:
Monitoring Process Controlling Processes Terminating Processes Kill Signals Running Long Processes Job Control
125
Monitoring Process
The ps command dislays process status information
$ ps -f
UID PID PPID ... TTY ... COMMANDjohn 202 1 ... tty0 ... -kshjohn 204 202 ... tty0 ... kshjohn 210 204 ... yyu0 ... ls -R /john 212 204 ... tty0 ... ps -f
126
Controlling Processes
Foreground Processes:
$ ls -lR / >lsout
Background Processes:
$ ls -lR / >lsout &
127
Terminating Processes
Foreground Processes:
ctrl-c Interrupt key, cancels a forground process
kill Sometimes the kill command is used to terminate foreground processes
Background Processes:
kill kill is the only way to terminate background processes
128
Kill Signals
Sig Meaning
01 hangup- you logged out while the process was still running02 interrupt- you pressed the interrupt(break) key sequence ctrl-c03 quit -you pressed the quit key sequence ctrl-\
09 Kill signal:The most powerful and risky signal that can be sentSignal cannot be avoided or ignored!
15 Termination signal(Default): Stop a processSignal can be handled by programs
129
Running Long Processes
The nohup command will prevent a process from being killed if you log off the system before it completes:
$ nohup ls -lR / >lsout & [1] 100
If you do not redirect output, nohup will redirect output to a file nohup.out
$ nohup ls -lR / & [1] 102Sending output to nohup.out
130
Job Control
jobs Lists all jobs
<ctrl-z> Suspends foreground task
fg %jobnumber Execute job in foreground
bg %jobnumber Execute job in background
stop %jobnumber Suspends background task
131
Job Control Examples
$ ls -R / > out 2> errfile & [1] 273
$ jobs [1] + Running ls -R / > out 2>errfile &
$ fg %1ls -R / > out 2>errfile<ctrl-z>[1] + Stopped (SIGTSTP) ls -R / >out 2>errfile &
$ bg %1% jobs[1] + Running ls -R / >out 2>errfile & $ kill %1[1] + Terminating ls -R / >out 2>errfile &
132
Module 11
Shell Programming Concepts
133
Objectives
Upon completing this module you should be able to understand the following:
echo & read commands Command Line Arguments The logical operators && and || The if Conditional Numeric comparison with test numeric, string and file comparison The case CONDITIONAL expr: Computation Command Substitution sleep & wait while loop, until and for loops
134
echo & read commands
echo used to display messages on the screen
read used to accept values from the users, make programming interactive
eg.
echo “Enter ur name “
read name
echo “Good Morning $name”
135
Command Line Arguments
Shell programs can accept values through
1. read [Interactive –used when there are more inputs]
2. From the command Line when u execute it[Non interactive- used when only a few inputs are there]
For eg. sh1 20 30 Here 20 & 30 are the command Line arguments.
Command Line args are stored in Positional parameters
$1 contains the first argument, $2 the second, $3 the third etc.
$0 contains the name of the file, $# stores the count of args
$* displays all the args
136
An example of Command Line args.
#! /bin/bash
echo “Program: $0”
echo “The number of args specified is $#”
echo “The args are $*”
sh sh1 prog1 prog2
137
The parameter $?
$? can be used to know the exit value of the previous command . This can be used in the shell scripts:
138
The logical operators && and ||
These operators can be used to control the execution of command depending on the success/failure of the previous command
eg.
grep ‘director’ emp1.lst && echo “Pattern found in file”
grep ‘manager’ emp2.lst || echo “pattern not found”
or grep ‘director’ emp.lst’ && echo “Pattern found” || echo “Pattern not found”
139
exit : Script Termination
exit command used to prematurely terminate a program. It can even take args.
eg. grep “$1” $2 | exit 2
echo “Pattern found – Job over”
140
The if Conditional
if condition is true
then
execute commands
else
execute commands
fi
else is not a must : if condition is true
then
execute commands
fi
141
An example of if
eg1: if grep “director” emp.lst
then echo “Pattern found”
else echo “Pattern not found”
fi
Every if must have an accompanying then and fi, and optionally else
142
Numeric comparison with test
test is used to check a condition and return true or false
Relational operators used by if
Operator Meaning
-eq Equal to
-ne Not equal to
-gt Greater than
-ge Gfreater than or equal to
-lt Less than
-le Less than or equal to
143
An example of test with numeric
echo “Enter age”
read age
if test $age –ge 18
then echo “Major”
else echo “Minor”
fi
144
[ ] short hand for test
echo “Enter age”
read age
if [ $age –ge 18 ]
then echo “Major”
else echo “Minor”
fi
145
if –elif :Multi-way Branching
if condition echo “Enter marks”
then execute commands read marks
elif condition if [ $marks –ge 80 ]
then execute commands then echo“Distinction”
elif condition elif [ $marks –ge 60 ]
then execute commands then echo “First Class”
else execute conditions else echo “Pass”
fi fi
can have as many elif , else is optional
146
test String comparison
test with Strings uses = and !=
String Tests used by test
Test Exit Status
-n str1 true if str1 is not a null string
-z str1 true if str1 is a null string
s1 = s2true if s1 = s2
s1 != s2 true if s1 is not equal to s2
str1 true if str1 is assigned and not null
147
file tests
File related Tests with test
Test Exit Status
-e file True if file exists
-f file True if fie exists and is a regular file
-r file True if file exists and is readable
-w file True if file exists and is writable
-x file True if file exists and is executable
-d file True if file exists and is a directory
-s file True if the file exists and has a size >0
148
The case CONDITIONAL
case used to match an expression for more than one alternatives and uses a compact construct to permit multi way branching
case expression in
pattern1) execute commands;;
pattern2) execute commands
esac
149
An example for case
echo “1. List of files 2. Processes of user 2. Quit”
echo “Enter choice”
read choice
case “$choice” in
1) ls ;;
2) ps –f ;;
3) exit;;
*) echo “Wrong Choice”
esac
150
case Matching Multiple patterns
echo “Do u want to continue (y/n) “
read answer
case “$answer” in
y|Y) echo “Good” ;;
n|N) exit ;;
esac
case can also use wildcards like [yY][eE]* or [nN][oO]
151
expr: Computation
Shell doesn’t have any compute features-depend on expr
expr 3 + 5
expr $x + $y
expr $x - $y
expr $x \* $y
expr &x / $y
expr $x % $y division gives only integers as expr
can handle only integers
152
Command Substitution
x=5
x=`expr $x +1`
echo $x it will give 6
153
sleep & wait
sleep 100 the program sleeps for 100 seconds
wait wait for completion of all background processes
wait 138 wait for completion of the process 138
154
while loop
while condition is true
do
execute commands
done
eg. x=1
while test $x -le 10
do
echo $x
x=`expr $x + 1`
done (while true will set infinite condition)
155
until : while’s complement
x=1
until test $x –gt 10
do
echo $x
x=`expr $x + 1`
done
156
For loopfor variable in list
do
execute commands
done
eg. for x in 1 2 3 4 5
do
echo “Value of x is $x”
done
157
The list in for loop1. List from variables
for var in $PATH $HOME $MAIL
do
echo $var
done
2. List from Wild cards
for file in *.c
do
cc $file –o $file{x}
done
3. List from positional parameters
for pattern in $*
do
grep”$pattern” emp.lst || echo “pattern $pattern not found”
done
4. List from command substitution
for file in `cat clist`
158
Review
Who shows all users on the unix machine.
Who am I shows the complete information about you.
Date shows the current date and time of the machine.
Id Shows the details of the user along with gid and groups.
Cpio it is the command which saves and restores file
Cpio –icvdum `usr2/*` < /dev/rmt0
How many shells?
Two types of shells are there Bourne shell & C shell
Bourne shell {sh, Ksh(korn), bash}
C shell {csh & tcsh(tenex/tops)}
History of shells
First introduced sh shell used for programming and csh shell for interactive use, then the ksh introduced having all the C shell’s interactive features into the bourne shell syntax’s and finally bash introduced with all C shell and Korn shell features.
Details in /etc/passwd
Guest:X:501:513:U-ENZEN\Guest,S-1-5-21-117609710-1677128483-725345543-501:/home/Guest:/bin/bash
Username:Password:UID:GID:UserID info:home directory:command shell
159
Review Cont.
Details in /etc/Shadow
Your password is stored in /etc/shadow file.
Your encrpted password is not stored in /etc/passwd file. It is stored in /etc/shadow file.
Details in /etc/group
wheel:*:0:root,rache
group name: group password: group GID: list of users in the group
Difference in /etc/profile, .profile & .bashrc
/etc/profile is managed by system administrator contains shell initialization information required by all users.
.profile is managed by you. You can add as much shell customization as you want. .profile is the users own login initialization .It is present in the root home directory of every user.
.bashrc file is an initialization file run by each interactive invocation of the bash shell
Viewing the file (ls, cat & wc)
Use ls, ls –a (for hidden files), ls –lrt, ls –F(for listing the items, whether it is file or directory, it show / for the directory), ls –d(to list only directories),
cat, cat –n(for numbering the lines), cat –b(it wont number the empty lines),
wc , wc –l(count the number of lines), wc –w(count the number of words),wc –m or –c(Count the number of characters).
160
Review Cont.
Copying, moving & Renaming (cp, mv & rm)
Cp(for files), cp –r(for directories) source destination
Mv source destination
Rm, rm –rf,
Absolute path & Relative path
There are two path define: Absolute path & Relative path
Absolute path always start from root directory whereas relative path can be described from the current directory also.
User permissions
Chmod rwx
Chmod go= *(block all other users to view ur files) or we can use chmod go-rwx.
Chmod go+rwx(allow users the full permission)
Chmod go-w <filename> (block other users writable permission).
Letters used permissions
U = owner, g = group, o = other, a = all, ‘+’ = adding permissions to file, ‘-‘ = removing permissions to file.
Difference between Process and a JOB
Every command we give on unix starts a new process.
Job can consist of multiple processes running in series or at the same time in parallel.
161
Review Cont.
Why exec command is used?
To change your shell interpreter completely.
Variables
Variables are the word that holds the value.
It is Fruit =Apple
Fruit is a variable and Apple is the value of the variable
Variable can be (a-z A-Z _ 0-9), but starts with (a-z A-Z _)
Array variables
Name[index]=value
Eg. rohit[0-9]=(sweet, innocent,smart,intelligent)
Accessing all items in an array variables:
${name[*]} & ${name[@]}
Read only Variables
To mark variable readonly by giving the readonly command.
Types of Variables:
Local, environment & Shell variables.
Local variables are present in the current instance of a shell, whereas environment variables are available to any child process of the shell. Both the variables are created by user. Shell variables are special variables that are set by shell and is required by shell in order to function correctly.
162
Review Cont.
Substitution
Filename Substitution
Using the wild cards (*) & (?).
Also use [a-z][A-Z][0-9]
To negate any character, for eg. -[!a]
Variable Substitution
${parameter:-word}(substituting the default value) – If the parameter is null or unset,word is substituted to parameter.
${parameter:=word}(Assigning the default value) – If the parameter is null or unset, parameter is set as value of word.
${parameter:?message}(Aborting due to variable errors) - If the parameter is null or unset, message is printed to standard error. This checks that variables are set correctly.
${parameter:+word} – if the parameter is set, word is substituted for parameter. The value of parameter does not change.
Command Substitution & Arithmatic substitution
For ex. Date=`date` , Users=`who | wc –l`
Quoting
Can be done in three ways
(\), (’) & (“) i.e. backslash, single quote and double quote.
Backslash – Any character following by backslash lose its special meaning
Single quote – All special character between the single quotes loses its special meaning.
Double quotes – it allows its special meaning.
163
Review Cont.
$? = saves the exit status of the last executed command
File Test
Options for files are
-b = if a block special file
-c = if a character special file
-d = for a directory
-e = if file exist
-f = for a regular file
-g = if has its sgid bit set
-h = for a symbolic link
-k = for sticky bit
-p = for a named pipe
-r = for readable file
-s = file size greater then zero
-u = if has its suid bit set
-w = for a writable file
-x = for executable file
-o = if owned by effective user ID.
164
Review Cont.
String Comparison
-n = string has nonzero length; -z = string has zero length
String1=string2; string1!=string2
Numerical Comparision
-eq, -ne, -lt, -le, -gt , -ge
Compound Expression
! expr = true if expression is false. The expr can be any valid test command.
Expr1 –a expr2 = true if both expr are true (Logical AND &&)
Expr1 –o expr2 = true if either expr is true (Logical OR || )
CASE Statement
Basic syntax,
Case <word> in
Pattern1)
List1;;
Pattern2)
List2;;
Esac
Here word is compared against each pattern. Pattern is a string of regular characters and wildcard characters
165
Review Cont.
Loops
Two main types of loops
While loop, until loop & for loop. Also one more type is there present in ksh and bash is select loop
While Loop
Basic syntax
While command
Do
List
Done
Here command is single command to execute, where list is a set of one or more commands to execute.
Nested while loop, validating user’s input.
Until Loop
Similar to while loop, it is used until a condition is true.
Basic syntax
until command
Do
List
Done
166
Review Cont.
For loop & Select loop
Basic syntax for “for”
For name in word1, word2 …. Wordn
Do
List
Done
Basic syntax for select
Select name in word1, word2 …. Wordn
Do
List
Done
Loop controls
Break & Continue
167
Review Cont.
Parameters
Special Variables
$0 – it defines the script name. the name of the command being executed.
$n – these variable corresponding to the arguments with which script was invoked.
$# – number of arguments supplied to script.
$* – All the arguments are double quoted.
$@ – All the arguments are individually double quoted.
$? – the exit status of last command.
$$ – the process number of the current shell.
$! – the process number of last background command.
Usage Statement
Usage : $0 [arguments] [file|directory]
Arguments & Options
Ls –aF fruit
-aF is option and (-aF and fruit is an argument)
Using basename
Syntax basename filename
Basename cmd takes the absolute or relative path and returns the file or directory.
168
Review Cont.
Option parsing using getopts
Syntax getopts option-string variable
Getsopts f:o:v option
F is infile, o is outfile, v is verbose.
Input/Output
Output to a terminal
Echo & printf
ECHO Embedding punctuation marks {*, (,),?}
Embedding variable substitution {$Home, $Fruit}
Embedding escape sequences {\n for new line, \t print a tab character, \c print a string without a default trailing newline}
Printf it is same as echo, it wont produce a newline
Syntax printf %-mn<string>
(-) is for left aligned, by default it is right aligned.
169
Review Cont.
Output Redirection
Command > file – to redirect the complete logs.
Command >> file – used for appending the logs in the file,previous data is not deleted.
Command | tee file – used to redirect o/p to a file and the screen
Input Redirection
Command < file , for ex. Mail [email protected] < Final_sheet
Here document, syntax command << delimeter {document}delimeter
For ex. Lpr << MYURLS {enter URL’s}MYURLS
Can be used with output redirection also, command > file << delimeter document delimeter
Reading User Input
Use the read command for taking the input from user.
Syntax read name
Exec Command
Exec n>file & exec n>>file. N is an interger
Redirecting STDOUT & STDERR in two separate files
Command 1> file1 2> file2
Command > file1 2> file2.
170
Review Cont.
Functions
It is created for reusing the commands.
Syntax name( ) { list all the commands}
Call the function using its name.
Text Editors
Head it used to list the top 10 files which are recently used
Ls –lrt |head <number of files>
Tail it is also same as head. But is can be used to view the logs of the file
Tail –f <filename>
Grep It is used for search.
-n for line number where the word is found, for ex. Grep –n ls /home/rohit
-I for case sensitive
-l to search a word in number of files.
-v to show the data except for that which is specified with –v option.
-c for counting the lines.
171
Review Cont.
Counting Words
tr, sort & uniq
TR It is used for replacing the strings. For ex cat file | tr ‘a-z’ ‘A-Z’
Tr –s, use to replaces consecutive occurrence of same with only one occurrence of the character
Sort it sort the data
Sort –rn is used to sort the output by numbers.
Sort –k is constructs the key in each line
Uniq it sorts only unique lines
For ex. # tr '!,"{}/=:$;()' ' ' <r4 | tr 'a-z' 'A-Z' | tr -s ' '| tr ' ' '\n' | sort | uniq -c
To replace the r4 file and then sort o tans then fibe unique files.
Uniq -c option is used for counting.
172
Review Cont.
File string Text using Regular Expression
Sed & awk
If the search is made for any word that is in the beginning use ^<word>
If the search id made for any word that is in the end use <word>$
SED p option for print, d for delete & s for substitute
For ex. sed ‘/pattern1/pattern2/p’ but for s { sed ‘s/pattern1/pattern2/}
For globally changing the setting use /g in the last.
Using multiple sed use the option {–e}
Miscellaneous Commands
Eval, : , sleep, find, exec, xargs, expr, bc
173
Labs
Lab:1 Write Script to see current date, time, username, and current directoryLab:2 Write script to print nos as 5,4,3,2,1 using while loop.Lab:3 How to write shell script that will add two nos, which are supplied as command
line argument, and if this two nos are not given show error and its usage.Lab: 4 Write script to print given numbers sum of all digit, For eg. If no is 123 it's sum
of all digit will be 1+2+3 = 6.Lab: 5.Write script to print given number in reverse order, for eg. If no is 123 it must
print as 321.
174
Labs contd.
LAB 4# 1) Input number n
# 2) Set sum=0, sd=0
# 3) Find single digit in sd as n % 10 it will give (left most digit)
# 4) Construct sum no as sum=sum+sd
# 5) Decrment n by 1
# 6) Is n is greater than zero, if yes goto step 3, otherwise next step
# 7) Print sum
#
if [ $# -ne 1 ]
LAB 5# Script to reverse given no
# 1) Input number n
# 2) Set rev=0, sd=0
# 3) Find single digit in sd as n % 10 it will give (left most digit)
# 4) Construct revrse no as rev * 10 + sd
# 5) Decrment n by 1
# 6) Is n is greater than zero, if yes goto step 3, otherwise next step
# 7) Print rev
175
Labs contd.
LAB:1echo "Hello, $LOGNAME"
echo "Current date is `date`"
echo "User is `who i am`"
echo "Current direcotry `pwd`“
LAB:2 i=5
while test $i != 0
do
echo "$i
"
i=`expr $i - 1`
done
176
Labs contd.
LAB:3if [ $# -ne 2 ]
then
echo "Usage - $0 x y"
echo " Where x and y are two nos for which I will print sum"
exit 1
fi
echo "Sum of $1 and $2 is `expr $1 + $2`"
177
Labs contd.
LAB:4 i=5
if [ $# -ne 1 ]
then
echo "Usage: $0 number"
echo " I will find sum of all digit for given number"
echo " For eg. $0 123, I will print 6 as sum of all digit (1+2+3)"
exit 1
fi
n=$1
sum=0
sd=0
while [ $n -gt 0 ]
do
sd=`expr $n % 10`
sum=`expr $sum + $sd`
n=`expr $n / 10`
done
echo "Sum of digit for number is $sum"
178
Labs contd.
LAB:5if [ $# -ne 1 ]
then
echo "Usage: $0 number"
echo " I will find reverse of given number"
echo " For eg. $0 123, I will print 321"
exit 1
fi
n=$1
rev=0
sd=0
while [ $n -gt 0 ]
do
sd=`expr $n % 10`
rev=`expr $rev \* 10 + $sd`
n=`expr $n / 10`
done
echo "Reverse number is $rev"
179
![UNIX Shell-Scripting Basics [Presentation2Go.com]_2](https://img.dokumen.tips/doc/110x75/5447ef1bb1af9ff9778b46eb/unix-shell-scripting-basics-presentation2gocom2.jpg)
