Upload
nagaaytha
View
508
Download
0
Tags:
Embed Size (px)
DESCRIPTION
UNIX
Citation preview
Unix Training Program
What is Unix? A multi-user networked operating system
“Operating System” Handles files, running other programs, input/output Looks like DOS…but more powerful The internet was designed on it, thus networking is an
intrinsic part of the system
“Multi-user” Every user has different settings and permissions Multiple users can be logged in simultaneously
Unix vs. Linux Age
Unix: born in 1970 at AT&T/Bell Labs Linux: born in 1992 in Helsinki, Finland
Sun, IBM, HP are the 3 largest vendors of Unix These Unix flavors all run on custom hardware
Linux is FREE! (Speech vs. Beer) Linux was written for Intel/x86, but runs on many
platforms
• Connecting: - Connect to the Unix boxes via telnet or ssh (ssh preferred) - After connection, you have a shell (command prompt)
• Shells: - Instead of clicking, you type long, cryptic commands - After login, you begin in your home directory - tab completion is nice
Logging in and home directories
The Command Prompt Commands are the way to “do things” in Unix A command consists of a command name and options called
“flags” Commands are typed at the command prompt In Unix, everything (including commands) is case-sensitive
[prompt]$ <command> <flags> <args>
localhost:~$ ls –l -a unix-tutorial
Command PromptCommand (Optional) flags
(Optional) arguments
Note: Many Unix commands will print a message only if something went wrong. Be careful with rm and mv.
Getting help with man man (short for “manual”) documents commands
man <cmd> retrieves detailed information about <cmd> man –k <keyword> searches the man page summaries
(faster, and will probably give better results) man –K <keyword> searches the full text of the man
pages
fiji:~$ man –k passwordpasswd (5) - password filexlock (1) - Locks the local X display
until a password is enteredfiji:~$ passwd
Directories In Unix, files are grouped together in other files
called directories, which are analogous to folders in Windows
Directory paths are separated by a forward slash: / Example: /homes/trainees/troo1
The hierarchical structure of directories (the directory tree) begins at a special directory called the root, or / Absolute paths start at /
Example: /homes/ Relative paths start in the current directory Example: troo1 (if you’re currently in trainees)
Your home directory “~” is where your personal files are located, and where you start when you log in.
Directories (cont’d) Handy directories to know
~ Your home directory
.. The parent directory
. The current directory
ls LiSts the contents of a specified files or directories (or
the current directory if no files are specified) Syntax: ls [<args> … ] Example: ls backups/
pwd Print Working Directory
Directories (cont’d further) cd
Change Directory (or your home directory if unspecified)
Syntax: cd <directory> Examples:
cd backups/unix-tutorial cd ../class-notes
mkdir MaKe DIRectory Syntax: mkdir <directories> Example: mkdir backups class-notes
rmdir ReMove DIRectory, which must be empty first Syntax: rmdir <directories> Example: rmdir backups class-notes
Files Unlike Windows, in Unix file types (e.g. “executable
files, ” “data files,” “text files”) are not determined by file extension (e.g. “foo.exe”, “foo.dat”, “foo.txt”)
Thus, the file-manipulation commands are few and simple …
Many use only 2 letters rm
ReMoves a file, without a possibility of “undelete!” Syntax: rm <file(s)> Example: rm tutorial.txt backups/old.txt
Files (cont’d) cp
CoPies a file, preserving the original Syntax: cp <sources> <destination> Example: cp tutorial.txt tutorial.txt.bak
mv MoVes or renames a file, destroying the original Syntax: mv <sources> <destination> Examples:
mv tutorial.txt tutorial.txt.bak mv tutorial.txt tutorial-slides.ppt backups/
Note: Both of these commands will over-write existing files without warning you!
Shell Shortcuts Tab completion
Type part of a file/directory name, hit <tab>, and the shell will finish as much of the name as it can
Works if you’re running tcsh or bash
Command history Don’t re-type previous commands – use the up-arrow to access
them
Wildcards Special character(s) which can be expanded to match other
file/directory names
* Zero or more characters
? Zero or one character Examples:
ls *.txt rm may-?-notes.txt
• The basics:• ls – lists your files• cp – copies a file• mv – moves (can be used to rename) a file• rm – remove a file• mkdir – makes a directory (synonymous to folder)• rmdir – removes a directory• cd – changes your directory• ps – list the current processes in the shells• kill – kill processes (-9 if you want to kill them violently)
Commands - Basic
Editing Text Which text editor is “the best” is a holy war. Pick one
and get comfortable with it. Three text editors you should be aware of:
pico – Easy! Comes with pine (Dante’s email program)
emacs/xemacs – A heavily-featured editor commonly used in programming
vim/vi – A lighter editor, also used in programming
Your opinion is wrong.
/ – root. The source of everything
/bin – Where executables (programs) are run
/sbin – executables generally of interest only to the super user
/home – where your “home directories” are
/tmp – same as c:\temp in windows. Just for temporary files
/var – system logs and other stuff (probably not intersting)
/usr – layed out like /, but with less system-critical stuff
/dev – has files that represent different devices on the system
/proc – has runtime system info (try: cat /proc/cpuinfo)
Unix File Hierarchy
Commands – Less Basic
•less basic:• ln – creating links (kind of like shortcuts, but they work right)• less, more – keeps your text from scrolling off your screen• man, info – gives you “help” on commands• grep – search in files for a string• find – search for a filename• finger – get some information on other users (like last login)• which – locate where a command is• pwd – print the current directory• diff – find the difference between two (text) files
• Wildcards: ? – matches a single character * – matches any number of characters [xy] – matches either x or y (note: you can use other letters too!)
• Redirection: > – pipes standard output (printf, stdout, cout) to a file < – pipes a file into standard input (scanf, stdin, cin) | – connects the output of one command to the input of another
Wildcards and Redirection
Ownership Files have two owners
Every file has exactly one user owner Every file has exactly one group owner
Everyone is a user Users are in at least one group
Processes have owners, too (known as an “id”) Every process has exactly one user id Every process has at least on group id
Users and groups are really just numbers with names Every username is mapped to a single numeric “uid” Every groupname is mapped to a single numeric “gid”
Who am I? Commands that tell you who you are:
whoami displays your username id displays your username and
groups
Commands that tell you who others are: finger [<name>] displays info for <name> id [<username>] displays info for <username>
Commands that change who you are: su <username> “switch user” to <username> login login as a different user
Types of FilesRegular FilesDirectoriesDevices
Character Devices Block Devices
Sockets and Named PipesSymbolic Links
Files and DirectoriesEverything is referenced via a fileDirectories
List of files & inodes “.” – Reference to the current directory “..” – Reference to parent directory Root (/) – “.” and “..” are the same
Device Files Character Devices
Transfer unit: byte Example: /dev/console
Block Devices Transfer unit: Group of bytes (block) Examples: /dev/hda
Device Numbers Major – Type of device Minor – Device number
Sockets & Named Pipes Enables communication between processes Socket
Processed must have a connection first Example: X Windows
Named Pipe Communication between unrelated processes FIFO Not used very often
Link Files Multiple names for same file Hard Link
Pointer to Inode Can’t cross partitions File removed when all links deleted
Symbolic (Soft) Links Pointer to file path name Dangling symlink – Real file which no longer exists
ln [-s] <real_file> <link_file>
Magic NumbersByte pattern at beginning of filePatterns listed in file called magic
RedHat: /usr/share/magicfile – Tests a file to determine type
Filesystem Test Magic Number Test Language Test
File Attributes
InodeInode
Permissions
Ownership
Time StampsChangeModificationAccess
File Size
Link Count
Pointers to data
Viewing File Attributes
File Type Permissions Link Count Ownership
File Size/Device # Modification Date File Name
ls –l: Long listing (includes attributes)
stat: Lists all attributes
File Type Attribute# ls -ld /home /etc/passwd /dev/console
crw--w--w- 1 root root 5, 1 Sep 27 11:27 /dev/console
-rw-r--r-- 1 root root 559 Sep 22 13:14 /etc/passwd
drwxr-xr-x 3 root root 0 Sep 26 10:42 /home
File TypeFile Type MeaningMeaning
- Regular File
d Directory
l Symbolic Link
b Block Device
c Character Device
p Named Pipe
s Domain Socket
Ownership
User Owner of file User names/UIDs defined in /etc/passwd
Group Organization of users accessing the file Group names/GIDs defined in /etc/group
# ls -ld /home /etc/passwd /dev/console
crw--w--w- 1 root root 5, 1 Sep 27 11:27 /dev/console
-rw-r--r-- 1 root root 559 Sep 22 13:14 /etc/passwd
drwxr-xr-x 3 root root 0 Sep 26 10:42 /home
Permissions
3 levels of access – Owner, Group, Other
# ls -ld /home /etc/passwd /dev/console
crw--w--w- 1 root root 5, 1 Sep 27 11:27 /dev/console
-rw-r--r-- 1 root root 559 Sep 22 13:14 /etc/passwd
drwxr-xr-x 3 root root 0 Sep 26 10:42 /home
OperationOperation FileFile DirectoryDirectory
Read Read file List files
Write Delete/Modify file Create/Delete file
Execute Run program Access file
Changing Access Users & Groups
chown [-R] user file… chgrp [-R] group file
Permissions chmod [-R] <op> file… Numeric: <op> = [#]### Symbolic: <op> = <who op perm>
Who: (u)ser (g)roup (o)ther (a)ll op: (+)add (-)remove (=)set Perm: (r)ead (w)rite e(x)ecute
Numeric/Symbolic Permissions
OctalOctal BinaryBinary SymbolicSymbolic
0 000 ---
1 001 --x
2 010 -w-
3 011 -wx
4 100 r--
5 101 r-x
6 110 rw-
7 111 rwx
Default Permissionsumask Shell Environment VariableDefines permissions to remove
NumericNumeric BinaryBinary Effective PermsEffective Perms
0 000 rwx
1 001 rw-
2 010 r-w
3 011 r--
4 100 -wx
5 101 -w-
6 110 --x
7 111 ---
Special AttributesSetuid (SUID) Bit
Run program with access of owner Symbolic: s Numeric: 4000
Setgid (SGID) Bit Run program with access of owner group Symbolic: s Numeric: 2000
Sticky BitPurpose
File: Force program to stay in RAM (obsolete)
Directory: Cannot remove file unless you own the file or directory
Symbolic: tNumeric: 1000Example: /tmp
Searching the Filesystem
find: Command line search tool Searches through directory hierarchy Search by any combination of file names and
attributes Display files or perform operations on them Examples:
find /var –mtime -1 find / -name core –exec rm –f {} \;
File Permissions Every file has three access levels:
user (the user owner of the file) group (the group owner of the file) other (everyone else)
At each level, there are three access types: read (looking at the contents) write (altering the contents) execute (executing the contents)
Strange Things There are three “strange” permissions:
setuid (run program as user owner) setgid (run program as group owner) text (stay in swap after executing)
Directories act differently “write” (creating/deleting files) “execute” (cd-ing to that directory) “setuid” (ignored) “setgid” (created files have same group
owner) “text” (deletion restricted to user owned files)
Examining Permissions A “long” ls listing shows file permissions:
[zanfur@odin zanfur]$ iduid=8774(zanfur) gid=100(users) groups=100(users),101(mp3)[zanfur@odin zanfur]$ ls -ltotal 524-rwxr-xr-x 1 zanfur users 512668 Jul 31 00:18 bashprw-r--r-- 1 zanfur users 0 Jul 31 00:24 fifo-rw-r--r-- 1 zanfur users 0 Jul 31 00:18 filedrwxr-xr-x 2 zanfur users 4096 Jul 31 00:16 normaldrwxrws--T 2 zanfur mp3 4096 Jul 31 00:14 shareddrwxrwxrwt 2 zanfur users 4096 Jul 31 00:14 tmpdrwxrwxr-x 2 zanfur www 4096 Jul 31 00:15 www[zanfur@odin zanfur]$ _
Permissions Listing Breakdown
Field Description
Valid Values
?---------
file type - (normal file), d (directory), p (fifo), b, c, s
-?--------
user read - (no read permissions), r (read permissions)
--?-------
user write - (no write permissions), w (write permissions)
---?------
user execute
- (no exec), x (exec), S (setuid), s (both)
----?-----
group read - (no read permissions), r (read permissions)
-----?----
group write - (no write permissions), w (write permissions)
------?---
group execute
- (no exec), x (exec), S (setgid), s (both)
-------?--
other read - (no read permissions), r (read permissions)
--------?-
other write - (no write permissions), w (write permissions)
---------?
other execute
- (no exec), x (exec), T (text), t (both)
What You Can Do With Permissions
Permission
File Directory
r (read) Read a file List files in …
w (write) Write a file Create a file in …
Rename a file in …
Delete a file in …
x (execute) Execute a file (eg shell script)
Read a file in …
Write to a file in …
Execute a file/shell script in …
Changing Ownership Changing user ownership:
The command is “change owner”:chown <username> <filename>
but, you can only do it if you are root so just copy it instead
Changing group ownership: The command is “change group”:
chgrp <groupname> <filename> but, you can only do it if you are in group <groupname>
and you must be the user owner of the file
Changing Permissions The “change mode” command:
chmod <level><op><permissions>[,…] <filename>
<level> string of: u, g, o, a (user, group, other, all)
<op> one of +, -, = (gets, loses, equals)<permissions> string of: r, w, x, s, t, u, g, o
(read, write, execute, set-id, text,same as user, same as group, same
as other), Examples:
chmod u+rwx,go-w foobarchmod g=u,+t temp/chmod u=rwx,g=rwxs,o= shared/
• Groups• In Unix, all users belong to at least one group. The idea is that each person in one group has a similar amount of access to the system.
• Permissions• Files and directories all have “permissions” associated with them.• Permissions tell the OS who can do what with your files and
directories. Permissions can be set for three categories, the user, the group and the rest of the world. The permissions are:
read, write, execute
Groups and Permissions - 1
To see the permissions on a file, do a ‘ls –l’ fiji:/u15/awong$ ls –l -r--r--r-- 1 awong ugrad_ce 17375 Apr 26 2000 rgb.txt -rw-r--r-- 1 awong ugrad_ce 17375 Apr 5 02:57 set10.csv drwxr-xr-- 1 awong ugrad_ce 1024 Jan 19 19:39 tests
Groups and Permissions - 2
Changing Permissions chmod [ugo]+[rxw] <filename> e.g. chmod u+w rgb.txt gives me permission to change rgb.txt
Changing Ownership chown <user>.<group> <filename> e.g. chmod awong.iuns rgb.txt makes rgb.txt owned by iuns group
-r--r--r-- 1 awong ugrad_ce 17375 Apr 26 2000 rgb.txt -rw-r--r-- 1 awong ugrad_ce 17375 Apr 5 02:57 set10.csv drwxr-xr-- 1 awong ugrad_ce 1024 Jan 19 19:39 tests
Groups and Permissions - 3
What they do: files: read – Allows you to read the file write – Alows you to modify the file execute – Allows you run the file as a script or binary program
directories: read – lets you get a directory listing of files write – lets you add or remove files from directory execute – lets you access files in the directory
user group other
• .login, .bash_login, .mylogin, .profile, .bash_profile One of these scripts get run at startup…. If you use bash, you should create a .bash_profile so that you know what is getting run.
• .logout, .bash_logout One of these gets run when you logout or quit a shell
• .cshrc, .mycshrc, .bashrc One of these get run you start a new shell (as opposed to logging in. An example would be when you start a new xterm). .cshrc and .mycshrc are run if you use tcsh and .bashrc is run if you use bash.
• .xsession This gets run if you login via xdmcp. This is the only thing that gets run. It must be set as executable (chmod u+x) to work.
Login Scripts (what happens at startup)
What is input/output redirection? Normally, a program’s standard output is
displayed on user’s terminal, and its standard input comes from the keyboard.
Redirecting the output of a program means asking the shell to put the program’s output (stdout [C++’s cout]) into a file.
Redirecting the input of a program means asking the shell to feed a file as the program’s standard input (stdin [C++’s cin]).
Note: redirection works with files.
Why redirect program’s output? You may want to save output and examine it
at your leisure: if the program generates a lot of output if the program takes a long time to run
You may want to present the output to another person
Having the program write to standard output may make the program simpler to write
Standard output vs Standard error By convention, “normal” output of a program
is sent to standard output (stdout [C++’s cout]), while debugging or error output is sent to standard error (stderr [C++’s cerr]).
How to redirect program’s output?
To redirect just the standard output:<program> > <FILE>
To redirect just the standard error:sh/ksh/bash: <program> 2> <FILE>csh/tcsh: ( <program> > STDOUT ) >& STDERR
To redirect both standard output and standard error:csh/tcsh/bash: <program> >& <FILE>sh/ksh/bash: <program> > <FILE> 2>&1
> vs. >>Both > and >> will create the output file, if it
doesn’t already exist If the file does exist, then:
Using > to redirect output will overwrite the output file:
ls > newlisting printenv > my_environment
Using >> to redirect output will append to the output file
cat ch1 ch2 ch3 > book cat ch4 ch5 ch6 >> book
Why redirect program’s input? To run the program repeatedly with the same
(or similar input) Having the program read from standard input
may make the program simpler to write.
How to redirect program’s input?
Simple!<program> < <FILE>
Examplesort < my_grades.txthead < really_long_book.txt
Piping Piping is connecting programs together by using
the output of one program as the input to the next. Syntax:
<program1> | <program2> | … | <programN> A simple example (view a sorted file-listing a page
at a time):ls | sort | less
Note: piping deals with the input/output of programs (that is, stdin, stdout, stderr)
Why piping? Because “the whole is bigger than the sum of its
parts. By combining Unix utilities in a pipeline, you can
build tools “on-the-fly” as you need them.
Piping examples How many .c files are in this directory?ls *.c | wc –l
What files were modified most recently?ls –t | head
What processes am I running?ps auxw | grep <mylogin>
Make an alias for the above, for other Linux boxen too:alias myps=’ ps auxw | grep `id –un`’
The cat utilityEspecially useful: cat
When used with pipes, “copies” stdin to stdout
cat can be used to redirect out of a pipe! Example: make a file containing currently running
processes ps aux | grep evgenyr | cat > my_processes
When used with a file, “copies” the file into stdout cat can be used to place a file’s contents into a pipe! Example: list all the items in a list in alphabetical ordercat my_grocery_list.txt | sort | uniq
Command substitution(aka “what’s up with the backquotes?”)
Command substitution means that the shell substitutes a command’s output for the command itself.
To get the shell to perform command substitution, enclose the command in backquotes (`)
Next stop: utilities (No, not electricity, water, and sewer)diff and patchgrepfind and xargs
diff and patch You have two versions of a file; how do you see
what’s changed between the two versions? diff shows the differences between two files; you’ll
want to use the –u or –c option to diff to produce output in human-friendly format:diff –u my_file my_file.orig | less
patch applies differences to a filediff –u my_file my_file.orig > my_patchpatch my_file < mypatch
my_file is now the same as my_file.orig
grep – search for patterns grep searches for patterns in texts:grep <string> <FILE>
grep uses regular expressions to describe the string(s) to search for
In a regular expression, most characters are treated as-is, but a few are magic Ordinary characters just represent themselves Magic characters do special things
grep’s magic charactersA few different kinds of magic characters:
Some characters “anchor” (parts of) a regular expressions to specific places in the string:^ - The beginning of a line$ - The end of a line
A dot (.) matches “any one character whasoever” (except for newline)
[ ] form a character class: [aeiou] – any single vowel [a-zA-z0-9] – Any single letter or digit [^0-9] – Any single character that isn’t a digit
Here, ‘^’means “not”
Even more of grep’s magic characters
Quantifiers say how many times the preceeding “thing” should be repeated:* means “zero or more times”? Means “zero or one time”
Frequently used grep options
-i : do case-insensitive search
-n : print line numbers
-v : print lines that do not match
-l : only list the files from which output would have been printed
grep examplesPrint all non-blank lines:grep –v ’^$’ my_file.txt
Print all the lines on which my_function is called (or declared):grep –n ’my_function *(’ my_code.c
Show all the xterms I am running:ps auxw | grep ”`id –un` .*xterm”
find
Traverse the tree(s) rooted at <PATHs> , and for each “thing” found, evaluate the <EXPRESSION> until the result of <EXPRESSION> is known.
The evaluation of <EXPRESSION> “short-circuits”, just like expressions in C/C++. false && some_expression can never be true
Options apply to the entire find command, not the individual expression
find <PATHS> <OPTIONS> <EXPRESSION>
Components of a find expression
An expression is zero or more primaries connected by operators
Two kinds of primaries: Tests: just return true or false Actions: do something, in addition to returning true or
false Operators work just as they do in C/C++.
! / -not -a / -and -o / -or , (comma)
Parentheses are used for grouping, just as in C/C++.
find examples Print all the *.c* and *.h* files in and below the current
directoryfind . –name ’*.[ch]*’ –a –print
Show line numbers myfunction calls in the above *.c* filesfind . –name ’*.c*’ | xargs grep –n ’myfunction.*(’
Change permissions on files under your home directory so they’re inaccessible to everyone but you (except for files in the www directory)cd; find . –path ”./www*” –prune –o –exec chmod go-rwx {} \;
How big are my *.c* files?expr `find –name ’*.c*’ –printf ”%k + ”` 0
Read the find manual (info find) for more examples
xargs: combine arguments
Feeds standard input as arguments to <COMMAND> Often used with find
(find … | xargs grep) But it doesn’t have to be used with find:cat filelist | xargs cat {} > concatenated
xargs <OPTION> <COMMAND> <INITIAL_ARGS>
Process ManagementWhat can you do with it?
Start programs in the background Run more than one program per terminal Kill bad and/or crashing programs Suspend programs mid-execution List all jobs running in a shell Move foreground jobs to the background More …
Three States of a Process Foreground
Attached to keyboard Outputs to the screen Shell waits until the process ends
Background, running Not attached to keyboard Might output to the screen Shell immediately gives you another prompt
Background, suspended Paused mid-execution Can be resumed in background or foreground
Background Processes Listing jobs:
jobs lists background “jobs” and job #’s ps lists processes and their process id %<job#> expands to the process id of the job
Stopping foreground jobs Press ^Z (Ctrl-Z) in the terminal window
Starting a process in the background Append a & character to the command line Examples: ls –lR > ls-lR.out &
xemacs my_program.cc & Resuming a stopped job
In the foreground: fg [<pid>] In the background: bg [<pid>]
Killing Processes The “kill” command:
kill [-<signal>] <pid>Send <signal> to process <pid>
The “killall” command:killall [-<signal>] <command>Send <signal> to all processes that start with <command>
Useful signals (kill –l for the complete list):TERM the default, “terminate”, kills things
nicelyKILL will kill anything, but not nicelyHUP “hangup”, used to reload configurationsSTOP stops (suspends) a running process
Programs & ProcessesProgram – file which can be run
Binary executables Shell scripts
Process – A running program Loaded in physical memory Stored in virtual memory (swap)
Virtual MemoryExtends amount of physical memoryUNIX
Size of swap = amount of usable memory Physical memory (RAM) mirrored in swap
Linux: Memory = swap + RAMSwapping
Moving pages to and from memory Page – block (unit) of RAM
Swap AllocationUNIX:
RAM:
Swap:
Linux:
RAM:
Swap:
Which works better when?
Process Components
Sleeping Stopped
Address space mapStatus
OwnerBlocked Signals PriorityResources used
Running Runnable
Zombie
Process OwnershipReal User (User starting the process)
UID: User ID Number GID: Group ID Number
Effective User (Permissions for process) EUID: Effective User ID Number EGID: Effective Group ID Number
Process LifecycleParent process “forks” a childProcess identifiers
PID: Process ID Number PPID: Parent Process ID Number
Child finishes Sends status message to parent Zombie – waits for parent acknowledgment Orphan – init (PID 1) becomes parent
Running a commandType in the command nameShell searched for the command If found, shell “forks” the commandHow is the command found?
PATH environment variablewhich – Displays full path to command Beware of “.” in your PATH
Background JobsForeground process
Returns control to shell after it finished Most common usage
Background process Returns control to shell immediately “&” typed after command name Runs concurrently
DaemonsSpecial programs which performs a
some task without interventionStarted in the backgroud
(often at system startup)Name of most ends in ‘d’
“Daemon” is actually a much older form of “demon”; daemons have no particular bias towards good or evil, but rather serve to help define a person's character or personality.
~Evi Nemeth
fg, bg, & jobsCan switch between modesSend signal to stop a processCommands
fg – Run process in foregroundbg – Run process in backgroundjobs – List shell child processes
SignalsUsed to communicate with processesTerminal signals
<Ctrl><c> - Terminate process<Ctrl><z> - Stop (suspend) process
kill [-signal] pid Sends signal to process PID Default is to terminate the process
Common Signals
## NameName DescriptionDescription
1 HUP Hangup
2 INT Interrupt
3 QUIT Quit
9 KILL KILL
15 TERM Software Termination
STOP Stop
Process PriorityHow much CPU time granted relative to
other processesnice – Sets the priority value at startrenice – Changes priorityRanges
SystemSystem RangeRange
Solaris 0 to 39
RedHat -20 to 20
System Load
Average number of “Runnable” processes
Measure of how busy the system isPerformance deteriorates at loads of 6+Good for creating a baselineuptime – Load average
(5, 10, 15 minutes)
Monitoring Processesps
BSD – Sorts by %CPU Usage SVR4 – Sorts by PID
top Full terminal screen display Sortable Can kill and renice processes
gtop – Gnome top
Information from ps PID USER PRI/NI - Priority/Nice level RSS/SIZE - Resident/Total Memory used STAT - Process State %CPU/%MEM - % of System CPU/Memory TIME - CPU time used Command
/proc Filesystem Pseudo filesystem Interface to memory-related kernel data
structures Each PID has it’s own directory
cwd – Symlink to current working directory exe – Symlink to program cmdline – command line options environ – command line environment stat – process status
Runaway ProcessesCan eat up your resourcesLists users and PIDs accessing a
resourcefuserlsof
Kill the process and clean up
• When you run a command, it starts a new “process” • Each process has a unique number called the PID (Proccess ID)• Unix is a multitasking operating system, so you can run multiple processes simultaneously• After you start a process, it is usually in what is called the “foreground.” That is, it takes over your shell.• You can suspend processes in the foreground with Ctrl-Z• The process is now frozen. You can pull it to the foreground again by typing “fg”.• Alternately, you can make it keep running, but put it in the background so can still use the same shell by typing “bg”.• You can also start a task in the background by putting a & at the end of the command.
Processes 1: suspend, background, foreground
• You can list all the processes that have been run from the current shell with “ps” • To list all the processes on the system, do “ps awux”
root 3723 0.0 0.1 3092 988 pts/20 S 16:21 0:00 –bashawong 3724 0.0 0.1 1406 712 pts/17 R 16.35 0:00 -bashawong 3725 0.0 0.1 2334 716 pts/17 R 16.36 0:00 ps awux
Processes 2: ps, kill, kill -9
If a process of yours freezes, you can login to the same machineagain, do a “ps awux” and find the process number and then kill it.
• If you want to end a process, you can do “kill <pid>” eg. kill 3724
• If that doesn’t kill it, you can do the “super kill,” “kill -9 <pid>” eg. kill –9 3724
• Sometime you want to know what process is taking up the most cycles on a machine. The program that tells you this is “top” • Another measurement utility for system load is “uptime.” uptime tells you how long the system has been running and how many processes are concurrently screaming for attention. The last three numbers tell you how the average number of processes screaming for attention over the last 1 minute, 5 minutes and 10 minutes, respecetively.
4:49pm up 1:15, 19 users, load average: 0.10, 0.11, 0.09
System State: uptime, top, nice
• If you are going to run a long, cpu intensive process, it is often a good idea to “nice” it. This puts it at a slighly lower priority so thatquick, interactive tasks (like checking e-mail) won’t get starved.To do “nice” a process, just do “nice <command as you would>”
What are environment variables? The environment is an array of strings of the formNAME=VALUE
Each process (program) has its own copy of the environment
Processes started by the shell get a (“deep”) copy of the shell’s current environment
Each process can examine and modify its own (and only its own) environment
The “meaning” of environment variables is merely a matter of established conventions
Viewing your shell’s environment To view a single environment variable’s value:
echo $<name> For example:
echo $HOMEwill display
/homes/iws/evgenyr
To view all environment variables at once: tcsh/csh/bash: printenv sh/ksh: set
Setting environment variables tcsh/csh:
setenv <name> <value> Example: setenv PRINTER ps329
bash/ksh: export <name>=<value> There is no space on either side of ‘=’! Example: export PRINTER=ps329
sh: <name>=<value>; export <name>
!
Appending to environment variables
Appending /uns/bin/ to your PATH bash/ksh: export PATH=$PATH:/uns/bin
tcsh/csh: setenv PATH=$PATH:/uns/bin
Prepending is similar: bash/ksh: export INFOPATH=/uns/info:$INFOPATH
tcsh/csh: setenv INFOPATH /uns/info:$INFOPATH
Common environment variables The “meaning” of environment variables is
purely a matter of convention. However, these environment variables are
quite common: PATH, INFOPATH, MANPATH, EDITOR, VISUAL, PAGER, HOME, MAIL, USER
The man page for a program will usually list the environment variables the program pays attention to.
Environment VariablesWhat are environment variables? Think of them as parametersthat get passed to your programs. All operating systems havethem. They give information about the context within which a program runs. The most commonly used environment variableis the PATH variable which tells the shell where to look for programs.
To create an environment variable in bash, type: export VARNAME=value
To create an environment variable in tcsh type: setenv VARNAME value
Don’t forget the export or the setenv, otherwise you create a “Shell variable” instead of an environment variable.
Useful shell features (a preview of upcoming attractions)
AliasesRedirecting input and outputCommand substitutionScripts
Shell as a work-saver: aliases Aliases: textual substitution, similar to
C-preprocessor macros. Syntax:
bash/ksh: alias <text>=’<replacement>’
tcsh/cshalias <text> ’<replacement>’
When you type <text>, the shell “substitutes” <replacement>
Typing alias by itself lists all your current aliases unalias <text> removes the alias Check the csh/tcsh man page for extra features
Alias examples Always print postscript files double-sided
tcsh/csh:alias lpr ’lpr –Zduplex’
bash/ksh: alias lpr=’lpr –Zduplex’
Collect a few tree-friendly options to enscript: tcsh/csh:alias print \ ’enscript –2rhB –SDuplex=DuplexNoTumble’
Line continuation character, just like in C/C++!
Shell as a work-saver: scripts Instead of typing the same series of commands over
and over again, put them in a file and have the shell execute the (commands in the) file!
The file must have execute permission The first line of the file should be:#! <YOURSHELL> (e.g., /bin/bash)
There must be no space (or any other character) between ‘#’ and ‘!’.
Whether to put space after ‘!’ is a matter of style Shell also has control flow, tests, etc. The shell
tutorial on the ACM web page goes into much more detail.
Why Learn Scripting?/bin/sh always thereUnderstand/Modify Existing System
Scripts RC Startup Scripts Daemon Startup Scripts
Automate repetitive tasksGreat for small tools
Writing/Debugging ScriptsKnow how to do job manually firstWrite in small (logical) partsTest each part before continuing
Print modified variables Print command lines to be executed Never work on production data
Comment code!!!
Starting ScriptsPassed as input to a shell
Example: /bin/bash mysript
Run directly from the command line First line of script (i.e: #!/bin/bash) Script must have execute bits set$PATH considerations
Commenting codeAnything following ‘#’ is considered a
comment and ignoredScript Header
Tells what the script does Common Entries:
• Author• Date written• Program name• Command line usage
• Purpose• Files used• Notes/features• Revision History
Argument Variables
VariableVariable PurposePurpose
$0 Name of the program
$1 - $9 Command line arguments
$# Number of Arguments
$* All arguments as 1 word
$@ Each argument is a word
Other Special Variables
VariableVariable PurposePurpose
$$ PID number of script
$!PID number of parent process
$?Exit status of script
(Checked by parent)
Basic Input
read variable [ < filename ]
Examples:read ANSread LINE < file
(only reads first line)
Basic output
echo [ -n ] [ -e ] “string”-n : No new line-e : Allow backslash options
Example: echo –n “Enter your name: “
Backslash Options
CharacterCharacter PurposePurpose
\a Alert (bell)
\b Backspace
\c Don’t display trailing newline
\n Newline
\r Carriage return
\t Horizontal Tab
\v Vertical Tab
\\ Backslash
\### ###: ascii value of character
Expression Evaluationexpr command
Math expressions ( +, -, *, /, % ) String expressions (match, substring, etc.) Logic expressions ( &, |, =, etc.)
Example: COUNT=`expr $COUNT + 1`
Flow ControlNeed to determine which commands to
run based on one or more conditionsUseful to catch errorsProvides flexibility in codeClassic if-then-else model
If-Then condition
Basic decision construct
if condition
then
commands
fi
String Tests
ExpressionExpression True if…True if…
-z string Length of string is 0
-n string Length of string is not 0
string1 = string2 If the 2 strings are identical
string1 != string2 If the 2 strings are different
string If string is not NULL
Numeric Tests
ExpressionExpression True if…True if…
int1 –eq int2 int1 equals int2
int1 –ne int2 int1 and int2 not equal
int1 –gt int2 int1 > int2
int1 –ge int2 int1 >= int2
int1 –lt int2 int1 < int2
int1 –le int2 int1 <= int2
File Tests
ExpressionExpression True if…True if…
-r file File exists and readable
-w file File exists and writable
-x file File exists and executable
-f file File exists and is a regular file
-d file File exists and is a directory
-h file File exists and is a symbolic link
-p file File exists and is a named pipe
File Tests (Con’t.)
ExpressionExpression True if…True if…
-c file File exists and is a character device
-b file File exists and is a block device
-u file File exists and is SUID
-g file File exists and is SGID
-k file File exists and sticky bit set
-s file File exists and size > 0 bytes
Logic Operators
ExpressionExpression PurposePurpose
! Negates the expression
-a AND
-o OR
( expression ) Groups the expression
If-Then-Else condition
if condition
then
commands
else
commands
fi
If-Then-Else If condition
if condition
then
commands
elif condition
then
commands
fi
Case condition
Perform a range of tests on a variable
case variable in
pattern1) commands ;;
pattern2) commands ;;
esac
*) matches all other cases
LoopsNeed to run block of code more than
onceModes
Repeat a set number of times Repeat while a condition is true Repeat until a condition is true
for loop
Repeat for each variable in list
for variable in variable_list
do
commands
done
while loop
Repeat until condition is false
while [ condition ]
do
commands
done
Reading Files
Read a file one line at a time
while read LINE
do
echo $LINE
done < $FILENAME
do-until loop
Repeat until condition is true
until [ condition ]
do
commands
done
Escaping out of loops Sometimes you need to exit loop early
Error conditions Special conditions
break command Example:
while [ 1 ]do
echo –n “Done? “read ANSif [ $ANS = “y” ]thenbreakfi
done
Other scripting languagesPerl
Very popular among sysadmins Many modules available
TCL/TKExpectPython
Intermezzo: quoting Problem: some characters are special to the
shell (*, ?), but you would like to pass those characters to the programs the shell runs as-is.
So you quote the character(s): A single character: by prefixing it with a \ (backslash) A string of characters: by enclosing them in
Single quotes: no characters are special. None Double quotes: some characters (notably $ and `) are still
special. You’ll need to prefix those with a backslash to pass to the program as-is.
Quoting examples List all the files in the current directory:ls *
List the file or directory called ‘*’ (or complain if it’s not there):ls \*
Print $HOME:echo ’$HOME’
Print the path to your home directory:echo ”$HOME”
Print `id –un`: echo ’`id –un`’
Print your login:echo ”`id –un`”
DebuggingThe ideal:
do it right the first time
The reality:
bugs happen
The goal:
exterminate, quickly and efficiently
Debugging Methods The best way: read the code
If you don’t understand it, it’ll happen again
Consistent use of error checking & handling most uncaught errors are not where they appear
to be
Examine the state at key points in the code wrap debug output with #ifdef DEBUG use a debugger, like gdb, to view the state
Efficiency in Debugging
The goal is to isolate the problem and fix it
Don’t try random things, looking for a solution if you don’t understand it…it’ll be back half the time, this takes an insanely long time you don’t learn anything from it
Look for the problem, not the solution figure out two points in code that the problem is
between, and close the gap from there once you know the line the bug is on, it’s easy
Breakpoints What are they?
A “break point”: a point in the code whereexecution pauses, or “breaks”, then continues
What are they for? To examine the state of the running program
How do you make them? modifying code to print information and pause use a debugger to interactively step through code
Debuggers
Advantages over the “old fashioned” way: you can step through code as it runs you don’t have to recompile to set a breakpoint you can examine the entire state of the program
call stack, variable values, scope, etc.
you can modify values in the running program you can view the state of a crash using core files
GDB, the GNU DeBugger Text-based, invoked with:
gdb [<programfile> [<corefile>|<pid>]]
Argument descriptions:<programfile > executable program file<corefile> core dump of program<pid> process id of already running
program
Example:gdb ./hello
Compile <programfile> with –g for debug info
Basic GDB Commands
General Commands:file [<file>] selects <file> as the program to debugrun [<args>]runs selected program with arguments <args>attach <pid>attach gdb to a running process <pid>kill kills the process being debuggedquit quits the gdb programhelp [<topic>] accesses the internal help documentation
Stepping and Continuing:c[ontinue] continue execution (after a stop)s[tep] step one line, entering called functionsn[ext] step one line, without entering functionsfinish finish the function and print the return value
GDB Breakpoints Useful breakpoint commands:
b[reak] [<where>] sets breakpoints. <where> can bea number of things, including a hexaddress, a function name, a linenumber, or a relative line offset
[r]watch <expr> sets a watchpoint, which will breakwhen <expr> is written to [or read]
catch <event> breaks on <event>, which can beused to catch many events, includingthe throwing and catching of C++exceptions
info break[points] prints out a listing of all breakpointsclear [<where>] clears a breakpoint at <where>d[elete] [<nums>] deletes breakpoints by number
Playing with Data in GDB Commands for looking around:
list [<where>] prints out source code at <where>
search <regexp> searches source code for <regexp>backtrace [<n>] prints a backtrace <n> levels deepinfo [<what>] prints out info on <what> (like
local variables or function args)p[rint] [<expr>] prints out the evaluation of <expr>
Commands for altering data and control path:set <name> <expr> sets variables or argumentsreturn [<expr>] returns <expr> from current functionjump <where> jumps execution to <where>
More and More GDB For the most complete and up-to-date
documentation on GDB, use the GDB help subsystem from within gdb. A close second is the GDB info node (info gdb).
There is also an emacs interface to GDB, which allows you to keep the source code open in a separate frame while debugging (documentation for how to use this is available in the GDB info node)
Find more info – on the web
http://www.faqs.org (comp.unix questions FAQ) http://www.google.com http://www.deja.com The support web page:
http://www.cs.washington.edu/lab/ about/ugradcomputing.html
http://www.cs.washington.edu/people/acm/tutorials/