Chapter 5:The Shell

The Man in the Middle

In this chapter …• The command line

• Input, output, and redirection

• Process management

• Wildcards and expansion

• Builtins

Let us recall …• Shell is an interpreter

• Sits between the user and the kernel

• We’ll be using bash as our point of reference

The Command Line• It is what allows us to talk to the shell

• A command line is a string of commands and arguments ending with Enter

• Shell takes the command line, interprets it, then instructs the kernel

Syntax• command [arg1] [arg2] … [argn]

• Spaces separate command and arguments

• Arguments can be optional

• Some arguments are called Options, and are proceeded by one or more hyphens (aka switches or flags)

Do I Need Arguments?• Some commands work without any

arguments specified (ex., ps)

• Others need one or more

• If a command expects at least one argument, often it will display a usage message if you execute the command without any arguments

Tokens• Each space-delimited word in a command

line is called a token and are numbered from left to right starting at zero

• That means the command is token 0, first arg is token 1, etc.

• Token can be a filename, a string, a number

Options• An argument that modifies the effect or

execution of the command• Traditionally, options precede other

arguments• Most utilities use a single hyphen; some

(such as several GNU utilities) also use double hyphens

• Sometimes you can combine options together with a a single preceding hyphen

Command Line Examplels -R -l -h public_html

Could also be written as:

ls –Rlh public_html

LiStContents

list directories recursively

display long listing

human readable format for sizes

directory to perform this ls command

upon

Mechanics• Once enter is pressed, command line is

parsed by the shell

• Reads the first token and tries to find the executable or builtin associated with it

• Passes remaining tokens to the called program, doing expansion and manipulation as necessary

Where’s the program?• If not given absolute pathname, shell

searches PATH variable for location of first token

• Can’t assume PATH includes your current working directory

• If it can’t locate it, shell returns command not found

• If no execute permissions, returns access denied

Execution• Shell begins a new process and hands

control over to called program

• Shell then goes to sleep, and waits for called program to finish running and return control

• Program also passes its exit status to shell

Streams• Streams contain data (usually text)

• Three standard streams– Input– Output– Error

• Each stream can be associated with any number of things (remember, everything is a file!)

Standard Input• Contains information to be passed to a

program (a utility, a shell, etc)

• Most commonly is the keyboard

• Also commonly a file

• Because everything is treated as a file, programs never know exactly where input truly comes from

Standard Output• Contains output from a program

• Again, program never sure where it’s sending output to

• Most commonly the screen

• Could also go to a printer, a file, or directly into standard input (to another program)

Standard Error• Another output stream

• Contains error messages

• Usually also goes to screen

• You can redirect both standard out and standard error to different places

Fun with cat• cat displays the contents of files

• Try calling cat with no argument

• It takes input from standard input (instead of a file)

• Will echo back every line you type

• Hit CTRL-D (EOF) to quit

Redirection• Temporarily changes where standard

streams point to

• After doing a redirection, streams return to normal

• There are ways to permanently change streams – beyond our scope

Redirecting Output• Syntax: command [arguments] > output

• Output is usually a file, but can be anything (for example, a printer)

• The greater-than sign instructs the shell to redirect standard out to whatever is to the right

• If output exists, it will be overwritten – be careful!

Redirecting Output con’t• To prevent overwriting, we can set the

noclobber variable

• You can override noclobber using >|• To append to the end of a file, use >>• You can’t redirect output to a file you’re using

as input – the shell allocates the file for the output stream before calling the program

Redirecting Errors• Syntax: command [args] 2> errors• You’re redirecting the second output stream

(standard error)

• Standard output can also be written 1>• You can redirect both streams:

– find whizzbang > results 2> /dev/null

Redirecting Input• Syntax: command [arguments] < input• Input usually is a file

• Not all that useful if the command supports a filename for one of the arguments

• Can be handy if the command prompts for input … you can automate the process with a file filled with the answers

Pipes• Redirects the output of one program to be

the input of another

• Reduces the need for intermediary steps

• Handy for when you have several quick steps to perform upon some data

• Great for getting output and quickly filtering it (ie. command | grep search_string)

Getting Crazy• tee allows you to send output to two

different locations at once

• It sends output to a file, while still passing the output onto standard output

• Cascading tee commands can go on indefinitely

Running Jobs in the Background

• Thus far we’ve always run commands in the foreground

• Have to wait until it finishes before you can access the command line again

• To run in the background, place a & at the end of your command line before hitting enter

Background, con’t• Once you hit enter, it will display the job

number, and the process ID number (PID)

• You may resume issuing commands

• Once the job finishes, the job number and the word Done will appear on your screen

• Remember, to check on things, use jobs

Back and Forth• Recall that pressing CNTRL-Z suspends a

process, and it gives it a job number

• To start it again in the foreground, use: fg job_num

• To start it again in the background, use:bg job_num

kill Revisited• Recall we could do:

kill %job_num

• You can also:kill PID

• Forgot the PID? Just use ps• Won’t die? Don’t forget kill -KILL

Metacharacters• Special characters interpreted by the shell

• Also called wildcards

• Shell expands ambiguous file references into a list of files matching that criteria

• Process is called globbing or expansion

? Character• Used to represent any single character

• For example: foo?– Would match food, fool, foot– Would NOT match fooey, footsie

• Can be anywhere in the filenameEx: b??kk??pper

• On exception – doesn’t match leading periods (for hidden files)

* Character• Similar to ?, but matches any number of

characters

• This includes zero characters

• Ex: foo*– Would match food, foot, footsie, foo

• Also does not match leading periods

[ ] Characters• Used to match a list of possible characters

• Also known as a character class

• Ex: [aeiou].jpg– Matches a.jpg, e.jpg, i.jpg, etc

• Can also do ranges of letters / numbers– Ex: [0-9] [a-z] [A-Z]

[ ] Characters con’t• You can negate a character set by placing

a ! or ^ after the first bracket

• Ex: [^0-9]– Does not match the numbers 0-9

Metacharacter Tips• You can combine multiple metacharacters of

different types to get really specific

• The shell does all the expanding, not the program or utility

• Programs never see the metacharacters

• To let the program see them, you must quote them or escape them

Builtins• Commands that are built into the shell

• Shell does not create a new process to run a builtin

• Run very quickly

• Shell will still do metacharacter expansion for builtins

• We’ll cover them in greater detail in Ch 10