Introduction to UNIX Documentation · Introduction to UNIX Documentation, Release 1.0 important for the course but to keep at the back of your mind because it means that anything

Introduction to UNIX DocumentationRelease 1.0

Oliver Beckstein

January 13, 2012

CONTENTS

1 Unix Basics 31.1 Help . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31.2 Navigating file system . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41.3 Copy, renaming, deleting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.4 Looking at files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Shell name generation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.6 Input/Output Redirection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.7 Pipelines . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81.8 Filter excercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101.9 Access rights (permissions) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 111.10 Other useful commands . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121.11 Unix variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 141.12 General Unix gotchas . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 151.13 Setting up an editor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2 vi/vim essentials 192.1 Moving . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.2 Quitting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.3 Modes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.4 Editing text . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 192.5 Undo . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.6 Saving file . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.7 More editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 202.8 More movement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.9 Searching . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 212.10 Search and Replace . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.11 More on inserting/replacing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 222.12 Operators and motions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.13 Indentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.14 Other stuff . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 232.15 Customization . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 242.16 HELP!! . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

3 Indices and tables 27

i

ii

Introduction to UNIX Documentation, Release 1.0

Contents:

CONTENTS 1


2 CONTENTS

CHAPTER

ONE

UNIX BASICS

We will talk to the computer using the text-based terminal also known as “the commandline”. On a Unix -like operatingsystems (typically used for high-performance computing) this is a very powerful way to interact with the computer.

On a typical Linux desktop system you open a “xterm” or “kterm” (or similar application) to get access to the com-mandline. On Mac OS X you open “Terminal.app” (in the Utilities folder in Applications).

• We use the “bash” shell bash (“Bourne again shell”, which replaces the Bourne shell sh). Bash is a very goodshell to write scripts in and to use on an every day basis. Pretty much everything written about sh also appliesto bash (including the original and very readable Introduction to the Unix Shell written by Steve Bourne in1978).

• Bash is available on most modern Unix-like operating systems; it is the default shell on Linux and Mac OS X.

• There are other shells out there (like csh and tcsh (“C-shells”) or the Korn shell ksh). We will not deal withthem. In particular, the C-shells should not be used for writing scripts, for many good reasons.

Command syntax: options and arguments:

command [-argument] [-argument OPTVAL] ... [--long-argument] [--long-argument=OPTVAL] [file ...]

• single-letter, can be combined

• long options

• arguments: if not supplied, often read from “standard input” and output to “standard output”

1.1 Help

Not all help functions described below are always available. Simply try them all. You need to learn to find out aboutcommands by yourself. This introduction can only point you in the right direction and give you hints at to what couldbe useful to you.

• built in help function:

command -hcommand --helpcommand -?command

• manual page (‘man page’):

man commandman -k search_phrase

This is where to look when someone tells you to RTFM.

3

http://en.wikipedia.org/wiki/Unix

http://darthtater.asurite.ad.asu.edu/unix/shell.html

http://www.faqs.org/faqs/unix-faq/shell/csh-whynot/

http://www.grymoire.com/Unix/CshTop10.txt

http://foldoc.org/RTFM


• help function of the shell:

help command

• info page (can be somewhat similar to man but can contain hyperlinks):

info command

And of course, you can always search on the web. Just make sure you understand which version of the commandis described. There can be huge differences. If in doubt, look if a command complies to the “POSIX standard”, alowest common denominator for various flavors of Unix (yes, there are many different Unix-like operating systemsout there!).

1.2 Navigating file system

See where you are and move around:

pwd

cd # cd to homecd ~ # cd to home: ~ stands for homecd directorycd . # cd to the current dir (mostly useful with -P/-L options)cd .. # cd to parent dircd - # cd to previous dir (try it out)

List files:

lsls directoryls -l # long formatls -a # hidden files -- try ’ls -la’ls -ld # directory information -- ’ls -l ~’ vs ’ls -ld ~’ls -R # recursive

Excercise

1. use ls:

ls ~/Documentsls ~/..ls -lR / # takes forever, cancel with CTRL+C (= ^C)

2. create directory space:

~/NAME/p01p01/tmpp01/editorpdbdocs

3. look at these special directories:

/ # "root" of the file system/bin # commands ("binaries") are stored here (needed at system boot)/usr/bin # standard commands stored here/Volumes # Mac OS X specific: disks (and your USB flash drive) appear here

4 Chapter 1. Unix Basics

http://en.wikipedia.org/wiki/File:Unix_history-simple.svg


1.3 Copy, renaming, deleting

• download from http://darthtater.asurite.ad.asu.edu/PHY598/01/ vim_basics.txt

• copy to your docs directory:

cp vim_basics.txt docs

• move file to editor directory:

mv vim_basics.txt p01/editor

• make a temporary copy:

cp docs/vim_basics.txt tmpcd tmpcp vim_basics.txt foo.txtmkdir tmp2cd tmp2mv ../foo.txt . # note ’.’ for current dir

• remove the copy:

cd ..pwdrm vim_basics.txt

• remove the tmp dir:

rmdir tmp2 # only empty dirs!## FAILS

rm -r tmp2 # deletes full dirs recursively (dangerous)

Warning: rm -r and rm -rf (“force”, even override permissions) are very dangerous!! It deletes everythingrecusively. It does not ask and it does not keep a backup (no “Trash”).

In general, Unix commands do what you ask them to. If they succeed they typically will not output anything (unlessit’s part of their job such as ls). Only if there’s a problem you’ll get a (terse) message.

1.4 Looking at files

Different ways to display a text file:

cat FILEless FILE # h for help, q for quithead FILEtail FILE

Try head -5 vim_basics.txt. For looking at log files of a running simulation, try

tail -f output.log

which will continuously update.

1.3. Copy, renaming, deleting 5

http://darthtater.asurite.ad.asu.edu/PHY598/01/


1.5 Shell name generation

“glob” patterns

• * means any characters (even zero):

ls *ls *.txt

ls /usr/bin/*grep

• ? means any single one character:

ls /usr/bin/?grep

[x-y] means a range:

ls /usr/bin/[a-y]grep

• Advanced: brace expansion

head_{a,b,42,xx}_tail --> head_a_tailhead_b_tailhead_42_tailhead_xx_tail

Can be useful when making a complicated directory layout, e.g. simulations for three systems S1,S2,S3 at fourtemperatures 273, 300, 310, 373 and 2 pressures (1 atm and 1000 atm) (3*4*2 = 24 directories):

mkdir -p {S1,S2,S3}/T={273,300,310,373}K/P={1,1000}atm

Will create

S1/T=273K/P=1atmS1/T=273K/P=1000atmS1/T=300K/P=1atmS1/T=300K/P=1000atm...

1.6 Input/Output Redirection

• commands read from standard input (stdin; by default, the terminal, i.e. the keyboard) and write to standardoutput (stdout):

stdin --> command ---> stdout

By default, standard output is printed to the screen.

(There’s also a second output channel, called standard error (stderr), which is used for error messages. Bydefault it is also sent to the screen.

• redirection operators:

command > file # create/overwrite output filecommand >> file # appendcommand < file # read contents of file into stdin

• Note on Unix philosophy: “Everything is a file”: a file is a file (of course) but whole disks are also files, theterminal is a file, memory can be treated as a file, a random number generator can appear as a file, ... Not overly



important for the course but to keep at the back of your mind because it means that anything you learn about“real” files can be applied in a wider context!

Excercise

Run

cat

and type something, using return to finish lines... what happens?

To end input, type CONTROL-D (press control and ‘D’ at the same time; often written “^D”) to end input. (Knowingthis is often useful.)

Note: To terminate a running command, use CONTROL-C (^C).

cat reads your input from stdin (keyboard) and writes it to stdout (screen).

example: cat as simple “editor”:

cat > TODO- learn Unix- learn vi^D

less TODO

example:

mkdir p01cd p01ls -R ~/Documentsls -R ~/Documents > Documents.lsR

less Documents.lsR

wc Documents.lsR

cat Documents.lsR > double1.lsR # create/overwritecat Documents.lsR >> double1.lsR # appendwc double1.lsR

or

cat Documents.lsR Documents.lsR > double2.lsRwc double2.lsR

wc can also read from stdin:

wc < double1.lsR

What’s the difference?

Excercise:

1. find out how to only print the number of words with wc

2. create a new file with this number at the top and the remaining Documents.lsR following

solution:: wc -w < Documents.lsR > Documents.NlsR cat Documents.lsR >> Documents.NlsR

or

1.6. Input/Output Redirection 7


(wc -w Documents.lsR; cat Documents.lsR) > Documents.NlsR

( command; command; ... ) runs a sequence of commands in a sub-shell whose output can again be redi-rected.

1.7 Pipelines

“|” is the “pipe” character. It connects stdout from one command with stdin from another one:

command1 | command2

output of 1 is input of 2 (filter):

ls ~/Documents | wc

One of the power of Unix comes from the fact that a Unix system contains many small programs that do one jobparticularly well and which can be strung together as filters in a pipeline.

Useful filter commands:

1.7.1 grep

(“get regular expression”)

Shows lines that match the expression REGEX:

command | grep ’REGEX’

Note: It is generally a good idea to enclose REGEX in “hard quotes” (single quote character “”’) so that the shelldoes not interprete special characters such as $ or ~.

Simple REGEX (“basic regular expressions”):

word matches "word" literally anywhere^word matches "word" at beginning of lineword$ matches "word" at end of linea *b matches ab, a b, a b, i.e. ’ *’ is zero or more

spacesa \+b matches a b, a b, ..., i.e. ’ \+’ is one or more

spaces (NOTE: in "extended regular expressions"as used in ’egrep’ this is just ’+’, i.e. ’a +b’)

a[A-Z]b matches aAb, aBb, ..., aZb (range expression)a[0-9][0-9]b a00b, a01b, a02b, ..., a99ba[A-Z]*b ab, aAb, ..., aZba[A-Za-z]b aAb,..., aZb, aab, ..., azba[Â-Z]b aab, axb, ab, a+b, ... ([^...] is a negation)a.b matches aXb a3b a_b a b but not ab: ’.’ stands for

a single charactera...b a123b aXYZb etc: ... are three charactersa.*b ab a1b a12b a123b etc: .* is zero or more characters

(this is used very often)a.\+b a1b a12b but not ab: .\+ is one or more characters

(Regular expressions are amazingly useful but it takes some time to learn them. See ‘man re_format’ for the barebones and various tutorials on the internet. The above barely scratches the surface.)



Examples:

ls /usr/bin | grep lpls /usr/bin | grep ^lp

Excercise

How many lp commands?

ls /usr/bin | grep ^lp | wc -l

1.7.2 sort

alphabetical or numerical sort, e.g.

who | sort

1.7.3 uniq

cat FILE | sort | uniq

(note: uniq -c: histogram)

1.7.4 cut

cut -c N-M,X-Y FILE --> data from cols N-M and X-Ycut -f 2,3 -d ’ ’ --> separate fields by space and print 1 and 2

(But for field splitting, awk works better (see below).)

1.7.5 sed

“stream editor”: reads a file line by line and applies a sed-program to each line in turn. It is rather complicated and atypical use is to search and replace in a file:

cat FILE | sed ’s/SEARCH/REPLACE/g’

where SEARCH is a “basic regular expression” as for grep.

Warning: sed sed-program FILE > FILE will destroy FILE. You must redirect to a temporary file, e.g.sed sed-program FILE > FILE.temp && mv FILE.temp FILE. Modern versions of sed have the-i (inplace) option to take care of that.

1.7.6 awk

awk also scans a file line-by-line and applies an awk program to each line. It is also fairly complicated (actually, awkis a full blown scripting language) but typical use is straight forward: awk splits the line into fields (separated by whitespace (i.e. space, tabs) and then allows you to access fields by the special variables $1 (first field), $2 (second field),etc. For most data files you can think of fields as columns.

1.7. Pipelines 9

http://sed.sourceforge.net/sedfaq.html


cat FILE | awk ’/REGEX/ {awk-command; awk-command; ...}’

e.g.:

ls -lR /usr/bin | awk ’/grep$/ {print $9, $5/1024}’

prints the file name and the size in kB instead of bytes but only of those commands that end in grep.

1.8 Filter excercises

Download 1AKE and 4AKE from the PDB (Protein Databank). Look at the files with less.

• search with the PDB code (e.g. “1ake”)

• download file (Files -> Download Files -> PDB File (gz))

• Or from the command line:

curl http://www.rcsb.org/pdb/files/1ake.pdb.gz -o 1ake.pdb.gzcurl http://www.rcsb.org/pdb/files/4ake.pdb.gz -o 4ake.pdb.gzgunzip *.gz

(curl means “cat URL”, i.e. by default it writes the file pointed to by URL to stdout — ready to be used in apipeline.)

Or (if wget is installed):

wget http://www.rcsb.org/pdb/files/{1ake,4ake}.pdb.gzgunzip 1ake.pdb.gz

Put files into your PDB dir.

Look at files with less and recognize the PDB file format.

We are particularly interested in the Coordinates Section where individual atoms are listed together with their coordi-nates. Move to the ATOM and HETATM sections (use / (e.g. /ATOM) to search; press n repeatedly to move forwardthrough the matches.).

0. count the number of residues 1

Hint: each residue has exactly one CA atom; protein residues are stored with ATOM records. Other moleculesare in HETATM.

Bonus: How many residues in each chain?

Solution: manual inspection of the file showed that there are only two chains, A and B, so we simply grep forthose separately:

grep ’ÂTOM.*CA.* A ’ 1ake.pdb | wc -lgrep ’ÂTOM.*CA.* B ’ 1ake.pdb | wc -l

214 (same for all)

Total:

grep ’ÂTOM.*CA’ 1ake.pdb | wc -l

4281 A protein is a polypeptide that is made up from a linear sequence of amino acids; each amino acid is called a residue. There are 20 natural

(and frequently) occuring amino acids. Each has a three-letter residue name. For instance, glycine is Gly, arginine is Arg, and Glutamine is Gln.


http://www.pdb.org

http://www.wwpdb.org/documentation/format33/v3.3.html

http://www.wwpdb.org/documentation/format33/sect9.html

http://www.wwpdb.org/documentation/format33/sect9.html#ATOM

http://www.wwpdb.org/documentation/format33/sect9.html#HETATM

http://www.ncbi.nlm.nih.gov/Class/MLACourse/Modules/MolBioReview/iupac_aa_abbreviations.html


1. histogram of residue names: how often does each amino acid occur in the protein? Are some rarer than others?

• find the CA

• use cut -c N-M to extract the name from the fixed (not white-space separated!) columns (check PDBATOM specs for N-M)

• use sort and uniq -c

Solution:

cat 1ake.pdb | grep ’ÂTOM.*CA’ | cut -c 18-20 | sort | uniq -c

Question: How to some up the totals?

cat 1ake.pdb | grep ’ÂTOM.*CA’ | cut -c 18-20 | sort | uniq -c \| awk ’{sum+=$1}; END {print "total: ", sum}’

1.9 Access rights (permissions)

Take a detailed look at the output of a long file listing:

ls -la ....

drwxr-xr-x 2 oliver oliver 68 Jan 11 02:34 tmp-rw-r--r-- 1 oliver oliver 495559 Jan 11 13:56 Documents.lsRuuugggooo owner group size date name

d = directory (see man ls for the full list)r = readw = writex = execute

Fields:u = user/owner = oliverg = group = olivero = other

Additionally, after the permissions there can also be a single character that shows if alternative access controls (suchas Access Control Lists) are applicable. This is typically signified through a + sign. The ls -l command on MacOS X also shows information about extended attributes (@), i.e. there exists meta data stored in the file system. Thisis only of concern if you copy the file to a non-Mac OS X formatted disk or USB flash drive because the “foreign”filesystem will not be able to store these extended attributes.

1.9.1 chmod

change permissions:

chmod go-rwx FILE # make it fairly privatechmod go+r FILE # let others read itchmod a+r FILE # let everyone read it

Excercise:

1. remove execute permission from tmp dir. Try ‘cd tmp’ and ‘ls tmp’

2. Fix the permissions so that everthing works again.

1.9. Access rights (permissions) 11

http://www.wwpdb.org/documentation/format33/sect9.html#ATOM


1.10 Other useful commands

1.10.1 df — file system information

(“display file system”):

df

1.10.2 du — file use in directories

(“disk usage”):

du -s DIR

1.10.3 compression

compress (shrink file size without loosing information):

zipgzipbzip2 (may not be installed)

uncompress:

unzipgunzip, zcatbunzip2

(These commands can typically be asked to take either a FILE as input or read from stdin. They can also write tostdout so that one can put a compress or uncompress step into a pipeline.)

file — guess file type

file FILE

1.10.4 diff — compare two files

diff FILE1 FILE2

The diff command (together with its cousin patch) is very powerful when it comes to big software developmentprojects. For us it is mostly useful to quickly compare two files. diff -U2, a so-called “unified diff” is generallymore readable than the standard diff output. Also look at sdiff, which shows the differences in a side-by-side view(pipe the output through less!).

1.10.5 find — find files

Complicated syntax but can be extremely useful:

find . -name ’*.txt’

Find files over 1M in size:



find . -size +1M -ls

1.10.6 history — all the commands you typed

The shell remembers all the commands that you typed in the “history” (typically a hidden file ~/.history orsimilarly named). This history allows you to

• go back with the Cursor-up key to recall commands

• search backwards with ^R

and see the commands with

history

The history is actually truncated at HISTSIZE (you can set the environment variable HISTSIZE yourself: e.g.HISTSIZE=500; export HISTSIZE)

1.10.7 Downloading files via the commandline

curl (“cat url”) treats a URL as a file. It is a great tool and well worth learning:

curl URL | commandcurl URL -o FILENAME

wget is straightforward for downloading files (but not installed on Mac OS X by default):

wget URL

or

wget URL -O NEWNAME

Excercise: Download http://darthtater.asurite.ad.asu.edu/PHY598/01/vimqrc.pdf and move it into your docs directory,using the commandline:

cd ~/NAME/p01/docscurl http://darthtater.asurite.ad.asu.edu/PHY598/01/vimqrc.pdf -o vimqrc.pdf

1.10.8 echo — printing a string to stdout

If you want to see what the shell thinks of a expression or if you want to have a script output a message you can usethe echo command:

echo "Hello world!"echo "nothing happens *"echo "all the files " *

(Note that in the last line the shell expands * to all the files in the directory.)

There’s also the printf command, which is more versatile but less often used (see man page).

1.10. Other useful commands 13

http://curl.haxx.se/docs/

http://www.gnu.org/software/wget/manual/html_node/index.html

http://darthtater.asurite.ad.asu.edu/PHY598/01/vimqrc.pdf


1.11 Unix variables

Variables are containers to store content in. Bash knows simple variables and arrays. It does not distinguish betweentext and numbers (essentially, everything is treated as text and if needed interpreted as a number).

1.11.1 Shell variables and variable expansion

Assign value to the variable NAME:

NAME=value

E.g:

WORK=$HOME/NAME/p01ls $WORKTMP_DIR=$WORK/tmpls ${TMP_DIR}/* # braces for variables unless only letters

By convention we use uppercase letters for NAME but it can be any mixture between upper and lower case charactersand numbers (though it can’t start with a number). Stick to letters, numbers, underscores.

The value is accessed (“expanded”) by prefixing NAME with the $ (dollar symbol).

Variables behave differently in quotes:

echo "My work directory is $WORK"

This will print something like

My work directory is /Users/USERNAME/NAME/p01

The value is expanded inside strings with double quotes (“soft quotes”). Within single quotes the variable is notexpanded (or “interpolated into the string”) hence we call them “hard quotes”:

echo ’The WORK variable can be accessed as $WORK’

will print

The WORK variable can be accessed as $WORK

In order to keep special characters such as $ in a string you can

• in double quotes, prefix it with a single backslash \

• put it in single quotes

1.11.2 Environment variables

A number of variables are already set to certain values. These “environment variables” have special meaning. Exam-ples

echo $HOMEecho $USER

Only modify if you know what you are doing!

Show the whole environment:



envenv | less

New environment variables are generated with

export NAME=value

or:

NAME=valueexport NAME

Note: In other shells (not bash), one has to use different commands, e.g. in csh and tcsh it is setenv NAMEvalue.

1.11.3 PATH

PATH is a very special environment variable. It lists the directories where commands are searched for.

echo $PATH

which ls

The second command shows the full path of the ls command (which is simply a file in a bin directory). It’s directoryis also listed in PATH. This is why one can simply type

ls

although one could alternatively use the full path to the command:

/bin/ls

Note: If an executable (e.g. a code that you compiled yourself) is not on the PATH then you will always have toprovide the path name in order to execute it.

• The shell searches directories for a command in the order listed in PATH.

• Directories are separated by ‘:’.

• If a command is not on PATH then one has to provide its path to the shell in order to run it.

• It can make sense to ‘.’ to the path to be able to run executables in the current directory.

Changes to PATH are typically done in the shell startup file ~/.bashrc. E.g. adding your own bin directory:

export PATH=$PATH:$HOME/NAME/bin

1.12 General Unix gotchas

• Unix is generally case-sensitive (however, Mac OS X is typically not!)

• directories are separated by the forward slash /

• filenames: may contain any characters, however:

– special characters need to be quoted, using

1.12. General Unix gotchas 15


* backslash:

* “soft” quotes: ” “

* “hard” quotes: ‘ ‘

– Avoid spaces , slashes /, backslashes \, dollar sign $, ampersand &, question mark ?, parentheses (),square [] and curly brackets {}, pipe |, binary relations >, <, back tick \‘ — it will make your lifedifficult.

Also avoid non-english (i.e. non ASCII) letters such as German umlauts (äöü ...) or accented characters(éîè ...) or special symbols (©–† ...).

– Good: standard letters, numbers, underscore _, dot ., dash -. The equality sign = can be used but it canlead to confusion.

1.13 Setting up an editor

We will use the vi editor (actually, the editor is really called vim but it will appear as “the vi editor”... there’s longishUnix history behind this , which does not need to concern us.) vi is very powerful and available on any Unix system.It’s learning curve is fairly steep, though, but “vi” is a very useful skill to have.

1.13.1 vi/vim

Go through vi/vim essentials (vim_basics.txt).

If this is not sufficient consider doing vimtutorial (takes about 20-30 mins): On the commandlinetype

vimtutorial

and follow the instructions.

1.13.2 nano

nano is a light-weight editor available on most modern Unix-like systems. If you absolutely hate vi (some peopledo) then you can try this one. It has fewer features than vi and requires a few more customizations in order to providea comparable experience.

To learn more about nano, launch it and read the help (^G, i.e. CTRL-G) and have a look at nano’s homepage

• customize ~/.nanorc (see also nanorc (5)). Example:

## Backup files to filename~.set backup

## Enable ~/.nano_history for saving and reading search/replace strings.set historylog

## The opening and closing brackets that can be found by bracket## searches. They cannot contain blank characters. The former set must## come before the latter set, and both must be in the same order.##set matchbrackets "(<[{)>]}"

## Enable mouse support, if available for your system. When enabled,## mouse clicks can be used to place the cursor, set the mark (with a


http://www.nano-editor.org/

http://www.nano-editor.org/dist/v2.2/nanorc.5.html


## double click), and execute shortcuts. The mouse will work in the X## Window System, and on the console when gpm is running.##set mouse

## Use smooth scrolling as the default.# set smooth

## Constantly display the cursor position in the statusbar. Note that## this overrides "quickblank".set const

## For python## Use this tab size instead of the default; it must be greater than 0.set tabsize 4## Convert typed tabs to spaces.set tabstospaces

## Use auto-indentation.set autoindent

• add syntax highlighting: Mac OS X’s nanomisses the files that describe syntax highlighting. You can downloadthem from http://darthtater.asurite.ad.asu.edu/PHY598/nanorc.tar.gz . Unpack into a new directory ~/.nano.Or do it in one go:

mkdir ~/.nanocd ~/.nanocurl http://darthtater.asurite.ad.asu.edu/PHY598/nanorc.tar.gz | tar zxvf -

(Note how the tar command can read from stdin (f -) and curl provides the archive to stdin.)

Now you have to add the enable the syntax highlighting in your ~/.nanorc file. Instead of doing this manuallywe use a simple for loop:

cd ~/.nano(for f in *.nanorc; do echo "include \"~/.nano/$f\""; done) >> ~/.nanorc

That appends the correct commands such as

include "~/.nano/asm.nanorc"include "~/.nano/awk.nanorc"include "~/.nano/python.nanorc"...

to the configuration file,

1.13. Setting up an editor 17

http://darthtater.asurite.ad.asu.edu/PHY598/nanorc.tar.gz



CHAPTER

TWO

VI/VIM ESSENTIALS

2.1 Moving

^k

< h l >jv

or cursor keys (but try to learn hjkl)

2.2 Quitting

:q:q! (quit ignoring changes)

2.3 Modes

1. Normal/Command mode (ESC)

2. Insert mode – INSERT –

i insert before cursor

Note: ESC always gets you back to Normal Mode

ESC can also be used to cancel a command.

2.4 Editing text

switch to insert mode and insert at cursor:

i

Now enter text. Backspace typically works. Text can, obviously, span multiple lines. Cursor keys might or might notwork as expected in insert mode. Now go back to normal mode by pressing ESC.

19


switch to insert mode and append at end of line:

A

delete character:

x

delete whole line:

dd

delete word:

dw

delete to end of line:

d$

(see more near bottom under Operators and motions)

2.5 Undo

u

(can undo multiple times)

2.6 Saving file

save the file you opened:

:w

save and quit:

:wq

or

ZZ

save file under new name:

:w filename

2.7 More editing

put (paste) command AFTER cursor:

p

2) delete this line with dd and put it after 1) 1) first line (position cursor anywhere in this line, then p) 3) of third linebeginning

(Also delete word ‘beginning’ with dw and put it at the beginning of the sentence of line 3. Cursor must be on space!)

replacing a character <c>:

20 Chapter 2. vi/vim essentials


r<c>

Marking text for copying:

v (enters -- VISUAL -- mode)

now move around with the movement commands to highlight the text you want to manipulate

d (delete = cut the highlighted text)y ("yanks" = copies the text)

(these commands terminate VISUAL mode)

Now insert the text with the p (put = paste) command.

2.8 More movement

moving the cursor along words

beginning of each word:

w (forward)b (backward)

end of each word:

e (forward)ge (backward)

Beginning of this line:

^ (caret character)

End of line:

$

beginning of file:

gg

end of file

G

page up:

CTRL+F (= CTRL+SHIFT+f all at the same time)PgDwn

page down:

CTRL+BPgUp

2.9 Searching

search for a <phrase> forward:

2.8. More movement 21


/<phrase>

Hit / (followed by RETURN) to find more.

search backward:

?<phrase>

Hit ? (followed by RETURN) to find more. Hit / to search forward again.

Get back to where you started searching from (can be used repeatedly):

CTRL+o (CONTROL and ’o’ at the same time)

(also try CTRL+i to go forward in the search matches)

Matching parentheses/brackets search (good for programming!):

% (while cursor is on (,),[,],{, or })

2.10 Search and Replace

To substitute new for the first old in a line type

:s/old/new

To substitute new for all ‘old’s on a line type

:s/old/new/g

To substitute all occurrences in the file type

:%s/old/new/g

To ask for confirmation each time add ‘c’

:%s/old/new/gc

Note: The pattern old is actually a basic regular expression, thus characters such as ., *, ^, $, [, and ] have aspecial meaning and are not interpreted literally. They need to be backslash-quoted. E.g. search for double asterisk::%s/\*\*/new/g

See the quick reference for details and ‘man re_format’

2.11 More on inserting/replacing

insert a new line above/below a line:

Oo

overwrite (replace) characters:

R (starts -- REPLACE -- mode)

change word/line/...



cw (delete word and INSERT)c$ (delete up to end of line and INSERT)c^ (delete to beginning of line and INSERT)

Using c is faster than using d i (delete, then insert) — it makes a difference once you get used to it.

2.12 Operators and motions

A change command (such as d, c) consists of

operator [number] motion

where

operator - what to do, e.g. d for delete[number] - optional count to repeat the motionmotion - how far to go. A motion can be one of the

movements such as w or e but also searchexpressions /<phrase> or end of line $

Thus

2w move cursor two words forward (empty operatoris the cursor movement)

10dd delete next 10 lines5dw delete next 5 wordsd2j delete current and next 2 linesy/XXX yank up to the text XXX but not furthery$ yank until end of line

2.13 Indentation

When editing source code (such as Python files) you should enable autoindent mode (:set autoindent) e.g. in your~/.vimrc. You can move along the indent levels with:

CTRL+d move one indent level to leftCTRL+t move one indent level to right

2.14 Other stuff

Read a file <FILENAME> and put it below the cursor:

:r <FILENAME>

Run a shell command <COMMAND>:

:!<COMMAND>

e.g. :!ls.

:r !ls also works, i.e. you can put the output of a command directly into your edited file.

Display the line number in the file:

2.12. Operators and motions 23


CTRL+g

Join two lines like these:

J:j

2.15 Customization

The :set command is used to switch on and off many wonderful features. Put your personal set commands into theconfiguration file

~/.vimrc

Example (for python) between the —– lines:

-----------------[BEGIN]------------" vim configuration file for python editing" much more on http://stackoverflow.com/questions/164847/what-is-in-your-vimrc

syntax on" colorscheme for light bg (see eg" http://vimcolorschemetest.googlecode.com/svn/html/index-pl.htm and" http://vim.wikia.com/wiki/Switch_color_schemes)set t_Co=256colorscheme default

" Python: see http://wiki.python.org/moin/Vimautocmd BufRead,BufNewFile *.py syntax onautocmd BufRead,BufNewFile *.py set aiautocmd BufRead *.py set smartindent cinwords=if,elif,else,for,while,with,try,except,finally,def,class

" indentation" add to sourcefiles:" # vim: tabstop=4 expandtab shiftwidth=4 softtabstop=4set modelineau FileType python setl autoindent tabstop=4 expandtab shiftwidth=4 softtabstop=4

-------------------------[END]----------

You can write the part between the lines to a file:

1. position cursor on the first line below the BEGIN line

2. v

(enter VISUAL mode)

3. move to blank line before bottom END marker line

4. :

(use highlighted region for next command)

5. w ~/.vimrc

(write highlighted region to file, overwrites any previous version)



2.16 HELP!!

Help:

:help

The vim tutorial: If you have time or want more practice the run the interactive tutorial that comes with vim. It takes20-30 Minutes. In the shell, enter

vimtutorial

and follow the instructions.

Quick reference “cheat” sheet: http://tnerual.eriogerg.free.fr/vimqrc.pdf (or download the local copy of vimqrc.pdf)

2.16. HELP!! 25

http://tnerual.eriogerg.free.fr/vimqrc.pdf

http://darthtater.asurite.ad.asu.edu/PHY598/01/vimqrc.pdf



CHAPTER

THREE

INDICES AND TABLES

• genindex

• modindex

• search

27