STUDY NOTES GNU/Linux: the command line interfaceharinathrokesh.weebly.com/uploads/4/7/6/1/4761172/fundamentals.pdf · 1.3 Why GNU/Linux? Linux can operate as a web, ﬁle, smb (WinNT),

STUDY NOTES

GNU/Linux: the command line interface

(A GPL’d Document)

Anthony A. Aaby

DRAFT Version 0.9. Edited February 25, 2004

This material is copyrighted under the following copyrights:

Copyright c© 2003 by Anthony A. Aaby

Walla Walla College204 S. College Ave.College Place, WA 99324E-mail: [email protected] sources available at http://cs.wwc.edu/~aabyan/Linux/linux.tgz

This is a free book; you may mirrow it, reproduce it, and/or modify it under the terms of version 2 ofthe GNU General Public License as published by the Free Software Foundation whose website is locatedat http://www.gnu.org/copyleft/gpl.html.

This book is an edited and expanded version of Linux Fundamentals: A Training Manual by PhilipCarinhas, Ph.D. of Fortuitous Technologies (www.fortuitous.com, [email protected]). The originalcopyright is reproduced here:

Copyright c© 2000-2001 Fortuitous Technologies, Inc.

Fortuitous Technologies Inc.6909A Hardy Dr.Austin, Tx 78757USAWWW: http://fortuitous.comE-mail: [email protected]

This training manual is a free book; you may reproduce and/or modify it under the terms of version 2 ofthe GNU General Public License as published by the Free Software Foundation whose website is locatedat http://www.gnu.org/copyleft/gpl.html.

This book is distributed in the hope it will be useful, but without any warranty; without even the impliedwarranty of merchantability or fitness for a particular purpose.

The author encourages wide distribution of this book for personal and commercial use, provided theabove copyright notice remains intact and the method adheres to the provisions of the GNU GeneralPublic License located at http://www.gnu.org/copyleft/gpl.html.

In summary, you may copy and distribute this book free of charge or for a profit. No explicit permissionis required from the author for reproduction of this book in any medium, physical or electronic.

Note, derivative works and translations of this document must be placed under the GNU General PublicLicense, and the original copyright notice must remain intact. If you have contributed new material tothis book, you must make the source code (e.g., LATEX source) available for your revisions.

Contents

1 Introduction to GNU/Linux 51.1 Linux Features . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.2 Multi-User Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.3 Why GNU/Linux? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61.4 User Interface: Graphical User Interface vs Command Line Interface . . . . . . . . . . . . 71.5 Conventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

2 Using X-Windows 92.1 X Display Manager, Window Managers and Desktops . . . . . . . . . . . . . . . . . . . . 92.2 Starting and Customizing the X Environment . . . . . . . . . . . . . . . . . . . . . . . . . 10

3 UNIX Command Line Basics 153.1 Logging In To Your Account . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 153.2 Command Structure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 163.3 The Linux Manuals and the man Utility . . . . . . . . . . . . . . . . . . . . . . . . . . . . 183.4 File System: Create, List, Copy, and Move . . . . . . . . . . . . . . . . . . . . . . . . . . . 203.5 I/O, Redirection, and Pipes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 223.6 Command Line Editing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

4 Text Files 274.1 Command Line Tools . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 274.2 Vi and related editors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 284.3 Editing With Pico . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.4 Editing With emacs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 334.5 Using Mail . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 344.6 Regular Expressions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

5 The Linux Environment 395.1 The UNIX Shell Game . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 395.2 Bash . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 405.3 Shell Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 425.4 Choosing the Right Path . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 445.5 Groups and Newgrp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44

6 User Level Security 476.1 Security . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 476.2 Social Engineering . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.3 Cryptography . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.4 Authentication and Integrity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 486.5 Privacy . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48

DRAFT COPY February 25, 2004 3

7 Filesystem Essentials 497.1 The Linux Virtual Filesystem (VFS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 497.2 File Attributes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 517.3 Changing File Attributes with chmod . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 527.4 Changing File Ownership with chown and chgrp . . . . . . . . . . . . . . . . . . . . . . 547.5 Devices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

8 Managing Filesystems 578.1 Creating Partitions with Fdisk . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 578.2 Creating a Filesystem . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 588.3 Mounting Filesystems by Hand . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598.4 Swap Partitions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 598.5 Automatic Mounting at Boot with /etc/fstab . . . . . . . . . . . . . . . . . . . . . . . . . 60

9 Process Control 619.1 Creating Foreground and Background Processes . . . . . . . . . . . . . . . . . . . . . . . . 629.2 Killing Processes With kill . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 639.3 Managing Process Priorities . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 659.4 Cron . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

10 Shell Scripting 6910.1 Shell Initialization Files . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6910.2 Utility Shell Scripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7010.3 General Scripting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7310.4 Script Automation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

11 Networking and The Internet 7511.1 Network Communication . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7711.2 Using X-Windows Over the Network . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7911.3 Network Security for Users . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

12 Program Development in Unix/Linux 8312.1 C & C++ Compilers . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8412.2 Debugging with gdb . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8512.3 The Unix make Facility . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8512.4 Concurrent Version Control (CVS) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8912.5 Packaging Your Software - Tar . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9012.6 Session Capture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9112.7 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

A Glossary 93

B Unix/Linux Commands by Location 95

C Unix/Linux Commands by Category 101

Chapter 1

Introduction to GNU/Linux

Linux is a computer operating system kernel originally developed by Linus Torvalds as aresearch project. Its rapid evolution to a full featured operating system in less than a decade isremarkable. Linux runs on a variety of hardware platforms including those from Intel, Apple,Sun, and others. The Linux kernel packaged with GNU software from the Free SoftwareFoundation is available in a variety of distributions. The best known include Red Hat, Suse,Mandrake, Gentoo, Debian, and Slackware. See www.linux.org and www.distrowatch.comfor more information on various distributions.

Linux is based on UnixTM which was developed for programmers by programmers. This workis designed to introduce computer science, computer engineering, and software engineeringstudents to Linux command line interface.

Only selected commands and features of the command line interface are covered. The useris assumed to need little more than an overview and a collection of useful examples. Thematerial presented is more than sufficient to enable the reader to effectively and efficientlyinteract with a Unix/Linux environment

1.1 Linux Features

The GNU/Linux distributions provide an operating system that

• is a full-featured, 32-bit multi-user/multi-tasking OS.

• adheres to the common (POSIX) standards for UNIX .

• provides native TCP/IP support.

• has a mature X Windows GUI interface.

• provides a complete development environment with C, C++, Java, editors, version control systems,and built tools.

• is fully Open Source.


CHAPTER 1. INTRODUCTION TO GNU/LINUX

1.2 Multi-User Operation

In UNIX and Linux all interactions with the OS are done through designated “users”, who each have anidentification ID (login name) and a password. UNIX allows different users to co-exist simultaneously andallows for different levels of users.

The most powerful user is called superuser or “root”, and has access to all files and processes. Thesuperuser does many of the system management tasks like adding regular users, file backups, systemconfiguration etc.

Common users accounts, which perform non-system type tasks, have restricted access to system-sensitivecomponents to protect Linux from being accidentally or purposely damaged.

1.3 Why GNU/Linux?

Linux can operate as a web, file, smb (WinNT), Novell, printer, ftp, mail, SQL, masquerading, firewall,and POP/IMAP server to name but a few.

It can act as a graphics and multimedia (audio, video), development (C, C++, Java, Perl, Python, SQL),and office workstation etc.

Linux

Server

FTP Proxy

FileSamba

SQL

Firewall

Httpd

Workstation

Graphics

AudioVideo

Communications

Documentation

C/C++/Java/Perl/DB

Figure 1.1: Linux Uses

Linux is a good solution for developers that need a stable and reliable platform that has open source code.Linux has many GUI software development interfaces. Linux is ideal as a workstation also, and offersmany customizable features not found in any other platform. It makes a good platform for dedicatedworkstaions that have limited functions like in an educational or laboratory environment.

6 DRAFT COPY February 25, 2004

1.4. USER INTERFACE: GRAPHICAL USER INTERFACE VS COMMAND LINE INTERFACE

1.4 User Interface: Graphical User Interface vs Command LineInterface

Unix supports both command line and graphical user interfaces. In some versions of GNU-Linux, thekey combination, Ctrl Alt F6, switches the X based GUI mode to terminal mode with its command lineinterface. In terminal mode, Ctrl Alt F2 switches the display to the X based GUI mode. A terminalwindow, launched from within the X display, may be used to learn the command-line interface.

1.5 Conventions

In order to take full advantage of this manual, students should execute every command that is listed inthe text as well as do all the exercises. The following is a list of conventions supported in this manual:

<bash>: indicates a command entered in the bash shell by the user. When you see this sign, you areexpected to enter these commands exactly as indicated and check that the results are consistentwith what is written.

<tcsh>: indicates commands in the tcsh shell. Most of these commands are to be completed after hours orat home. Since Linux advocates freedom of choice, we wish to make students aware of this optionto bash.

<Super>: indicates a command entered by the System Administrator or Root.

Bold indicates a command entered at the prompts above.

Big Bold TW is used to identify commands in the text.

Underline indicates a file or a directory in the text.

Slant indicates command options.

Plain TW indicates a screen text of command output or editor.


CHAPTER 1. INTRODUCTION TO GNU/LINUX


Chapter 2

Using X-Windows

The X-Windows system is the Linux answer to a graphical user interface or GUI. It allowsusers a great interface to work from.

The X Window System (www.x.org), known simply as “X”, is a portable, network-transparentclient/server interface between hardware and the desktop environment. It is frequently usedin conjunction with the UNIX and Linux operating system. The current version of the XWindow System is “X11R6”. A freely available version of the X protocol for the Intel platformis XFree86 (www.xfree86.org).

If your system is running X-Windows, there should be a prompt to enter a user name andpassword. As the focus of this text is on the command line interface, this chapter is here forcompleteness and may be skipped.

2.1 X Display Manager, Window Managers and Desktops

An X display manager is used to provide login services and session management. The X environmenthas many desktops and window managers that completely control the look and feel of the environment.A window manager is the component which controls the appearance of windows and provides the meansby which the user can interact with them. Virtually everything which appears on the screen in X isin a window, and a window manager quite simply manages them. A desktop environment aims toprovide a more complete interface to the operating system, and supplies a range of integrated utilitiesand applications. Two freely available desktops are KDE (K Desktop Environment, www.kde.org) andGNOME (www.gnome.org). KDE is supported by the Kdev window manager while Gnome is supportedby a variety of window managers such as MetCity, Sawfish, Englightment, and IceWM.

Here is a list of the most popular desktops, window managers, and display managers:

DesktopsCDE Uses the Motif libraries.GNOME Uses the GTK+ libraries..gnome Configuration files.

KDE Uses the QT X-library..kde* Configuration files.

X-Window Managers for Linux


CHAPTER 2. USING X-WINDOWS

AfterStep NextStep feel and beyondEnlightenment An extreme, detailed, and configurable environment.FVWM Young nephew of TWM.fvwmrc Configuration files.FVWM2 Clever brother of FVWM.fvwm2rc Configuration files.IceWM A fast and flexible window manager.Kwin Window manager distributed with KDE.MWM Motif window manager.MetaCity Window manager distributed with Gnome. Built using

Glade and the GTK+ tool kit.OLWM Open Look window manager. Once the standard win-

dow manager from SUN.Sawfish An extensible window manager.TWM One of the oldest and simplest managers. Very basic..twmrc Configuration files.Window Maker NextStep look and feel. Easy to configure

X-Display Managerskdm KDE display managergdm Gnome displays managerxdm X display manager

The following subsections provide a brief description of the desktops.

2.1.1 Gnome

Gnome is one of the two very popular desktops out there.

Lets look at Figure 2.1. First notice that the bottom panel has several typical icons like a tool-box,terminal window, and netscape.

It has a four panel virtual display which gives you four whole screens to work in. This gives some roomto spread out. Inside the window we see some drop down panels that set the appearance. There is alsothe file manager visible. There are click-able icons on the desktop itself, much like the other OS’s youhave used. Not too shabby for a free OS.

2.1.2 KDE

KDE is the other popular desktop. Figure 2.2 is a sample of what the KDE desktop looks like.

Notice that KDE also has four virtual panels to work on, and has all the bells and whistles a high qualityGUI would provide.

2.2 Starting and Customizing the X Environment

If X-Windows is installed and no X-display manager is running on your system, start X from the console,by typing

startx

If everything works, you should have a terminal that has a few big black windows and a clock or two.This is just a single version of how X can be presented. X has many toys to play with. Here are a few:


2.2. STARTING AND CUSTOMIZING THE X ENVIRONMENT

Figure 2.1: Gnome Interface

Basic X Tools

xterm [options] Start an X-Terminalxclock [options] A basic wall clockxman -options Display the manual X-stylexbiff -option Mailbox flag to keep you waiting for mailxboard A chess gamexrdb -merge Xfile Incorporate Xfile into your environmentxload Show the system’s usage load

There are actually too many applications in X to list. Almost everything these days has an X interface.

Just about everything in X can be customized, depending on how hard you want to work on it. Themost simple thing you can do to customize your environment is to keep a defaults file, normally called.Xdefaults located in your home directory.

A typical .Xdefaults looks like this:



Figure 2.2: KDE Interface

!---------------------------------! .Xdefaults X-Resource File!---------------------------------Netscape.Navigator.geometry: =700x700XTerm*visualBell : onXTerm*Font : 9x15boldXTerm*scrollBar : onXTerm*saveLines : 4940XTerm*borderColor : whitexclock*geometry : 60x60-0+0


2.2. STARTING AND CUSTOMIZING THE X ENVIRONMENT

xclock*background : blackxclock*foreground : white

The initial instance in the variables is a class name. The subsequent variables in string are called X-resources or just Resources which identify the property of the program or property to modify. The *’sare wildcards and can match anything.

Finding what is and isn’t a valid X-resource can be difficult, but normally most man pages have a listingof these and examples of how to set them. Once you have edited this file to your liking, you must activatethe changes by using the xrdb utility like this:

<bash>: xrdb -merge .Xdefaults

Afterwards, try a command to see if your changes take hold. Normally, there is an initialization script ina system folder or in your home directory that will read your .Xdefaults for you when you start your Xsession. This file is sometimes called .xinitrc and is easy to customize:

#!/bin/shuserresources=$home/.Xdefaultsusermodmap=$home/.Xmodmapsysresources=/usr/X11R6/lib/X11/xinit/.Xresources

# merge in Xdefaults and keymapsif [ -f $home/.netscape/lock ]; then

rm -rf $home/.netscape/lockfiif [ -f $sysresources ]; then

/usr/bin/X11/xrdb -merge $sysresourcesfiif [ -f $sysmodmap ]; then

xmodmap $sysmodmapfi

# start some nice programsxterm &xclock &# fix the bell- 11 Nov 96 rcr/usr/bin/X11/xset b 50 4500 100 &exec fvwm




Chapter 3

UNIX Command Line Basics

This chapter introduces the commandline interface and commonly used commands.

3.1 Logging In To Your Account

Access to your account may be through a terminal window or a X based login screen. The process oflogging into the system involves the use of a user name and a password.

Text-based interfaces occur in five different contexts. They are:

1. X-Windows GUI interface. Open a shell or terminal window to use the command line interface.

2. Text-base terminal: A single use (at a time) Unix/Linux system or a text-based terminal on a localarea network (LAN).

3. Dialup/PPP connection: A connection to a remote computer through a dial-up interface. The localmachine provides either telnet or ssh to provide login to a text window interface.

4. Local area network (LAN) connection: The local machine provides either telnet or ssh to providelogin to a text window interface.

5. Internet connection: A local host provides access to a remote host through telnet or ssh.

<login:>: YourUserNamePassword: - for security, nothing will be displayed.

When you log into your system with the login name and password given to you, you will see a prompt,something like

<bash>:

You may see a variety of prompts depending the shell your are using and the set prompt command inthe .login file for the c-shell. Your system should always prompt you with the name of the shell (bash)and your login name. This is a customizable feature in the bash shell which you are now using.

To logout enter the command logout, exit, or a period.


CHAPTER 3. UNIX COMMAND LINE BASICS

<bash>: logoutor<bash>: exitor<bash>: .

You may change your password with the passwd command

The most basic command in Linux is the directory listing “ls” command. You can see the contents ofyour account by typing

<bash>: ls -al

You should see files like

drwxr-xr-x 13 joe user 1024 Nov 9 17:06 .dt-rwxr-xr-x 1 joe user 5111 Sep 30 15:19 .dtprofiledrwxr--r-- 2 joe user 96 Dec 1 12:25 .elm-rwxr--r-- 1 joe user 1178 Nov 19 10:58 .emacs

User level filesystem commands are covered in Section 3.4 and the essentials of the filesystem are coveredin Chapter 7.

3.2 Command Structure

We’ve already seen ls which gives a directory listing of the current working directory (cwd). Commandsin Linux follow a general format:

Command -options Other parameters

for example:

<bash>: ls -l .bashrc

First comes the command name, followed by options. Options are normally preceded by a dash or minussign. There is always a space between the command and the dash. Some commands use no options atall. After the options comes any other parameters or informations that command may need.

Let’s talk about some of the workhorse commands. Please note that these definitions are purposelyabbreviated and incomplete! For more details on each of the commands below, you can consult the onlinemanual using the man and info commands, see Subsection 3.3. In the commands below, parametersthat are enclosed in square brackets [...] are optional to that command. [-opts] refers to options in thestyle just mentioned.

Special Keys Strokes:q quits from many commands like more, less, and manCtrl+c also quits out of many commandsCtrl+l clears the screenCtrl+a puts the cursor at the beginning of the command lineCtrl+e puts the cursor at the end of the command lineCtrl+z suspend the current process


3.2. COMMAND STRUCTURE

Get Help and Informationman cmd Consult the manual page for cmdman -k keyword List man pages with keyword (same as apropos key-

word)info keyword List info help pages containing keywordgrep pattern dir look for pattern in the file in dirgrep pattern file look for pattern in fileenv [-opts] [exp] Print environment or run a command with another envi-

ronment.find [path] [exp] Find files in path using explocate keyword Locate all files of name keyword in a databaseprintenv Print environment variables (see set)set [vars] Print/Set environment vars and functionswhatis keyword Search the whatis DB for keywordwhereis command Locate source/binary and manuals for commandwhich command Display path of command

Text Files and Printingcat file(s) Concatinate files, send result to standard output.cat file Display contents of file without pagingmore file Display & page the text file (See less)less file Display & page the text filehead file List the first few lines of filetail [-opts] file List the trailing lines of filediff A B List the differences between A and Bgrep pattern file Find lines containing pattern in filewc files List the number of characters, words, and lines in filesclear Clears the screen. Same as Ctrl+Ltr chars1 chars2 file Change chars chars1 in file to chars2awk| gawk [pgrm][file] Filter file by pgrmsed [script] file Stream edit/filter file using script

Printinglp file System V print commandlpr [-Pprinter] file Print file using printer. See /etc/printcap for list of print-

ers.lp -Pprinter file Print file using printerlpr -Pprinter file Print file using printerlpq [-Pprinter] Check the print queuelpstat System V: Check print queuelprm [-Pprinter] job# Remove job from printer queuecancel job System V: Remove job from print queue

General Tools. Current working directory.. Parent directory

User’s home directorycd dir Change cwd to dir (home if dir omitted)cd .. Change cwd to parent directorycd /path Change cwd using absolute pathcd path Change cwd using relative pathchmod perms files Change file permissions of files



chown owner.group files Change file owner and/or groupchsh Change the default shellcp [-opts] f1 (f2—dir) Copy file f1 to f2 or directory dirdate Displays the datedu -h Report disk space used by the current directory and sub-

directorieskill pid Kills process ID pidln [-opts] Old New Link Old to Newlogin [username] Login to system with UID usernamelpr file Print file on the default printerls [dir] Listing for dir (cwd if directory is omitted)ls [-l] List contents in long (detailed) formatls [-a] List hidden files as wellmkdir dir Creates directory dirrmdir dir Remove directory dir. dir must be emptymv file1 file2 Rename file1 to file2passwd [-opt] username Change passwordps [-opts] Output a list of currently active processespwd List the path to the current working directoryrm files Removes filesrm -r * Recursively remove all files and directories in cwdstartx Start the X-Windowing systemtar [-opt][arch][file] Manage tar archivestelnet [host [port]] Connect to the remote hostuname [-opts] Output name and version number of OSwho List users logged into this systemxterm [-opts] Start a brand new X-terminal window

3.3 The Linux Manuals and the man Utility

Virtually every command that is worth knowing has an entry in the man pages, and is accessed by doinga man command . To get all related commands to a keyword word, use man -k keyword as inthe following example:

<bash>: man -k manualman (1) - format and display the on-line manual pagesman2html (1) - format a manual page in htmlperlxs (1) - XS language reference manualwhereis (1) - locate the binary, source, and manual page filesxman (1x) - Manual page display program for the X Window System

When in doubt, use man and man -k keyword to get info on a command or UNIX related term.. Findthe man page for the ls command. ls will list directory contents:

<bash>: man ls

Here is a man entry for cd which changes your current working directory (folder).

<bash>: man cd


3.3. THE LINUX MANUALS AND THE MAN UTILITY

LS(1) FSF LS(1)NAME

ls - list directory contentsSYNOPSIS

ls [OPTION]... [FILE]...DESCRIPTION

List information about the FILEs (the current directory bydefault). Sort entries alphabetically if none of -cftuSUXnor --sort.-a, --all

do not hide entries starting with ....... etc ........

Figure 3.1: Manual Page for ls

cd(n) Tcl Built-In Commands cd(n)NAME

cd - Change working directorySYNOPSIS

cd dirNameDESCRIPTION

Change the current working directory to dirName, or to thehome directory (as specified in the HOME environment variable)if dirName is not given. Returns an empty string.

Figure 3.2: Manual Page for cd

3.3.1 Exercises

1. <bash>: man man

2. <bash>: man 7 signal3. Man cd and look for information on “.” and “..”4. <bash>: cd .. ; pwd

5. <bash>: cd . ; pwd

6. <bash>: cd ; pwd7. Discusss what “.” and “..” are in terms of the filesystem.



3.4 File System: Create, List, Copy, and Move

The fundamental file system commands for files and directories are those to list, create, copy, move(rename), and remove files. This section will show you the basics. Lets start with creating some filesnow, so we can list them later.

3.4.1 Creating New Files

File creation can be done by invoking an editor on a new filename. Before we try this, there is an easierway to make a new empty file by using the touch command. This is how it works:

<bash>: touch file1

This will make a new file named file1 that is empty. There are many other ways to make new files thatwe will see later.

3.4.2 Creating New Directories

Directories are created with the mkdir command. Thus

<bash>: mkdir dir1

will create a new directory named dir1 located in your current working directory. List this file now withthe “long” format we just spoke about. Check your files with

<bash>: ls -l

3.4.3 Listing Files and Directories

By far, listing a file is the most basic of all commands. As we have seen before, you list files with the lscommand:

<bash>: ls file1file1

<bash>: ls -l file1-rw-r--r-- 1 joe users 0 Oct 28 22:05 file1

Using the -l flag causes a “long” listing that shows more information about file1.

You list your directories in a similar way (note that the second example shows that dir1 is empty).:

<bash>: ls -ltotal 10drw-r--r-- 1 joe users 0 Oct 28 22:05 dir1

<bash>: ls -l dir1total 0

3.4.4 Copying Files and Directories

You can copy a file by using the cp command. The following statement

<bash>: cp file1 file2


3.4. FILE SYSTEM: CREATE, LIST, COPY, AND MOVE

will copy file1 to file2. Copying a folder or directory requires the use of the recursive or -r flagindicating that cp should decend into the directory and copy all sub-files and sub-folders:

<bash>: cp -r dir1 dir2

Exercise 3.4.4: Please create a directory like this now. Explain clearly how the following examplesare different from the above:

<bash>: cp file1 dir1

<bash>: cp -r dir1 dir2/

<bash>: cp -r dir1 dir2/dir3

3.4.5 Moving Files and Directories

Moving a file is the same as renaming it. This allows for the possiblity that you move the file to anotherdirectory as well:

<bash>: mv file2 file3

<bash>: mv file1 dir1/file2

You move a directory in EXACTLY the same way as a file:

<bash>: mv dir1 dir3

3.4.6 Changing Directories

Changing your current directory is done with the cd command:

<bash>: cd dir1

<bash>: cd ../dir2

The cd command without any options or parameters changes working directory to the users homedirectory.

3.4.7 Path to the Working Directory

It is easy to get lost in a filesystem. The command pwd displays the path from the root directory to thecurrent working directory.

<bash>: pwd

3.4.8 Disk Space Usage

Disk space utilization is reported by the command du -h which reports the the disk space used in thecurrent directory and all subdirectories. The option -h appends a size letter to each size reported.

3.4.9 Removing Files and Directories

Normal files are removed with the rm command:

<bash>: rm file1



Removing directories is also done with the rm command, but if the directory is not empty, you need touse the “recursive” or -r option:

<bash>: rm -r dir1

After you try the above, make sure file1 and dir1 are gone. Empty directories may be removed withthe rmdir dir1.

3.5 I/O, Redirection, and Pipes

I/O refers to Input (I) and Output (O). This section talks about input and output of commands andhow you can manipulate these. These prinicipals are very important because they are used constantly inUnix.

3.5.1 Standard I/O

In UNIX , Standard Input (stdin) and Standard Output (stdout) are mechanisms that allow you to inputor output data from a command line. Simple commands like ”cat file1” send their results to stdout(normally to your terminal screen) while the word file1 is an example of stdin which is fed to thecommand cat.

Independent of stdin and stdout, there is the also standard error (stderr) which normally goes to yourscreen when the command detects an error. Its manipulation is shell specific.

Bash assigns special numbers, called File Descriptors, to stdin, stdout, and stderr:

Name Abbreviation File Descriptor Standard Device

Standard Input stdin 0 Keyboard

Standard Output stdout 1 Console

Standard Error stderr 2 Console

Table 3.1: Bash Standard I/O

3.5.2 Redirection

Redirection refers to the art of redirecting input and output traffic from commands. Shells like bashallow for redirection of stdin and stdout with the < and > operators respectively.

As an example, lets say you want to list some files and send (redirect) the output to a file instead of thescreen. Do it the easy way:

<bash>: ls -al > output.txt

To check the output, you can use cat (short for concatenate). cat is uselful when you want to viewshort files:

<bash>: cat output.txt-rwx------ 1 carinhas users 1606 Aug 17 23:14 .acrorc-rwx------ 1 carinhas users 153 Dec 20 08:47 .bashrc-rwx------ 1 carinhas users 3189 Dec 23 15:34 .cshrc................


3.5. I/O, REDIRECTION, AND PIPES

Examples of stdin redirecting:

<bash>: cat < output.txt

<bash>: wc -l output.txt7 output.txt

<bash>: wc -l < output.txt7

Note that “cat output.txt” and “cat < output.txt” give the same result, but the ”wc -l”examples give something slightly different.

In bash stderr is redirected with with the 2> operator, while in tcsh, the >& operator. Just relax andwe will see real examples of this shortly.

Linux Warning: Please note that > will overwrite anything in the output file, if it exists, or create thefile if it does not exist. In contrast, the >> operator will append to the existing file.

Here is a brief summary of the redirects:

Name Operator Description

Redirect stdin < Feeds the file to input

Redirect stdout > Creates or overwrites

Append stdout >> Creates or appends

Redirect stderr >& Both stdout and stderr

Redirect stderr 2> Only stderr in bash

Table 3.2: Standard Redirection I/O

3.5.3 Pipes

When you want to take the output of one command and use that is input into another, use the “pipeoperator” | . Think of actually connecting a metal pipe from one command to another. The followingexample sorts a simple ls command in reverse order (do it!):

<bash>: ls | sort -r.doomrc.cshrc.bashrc.acrorc



3.5.4 Examples to Try

<bash>: cat noname 2> error.txt # Sends error to error.txt

<bash>: cat noname > error.txt 2>&1 # Send stdout + stderr to error.txt

<bash>: cat noname >& error.txt # everything (as above) goes to error.txt

Try these examples in tcsh:

<tcsh>: tcsh

<tcsh>: (cat noname > output.txt) >& error.txt # Send stderr to error.txt

<tcsh>: more output.txt # Same as in bash.

<tcsh>: more error.txt # Just like in bash.

<tcsh>: cat testfile.txt >& out.txt # ditto

<tcsh>: cat < testfile.txt | sort | more # ditto

<tcsh>: locate .bashrc | xargs grep alias # find ‘alias’ in .bashrc.

<tcsh>: exit

Don’t forget the last “exit” to get out of tcsh and back into fabulous bash.

3.6 Command Line Editing

A great shortcut in bash and tcsh is to use the command editing facilities. Just hit the up-arrow keyto a previous command and move the cursor to edit that command. Just try it. Command editing isvery useful in repeating and correcting previous commands.

Now we discuss other features of command line editing.

3.6.1 Command and File Completion

Most shells support command and file completion typing shortcuts. These shortcuts allow you to hit theTab key to finish off the name of a command or file after only hitting a few keys. Try this example of afilename (directory):

<bash>: cd /usr/inc<TAB> will produce

<bash>: cd /usr/include/

Now try this example for a command:

<bash>: ghostv<TAB> should produce

<bash>: ghostview

Note that command and file completion can only complete upto a unique set of commands. This meansyou have to provide a unique start string.

Thus <bash>: ghost<TAB> won’t work because it could complete to ghostview or ghostscript.

tcsh has the same file-name completion mechanism as bash. The <TAB> key will complete the filenameand command up to unique names.


3.6. COMMAND LINE EDITING

3.6.2 Possible Command Completion

bash will show the possible choices are by hitting <TAB>twice (<Ctrl-D> in tcsh ):

<bash>: ls /etc/h<TAB><TAB>host.conf hosts hosts.allow hosts.deny httpd/

<bash>: ls /etc/h

The shell tells you what your choices are and is again ready for more input. This also works on commandstoo:

<bash>: mo<TAB><TAB>modemlights_applet montage mount.smbfsmodemtool more mouse-properties-cappletmodinfo morepgp mouse-testmodprobe mount mouseconfig

<bash>: mo

This shows us the possibilities, and again returns us so we can continue typing a command.

Remember that tcsh has the same possible-completion mechanism as bash does above. Just use <Ctrl-D> instead.

3.6.3 Command Line Substitution and History

In both tcsh and bash we have the facility of Command Line Substitution and Command History.

Command Line Substitution allows you to substitute a dynamic expression inside a command:

<bash>: echo My name is $USERMy name is Joe

<bash>:

The $ symbol allows bash to process the expression in-line and later provide the results to the echocommand.

Command History allows you to use stuff from your old commands in your current command. Remember:recycling is good for the environment ♣. The following list shows the history reference syntax commonto both bash and tcsh:

!! Redo the last command. Same as !-1!−N Repeat the N th most recent command (see next)!−3 Repeat the 3rd most recent command!N Redo the N th entry in the history list!string Redo last command starting with the text “string”!?string? The most recent command which contains the text “string”

Figure 3.3: History Referencing Specifications

On top of history referencing, you can add these modifiers to the shell command line by appending themto the history reference after a colon (:)



0 The first (command) wordn The nth word on the command line besides the command.^ The first word, equivalent to ‘1’$ The last word% The word matched by an ?s? searchx-y A range of words. ‘-y’ abbreviates ‘0-y’.* Equivalent to ‘^-$’, but returns nothing if

the event contains only 1 wordx* Equivalent to ‘x-$’x- Equivalent to ‘x*’, but omitting the last word (‘$’)

Figure 3.4: History Modifiers

3.6.4 Exercises

You can get a list of your history by typing simply history. Create two files called boogie.man andboogie.man.old. The file contents are not important for this exercise. Try these examples of the abovehistory machinery to try. The exact history will vary depending on what you do so please adapt these toyour current situation:

<bash>: history | tail -49 8:30 touch boogie.man

10 8:31 cp boogie.man boogie.man.old11 8:36 echo hello >> boogie.man12 8:37 diff boogie.man.old boogie.man

<bash>: !-2vi boogie.man

<bash>: diff !-2:2.old !-2:2diff boogie.man.old boogie.man

<bash>: diff !!:1.old !!:2diff boogie.man.old.old boogie.mandiff: boogie.man.old.old: No such file or directory

<bash>: !?cat?:0 !!:2cat boogie.man


Chapter 4

Text Files

4.1 Command Line Tools


Colaboration and Productivity Toolscal [[month] year] Display calendar for month or yeardate Display or set system date and timew Display information on users currently logged onwho Display information on users currently logged onwhoami Display userid of current usertalk user@hostname Initiate a talk session with another userwrite user@hostname Send a message to another usermsg y | n Enable, disable reception of talk and message from other

usersmail user@hostname ¡ file Send file as an email message to user@hostnameispell file Look for misspellings in file and offer suggestionsaspell file Aspell (replacement for Ispell) looks for misspellings and

offers suggestionslook prefix Display words beginning with prefixsort [rules] file Sort file according to ruleswc file List the number of characters, workds, and lines in file


CHAPTER 4. TEXT FILES

diff file1 file2 List the differences between file1 and file2cat list of files Concatentate files and sent to standard outputmore file Display file one page at a timeless file Display file one page at a time and allow paging backwordshead file Display the first few lines of a filetail file Display the last few lines of file






Text files may require special handling to print them on a postscript printer. The command

cat textfile | a2ps | lpr

converts textfile to postscript and prints it on a postscript printer.

4.2 Vi and related editors

Although vi is considered one of the most primitive editors in Linux , it is still the most native of theUNIX editors. In a pinch, you will have to know how to use vi if your favorite editor is not available.

vi and vim are screen oriented keyboard driven editors: they display the text one screen full at a timeand editing actions take place at the cursor. They are also modal editors; they work in two modes: acommand mode and an editing mode. There is also a line oriented editing mode in which editing actionstake place at designated locations. VI is case sensitive in both modes.


4.2. VI AND RELATED EDITORS

4.2.1 The basics

Interaction with vi is best thought of in terms of a finite state machine.

• viYourFile (launches the editor in command mode)

• insert mode: i or o (text is then entered by normal typing)

• return to command mode: ESC

• save file and exit vi: ZZ or :wq Return

• exit without saving: :q! Return

• save file and continue in editing: :w Return

On occaision other applications (mail, write, talk) will display text over an editing session. Type CTRL-lto redisplay your editing session.

4.2.2 To enter vi

at the system prompt, enter

• vi fileName or

• vim fileName

where fileName is either a pre-existing file or the name of a new file. You are now in thecommand mode.

4.2.3 To exit vi

press ESC to make sure you are in command mode

1. Type ZZ or :wq to write (save the file) and quit

2. Type :q or :q! to quit without saving the file

4.2.4 Insert text

(press ESC to make sure you are in command mode)

• enter i (aA - append; iI - insert; oO - open lines)

• enter the desired text

• press ESC to return to command mode



4.2.5 Move the cursor


• Use either the arrow keys or use

– h to move left one character

– j to move down a line

– k to move up a line

– l to move right one character

• Type H to move to the head of the file (top left)

• Type L to move to the end of the last line (bottom right)

• Type G to go to the beginning of the last line

• Type nG or :n to go the beginning of the n-th line from the current position

• Type w to go to the beginning of the next word

• Type b to go back one word

• Type 0 (zero) to go to the beginning of the current line

• Type $ to go to the end of the current line

• Type CTRL+u to go up one-half screen

• Type CTRL+d to go down one-half screen

• Type CTRL+f to go forward one screen

• Type CTRL+b to go back one screen

4.2.6 Delete text


n x to delete the character above the cursor; n is an optional number of charactors

• [n]dw to delete the current word

• [n]dd to delete n line(s)

• Type D or d$ to delete from the cursor to the end of the current line

• Type d0 to delete from the start of the line to the cursor


4.2. VI AND RELATED EDITORS

4.2.7 Additional commands

• Undo a command

– Type u to undo the previous command

• Search for a Pattern (press ESC to make sure you are in command mode)

– Type /pattern position the cursor at the first occurrence of the pattern following the cursor

– Type ?pattern to position the cursor at the first occurence of the pattern preceding the cursor.

– Type n to move the cursor to the next occurrence

• Seach and Replace (press ESC to make sure you are in command mode)

– Type :from,to s/target/replacement/g to replace all occurances of targetwith replacementfromline from to line to. The command :1,$s/target/replacement/g to replaces all occurances oftargetwith replacement.

• Save a file

– Type :w to write the file to storage

– Type :w fileName to write the file to storage as file fileName

• Add text from another file

– Type :r fileName to read the file fileName and insert it at the cursor

• Copying or Cutting and Pasting text

– Delete comands put deleted text into a buffer

– Type p, P to copy (put) text from buffer into editor at the cursor

• Join lines

– Type J to join lines

• Open lines

– Type O to open lines

vi has 2 basic modes: Command mode, and Insert mode. First we will start to work on our project.Basic command mode functions are listed below. Those with and star beside them indicate commandswhich enter insert mode.

These commands may not seem obvious from a mnemonic point of view, but were designed for simplicityand keyboard efficiency.

You can read and write files in command mode by using ex mode which is invoked by typing : (whichwill prompt you at the bottom of the vi screen)

:w Write this file out.:q Quit vi w/o writing.:q! Force quit vi w/o writing.:w! Force write.:wq Write this file AND quit vi.:r file Read in ‘‘file’’ before cursor.



key Insert DescriptionA * Append after this linea * Append after this characterC * Replace to end of linecw * Replace this wordi * Insert (start typing here)O * Open to the line above cursoro * Open to the line below cursorR * Overwrite Moder Overwrite with the next charj Move cursor one line downwardk Move cursor one line upl Move cursor one char righth Move cursor one char leftx Delete characterw Move one word forwardb Move one word backD Delete to end of linedd Delete this linedw Delete wordyy Yank a line into bufferp Insert buffer lines after lineP Insert buffer before this line

<esc> Exit insert mode

Table 4.1: vi Command Mode Operations

Find, replace, and execute commands are done in a similar way:

/pattern F ind ‘‘pattern’’ and move cursor there

:1,100 s/blue/green/g F or line 1 to 100, substitute green for blue

:1,$ s/blue/green/g F or all lines, substitute green for blue

:’a,’b s/pie/cake/g F rom mark ‘‘a’’ to mark ‘‘b’’, substitute out pie

:1,$ !sort sort all lines

Once we get familiar with vi we will discuss these modes further. The first thing to do is to start theeditor by typing the following:

<bash>: vi input.txt

You should now have an empty screen that looks like

˜˜˜

Now type i and start typing some meaningful text like

Mary had a little lambda, who’s feet was white as snow.˜<esc>˜˜


4.3. EDITING WITH PICO

The i puts you into insert mode, and the final <esc> puts you back into command mode. Now writeand quit by doing a :wq

Mary had a little lambda, whose fleets was white as snow.˜˜˜˜:wq

"input.txt" 1 line, 58 characters written

4.3 Editing With Pico

Pico is by far the easiest editor to use since it usually displays its commands at the bottom of the screen.Start by typing pico input.txt:

UW PICO(tm) 3.5 File: input.txt

Mary had a little foo, whose fleece was white as snow....Start xxx Regular

bar baz foo [email protected] RegularLinux Rocks TurboLinux eats apples RespectSome say Linux Rocks

^G Get Help Ô WriteOut ^R Read File ^Y Prev Pg ^K Cut Text ^C Cur Pos

^X Exit ^J Justify ^W Where is ^V Next Pg Û UnCut ^T To Spell

The only problem with pico (and other non-vi editors) is that it may not be on your system in anemergency situation, which is when you need it the most. Hit ^X to exit pico.

4.4 Editing With emacs

emacs is the most powerful off all the editors commonly used under Linux . The X-Windows versionis fully loaded with all kinds of option menus and panels. Despite its complexity, you can get by with ahandful of commands. In this section, C- represents Ctrl-, so for example C-x is Ctrl-x.

emacs Exiting and File Manipulationsave a file back to disk C-x C-ssave all files C-x sexit emacs permanently C-x C-cread a file into emacs C-x C-finsert contents of another file into this buffer C-x isuspend emacs (or iconify it under X) C-zgoto beginning of line C-agoto end of line C-edelete character C-dcut to end of line C-k



cut and buffer text C-wset mark C-spacepaste buffer C-y

emacs Error Recovery and Searchingundo an unwanted change C-x uabort partially typed or executing command C-gredraw garbaged screen C-lsearch forward C-ssearch backward C-rregular expression search C-M-sreverse regular expression search C-M-r

emacs Formattingindent current line (mode-dependent) TABindent region (mode-dependent) C-M-indent sexp (mode-dependent) C-M-qindent region rigidly arg columns C-x TAB

emacs Helpbasic help C-hscroll help window C-M-vemacs Tutorial C-h tapropos: show commands matching a string C-h ashow the function a key runs C-h cdescribe a function C-h f

4.5 Using Mail

Linux comes with several mail handlers. I prefer elm because I know it well and it is powerful, howeverpine is very user friendly and easy to use. We will discuss both.

If the system you use provides email service but you do not wish to use the local service, create the file.forward in your home directory containing the email address where you wish to receive your email.

4.5.1 Elm

Fire up elm by typing “elm”. When it first starts, it will ask you some questions about files, just sayyes to all.

Mailbox is ’/var/spool/mail/carinhas’ with 2 messages [ELM 2.5 PL1]

1 Jan 12 * Mail Delivery Subs (61) Returned mail: Host unknown2 Jan 12 * Mail Delivery Subs (62) Returned mail: Host unknown

| =pipe, !=shell, ?=help, <n>=set current to n, /=search patterna)lias, C)opy, c)hange folder, d)elete, e)dit, f)orward, g)roup reply, m)ail,

n)ext, o)ptions, p)rint, q)uit, r)eply, s)ave, t)ag, u)ndelete, or e(x)it


4.6. REGULAR EXPRESSIONS

Command:q

Once inside elm, just hit “m” at the command prompt, and follow the signs. elm is more commanddriven and UNIX -like. When in doubt, just keep hitting “q” for quit. Use the arrow keys to go up anddown the message list.

4.5.2 Pine

Pine is easier to learn and use, and is compatible with elm, so you can use them interchangeably. Usethe up-arrow, down-arrow, <, and > arrows to navigate.

PINE 4.10 MAIN MENU Folder: INBOX 2 Messages

? HELP - Get help using PineC COMPOSE MESSAGE - Compose and send a messageI MESSAGE INDEX - View messages in current folderL FOLDER LIST - Select a folder to viewA ADDRESS BOOK - Update address bookS SETUP - Configure Pine OptionsQ QUIT - Leave the Pine program

? Help P PrevCmd R RelNotesO OTHER CMDS > [ListFldrs] N NextCmd K KBLock

4.6 Regular Expressions

Regular expressions as used in vi, grep, lex, flex etc. Regular expressions are formed from single characters,concatenation of regular expressions, choice between two regular expressions, and zero or more occurencesof a regular expression.

RE ::= c | RE RE | RE + RE | RE*

UNIX extends regular expressions with additional metasymbols and extensions to the regular expressionnotation. Regular Expressions are search patterns that give you control over how text patterns are found.Here are the basics pattern matching symbols:

. Match any single character. ‘‘.*’’ matches anything any number of times. See ** Match the last char or exp 0 or more times. a* matches (), a, aa, aaa,+ Match the last char or exp 1 or more times. a+ matches a, aa, aaa, ..? Match the last char or exp 0 or 1 time. a? matches (), a^ Match the beginning of the line. âbc matches lines starting with abc.$ Match the end of a line. ^$ Matches empty lines.[] Match chars inside. [abc] can match a, b, or c or in a range [A-Z].[^] Negates the match chars: [âbc] matches anything except a, b, or c.() Char grouping. (abc)+ matches abc, abcabc, abcabcabc, ....\ Match regex chars like \+, \*, and \^.{n} The preceding item is matched exactly n times.{n,} The preceding item is matched n or more times.



{,m} The preceding item is optional and is matched at most m times.{n,m} The preceding is matched at least n times, not more than m times.\b Match a word boundary: abc\b matches abc but not abcd.\B Match a non-boundary: abc\B matches abcd but not abc.\w Match a word: abc\w matches abcdef but not abc@.\W Match a non-word: abc\W matches abc& but not abc.

4.6.1 GREP

grep is a pattern search tool. It will find patterns in a file and print it on the terminal. These patternsare regular expressions as well. Test grep with the file as before.

grep [ options ] pattern file..

# 1. Find lines with ‘‘Linux’’ ANYWHERE in the file.grep Linux input.txt

# 2. Find ‘‘Linux’’ in a non word boundary like ‘‘TurboLinux’’grep ‘‘\BLinux’’ input.txt # Remember \B?

# 3. Find ‘‘Linux’’ at beginning and the end of line, regular expressions.grep ’Linux$’ input.txt # Note the single quotes to trick the shell,grep ^Linux input.txt # because shell tries to interpret stuff like $.

# 4. Lines start with ‘‘Linux’’, end with ‘‘Respect’’.grep ’^Linux.*Respect$’ input.txt # ‘‘.*’’ means any number of anything.

# 5. Find some tough patterns:grep ’<.*>’ input.txt # Find lines that have ‘‘<something>’’. Html tags?

4.6.2 AWK

Awk is a pattern scanning and process language. Its Syntax is

awk ’program’ input-files

When the program is long, it is usually more convenient to put it in a file and run it with a commandlike this:

awk -f program-file input-files

Take a look at the manual now. Here are some one-liner examples:

awk ’/xxx/’ input.txt # Find ‘‘xxx’’ in input.txtMost all regular expression syntax works inside the //’s.

awk ’NF > 0’ input.txtThis program prints every line that has at least one field. Its aneasy way to delete blank lines from a file.

awk -F: ’{ print $1 }’ /etc/passwd | sort


4.6. REGULAR EXPRESSIONS

This program prints a sorted list of the login names of all users.awk ’END { print NR }’ input.txt

This program counts lines in input.txt.awk ’NR % 2 == 0’ input.txt

This program prints the even numbered lines of input.txt.If you’d use the expression ‘NR % 2 == 1’, it’d print odd numbered lines.

awk ’{if (length($0) > max) max=length($0)} END {print max}’ input.txtThis program prints the length of the longest input line.

awk ’length($0) > 80’ input.txt # Print lines longer than 80 characters.Since the sole rule has a relational expression as its pattern,and has no action, the default action, printing the record, is used.

awk ’BEGIN { for (i = 1; i <= 7; i++) print int(101 * rand()) }’This program prints seven random numbers from zero to 100, inclusive.

ls -lg *.txt | awk ’{ x += $5 } ; END {print "total bytes:" x }’This program prints the total number of bytes used by all .txt files.

4.6.3 Perl

Perl, “Practical Extraction and Report Language” is the Swiss Army Knife of all scripting languages.It has everything except a Megawatt laser, and a kitchen sink. Perl has a very complete set of regularexpressions and a large set of UNIX system calls that make it a steroid-pumped tool:

perl [options] [ program ]

On the command line you can use the syntax

perl -ne ’command’ file

where the options are

-e ’command’ execute command instead of script-file-n run the script on each line of input file-w Print WARNINGS for your script file

Even though complex, we can learn a few perl one liners that work miracles:

# 1. Find lines starting with ‘‘Linux Rocks’’ in input.txt.perl -ne ’if ( /^Linux Rocks/ ) {print} ’ input.txt

# 2. Find ‘‘Linux’’ in a non word boundary like ‘‘TurboLinux’’perl -ne ’if ( /\BLinux/ ) {print} ’ input.txt

# 3. Find ‘‘Linux Rocks’’ and replace it with ‘‘Rocks Linux’’.# Regular expressions matching ()’s get named $1, $2, $3, etc..perl -ne ’s/^(Linux) (Rocks)/$2 $1/; print ’ input.txt

# 4. Lines start with ‘‘Linux’’, end with ‘‘Rocks’’.perl -ne ’print if /^Linux Rocks$/ ’ input.txt

# 5. Find email addresses in input.txt:perl -ne ’print if (/(\w[.\w]*\@\w[\w_-]+\w)/gm)’ input.txt



4.6.4 SED

sed is a stream editor, which means that it works like a conveyer belt on your terminal. The text fliespast you while it works.

sed [script ] file..

sed is complex so we’ll just show some simple usage and examples:

# 1. From line 1 to 32, print the line numbersed ’1,32 {= ; }’ input.txt # Almost works in VI

# 2. Delete lines 1 through 32sed "1,32 d" input.txt # :1,32 d in VI

# 3. Under UNIX: convert DOS newlines (CR/LF) to Unix formatsed ’s/^M$//’ input.txt # In bash ^M = C-v C-m, tcsh ^M = C-v C-j

# 4. Under DOS: convert Unix newlines (LF) to DOS formatsed ’s/$//’ input.txt # method 1sed -n p input.txt # method 2

# 5. Delete leading whitespace (spaces/tabs) from front of each line# ’^t’ represents a true tab character. ^t => Ctrl-V Ctrl-I.sed ’s/^[ ^t]*//’ input.txt # OK in VI without single quotes.

# 6. Delete trailing whitespace (spaces/tabs) from end of each linesed ’s/[ ^t]*$//’ input.txt # see note on ’^t’, above

# 7. Delete BOTH leading and trailing whitespace from each linesed ’s/^[ ^t]*//;s/[ ^t]*$//’ input.txt # see note on ’^t’, above

# 8. Substitute "foo" with "bar"sed ’s/foo/bar/’ input.txt # replaces 1st instance foo with bar.sed ’s/foo/bar/4’ input.txt # replaces only 4th instance in a linesed ’s/foo/bar/g’ input.txt # replaces ALL instances within a linesed ’/baz/s/foo/bar/g’ input.txt # Sub ONLY on lines which contain "baz"

# 9. Match lines that have ‘‘Regular’’ in them:sed ’/^Regular/d’ input.txt # Delete lines starting with ‘‘Regular’’sed ’/Regular$/d’ input.txt # Delete lines ending with ‘‘Regular’’

#10. From ‘‘Start’’ to ‘‘Linux’’, substitute apples for orangessed /Start/,/Linux/s/apples/oranges/ input.txt


Chapter 5

The Linux Environment

In Linux and UNIX , you carry many local attributes that control how your immediate shellinterface interacts with the system. These attributes are called Shell and Environment Vari-ables, and they are critical to Linux operation. UNIX and Linux use many variables for internalprocessing. These help the system manage resources for users. This chapter discusses the barenecessities that all Linux survivalists need to know.

Personal Information and Environment.login File in home directory which Personalize the csh and tcsh environment. Read upon login..cshrc Files in home directory which personalize the csh, tcsh environment. Read upon login..profile File in home directory which personalizes the sh, bash environment.forward File in user home directory containing email forwarding information.signature Automantically added to your email.plan File in user home directory containing public plans.project File in user home directory.nofinger If present denies existence of user outside of local host. Created by touch .nofinger

finger user[@RemoteHost] Lookup user information contained in password file andthe following files.

chsh Change user’s login shellchfn Change use name and information in /etc/passwd filesource file Update environmental information from file (.cshrc).

5.1 The UNIX Shell Game

One prominent feature of UNIX is that you can control the interface by which you communicate with it.These interfaces are called shells and there are too many. The original shell is called sh and it has manyclose cousins including ash, bash, and ksh. bash is the “native” Linux shell and is the default shellfor Linux . Thus, bash is used for almost all startup files (scripts) and maintenance scripts for Linux .

Many users prefer to use the Berkeley UNIX C-type shells like csh and tcsh. Your choice is largely amatter of preference, however, we will use bash as the default in the course. We will also discuss tcshbecause of its easy to understand C-like syntax.

Normally a shell can be invoked anywhere in your current login session simply by typing the shell name.This is useful for testing, but normally, you will use your default shell.


CHAPTER 5. THE LINUX ENVIRONMENT

A user can control fundamental aspects of his/her environment by setting shell variables as you work,or inside of a startup script file which gets invoked when you login to your account. Simple examplescommon to each shell are your prompt (the thing that stares at you and waits for your commands) andaliases. An alias is a shorthand for some other command which a user can define, normally in the shellstartup files like .bashrc and .cshrc. We’ll work with these concepts very shortly.

5.1.1 Shell Features

Most Linux distributions come with bash and tcsh which replace sh and csh respectively. These arethe two big shells that 99.9% of the Linux world uses. There are several key commonalities and differencesbetween the two. The following tables outline these:

Action bash tcshDefault Prompt $ #Force Redirection >| >!Force Append >>!Variable assignment var=value set var=valueSet Environment var export var setenv VAR valueNumber of arguments $# $# ($#argv in csh)End loop statement done endEnd case statement esac endswLoop for/do foreachIf statement if [ ... ] if ( ... )End if fi endifRead from terminal read $<Start until loop until/do untilWhile loop while/do while

Table 5.1: Bash-Tcsh Difference

5.2 Bash

Bash is the default shell for Linux . The default shell is like an environment where all the basic commandscome from bash. Let’s look at the bash manual page:

<bash>: man bash

Look at the default prompt on your screen. Notice that the prompt contains the shell name and username.The prompt in bash is controlled by the variable PS1. Type in the command “set” to get the followingsort of output:

PRINTER=astroPS1=<\s>:PS2=>PS4=+PWD=/home/joeSHELL=/bin/bashSHLVL=2TERM=vt100


5.2. BASH

Symbol/Command Action> Redirect output>& Redirect stdout and stderr>> Append output to file| Pipe output& Run process in backgroundTAB Filename/command completion!n Repeat command #n in history$var Variable substitution# Comment linebg Put process in backgroundfg Put process in foregroundcd Change directoryecho Echo execution outputjobs List current shell processeskill Kill specified processesumask Set default file permissions

Table 5.2: Bash-Tcsh Commonality List (Incomplete)

5.2.1 Setting Your Prompt

Note that set provides the definition for PS1 and several other variables. Lets look up the manual onbash by typing “man bash”. You are looking at an endless stream of information from the native Linuxmanual pages. Type “/prompt” to search for the word “prompt”. You can find the next occurrence bytyping “n”. Keep going until you see something similar to

PROMPTINGWhen executing interactively, bash displays the primaryprompt PS1 when it is ready to read a command, and thesecondary prompt PS2 when it needs more input to completea command.

\t the current time in HH:MM:SS format\d the date in "Weekday Month Date" format\s the name of the shell, the basename of $0\w the current working directory\W the basename of the current working directory\u the username of the current user\h the hostname\# the command number of this command\! the history number of this command\$ if the effective UID is 0, a #, otherwise a $

Lets change the bash prompt by adding the command number. This is an example:

<bash>: PS1=’ \s: \u \t:# \#: ’

which should produce something like



bash: joe user 00:02:07:# 13:

Now lets set your prompt so that it always displays your path. Even though this manual will specify<bash>: , you should still use a prompt like this:

<bash>: PS1=’<\w>: ’

5.2.2 Creating Aliases (Shortcuts)

Now as it happens, I get very tired of typing out ls -Agl every time I want a long directory listing, soI want to alias that command to d ( for “directory”) as follows:

<bash>: alias d=’ls -Agl’

We really would like to make this definition permanent, by putting this command into .bashrc, and wewill learn how to do this in Chapter 10 on scripts.

The same alias in tcsh (or csh) is written without the equals (=) sign: You start tcsh by typingtcsh:

<bash>: tcsh

<tcsh>: alias d ’ls -Agl’

<tcsh>: d.............<tcsh>: exit

5.3 Shell Variables

In Linux , you always keep shell variables user, directory, path, group, and many others. These variableshelp your shell cope with all the information it needs to deal with normal operations. As we discussedbefore, the set, env, printenv commands will display your environment variables: (without #comments)

PWD = /home/carinhas/misc # Your current working directoryGID = 100 # Your current group ID numberGROUP = users # Your current group nameHOME = /home/joe user # Your home directoryHOST = astro # The name of your workstation.PATH = /bin:/usr/bin # Your current path to executable commands.PS1 = <\s>: # Your current promptSHELL = /bin/bash # Your current shellUID = 501 # Your user ID numberUSER = carinhas # Your user name

Your current working directory (cwd) is where all files are sought and written by default. uid and gid arethe numerical values for your user and group, which identify you as a user and group member respectively.Your home is where you (and your files) live, and your path is where you look for your prey (executablecommands).

You can always determine your cwd with the pwd command. Your cwd is always referred by the “dot”(.) symbol, and the directory directly below is referred to by the double-dot (..) symbol as in the resultof an ls -Agl:


5.3. SHELL VARIABLES

<bash>: ls -gltotal 608drwxr-xr-x 2 joe user users 1024 Dec 20 19:09 .drwxr-xr-x 15 root root 1024 Dec 10 19:04 ..-rwx------ 1 joe user users 3185 Dec 19 15:51 .cshrc-rwx------ 1 joe user users 3185 Dec 19 15:51 .bashrc-rw-r--r-- 1 joe user users 33 Dec 20 10:49 doubletakedrwxrwxrwx 1 joe user users 33 Dec 20 10:49 mail

Note that “..” refers to /home when you are in your home directory /home/joe user. The first characterin the long file listing is the file type, which tell us what kind of animal we have. Also ∼ refers to yourhome directory in your prompt, and changes to the /fully/qualified/path only when moved out of/home/joe user .

5.3.1 Environment Variables

Environment variables are shell variables that control the so-called ’environment’, or shell options. Envi-ronment variables are used internally by the shell at all times. Normally, user defined shell variables willnot affect how the shell works.

Environment variables are all uppercase words like PATH, SHELL, TERM, PAGER, PS1, MANPATH.

In bash, you set environment vars with the export:

NAME=valueexport NAME

Or you can do it all in one command:

export NAME=value

Examples in bash are

<bash>: export PATH=/bin:/sbin:/usr/sbin

<bash>: DISPLAY=groucho:0

<bash>: EDITOR=/bin/vi

<bash>: export DISPLAY EDITOR

Environment variables are set in tcsh in the following way:

setenv NAME value

5.3.2 Exercises

Exercise 1

Find the hostname of your machine and replace “groucho” in the DISPLAY variable as above.



Exercise 2

Go into the tcsh shell and work the following examples:

<bash>: tcsh

<tcsh>: setenv PATH /bin:/sbin:/usr/bin:/usr/sbin

<tcsh>: setenv DISPLAY groucho:0

<tcsh>: setenv EDITOR /bin/vi

<tcsh>: env

5.4 Choosing the Right Path

Your PATH is used by the system to find commands. Its important to set this correctly so that you canaccess everything that you need.

To set you path in bash we edit the .bash profile file:

<bash>: pico .bash profilePATH=$PATH:$HOME/bin # Add $HOME/bin to $PATH.... write and quit .... (hit Ctrl-X, type Y, return)

<bash>: source .bash profile # Initialize $PATH.

Once this is done, bash will look in your own /home/user/bin folder to find any commands you putthere. This is a good thing which will come in handy later when we work on scripting.

5.4.1 Exercises

Set your path in tcsh by editing your .login file, and add ∼/bin to your path.

<tcsh>: pico .loginset path = (/bin /sbin ~/bin /usr/bin /usr/ucb /usr/bsd ) # added ~/bin.... write and quit ....

<tcsh>: source .login # Install the new $path.

<tcsh>: rehash # Tell csh to find files in $path.

tcsh uses its rehash function to re-read all the folders in your path and find the new commands.

5.5 Groups and Newgrp

Groups allow users to have access to files and programs jointly. This is what the group attribute is for.Users can belong to many groups. You can belong to more than one group, however you can only beactive in one group at a time. All users have a default group that they enter at login. You change yourgroup interactively by using the newgrp command:

<bash>: newgrp groupname


5.5. GROUPS AND NEWGRP

which will spawn a new shell (on top of the old one) with your default group given by groupname. Forthis trick to work, you must belong to this group to begin with.

Once you are in this new group state, all process and files created will carry the group name as well asyour user name. Having multiple groups can be useful when accessing restricted hardware or working ina team of developers, where everyone needs access to common software and systems.

To exit this group-shell, simply type exit.

5.5.1 Exercises

1. Look at the files /etc/passwd and /etc/group. Determine how groups are created and assignedwith the help of the manual pages. Write your observations down here.




Chapter 6

User Level Security

Protection and Security.nofinger File in users home directory created using the touch com-

mand limits finger to local machinepasswd Change user passwordumask Set default file access permissions (permissions taken awaychmod [ugo][+|-][rwx] file Change file mode (permissions)chmod octalmode Change file mode (permissions)ssh [user@]RemoteHost[command] Secure shell, replacement for telnetsftp RemoteHost Secure file transport, replacement for ftpvim -x file Create encrypted file. You will be promped for an

encryption keyExamples:

• ls -l

• Protection bits: u - user, g - group, o - other.[-dl] rwx rwx rwx

• chomd mode filenamechmod who op permissions filenameExamples: chmod go-w ..., chmod o+x ..., etc

• umask - set file creation mask (ppermissions taken away; in .login file.

• chown – must be performed by root

• ln -s q sl – create a symbolic link sl in working directory to file/directory q

6.1 Security

The primary concern is integrity. Accuracy or validity is fundamental and security is secondary. Securityis protection against unauthorized disclosure, alteration, or destruction of data and protection againstunauthorized use.


CHAPTER 6. USER LEVEL SECURITY

6.2 Social Engineering

passwords ... system access ...

6.3 Cryptography

6.3.1 SSH, SCP, and SFTP

6.4 Authentication and Integrity

6.4.1 Digital Signatures

6.4.2 Cryptographic Checksum

6.5 Privacy

6.5.1 Privacy-Enhanced Electronic Mail (PEM)


Chapter 7

Filesystem Essentials

In this chapter, we will learn how to master files and filesystems. This is a very importantaspect of Linux , and is used constantly.

Compression and Archives:compress file file.Z Compress a fileuncompress file.Z file Uncompress a filegzip file file.gz Compress a filegunzip file.gz file Compress a filetar Tape archive untilitytar cf Archive.tar Dir Create an archive (Archive.tar) of the directory (Dir) and

its contents.tar cfz Archive.tar Dir Create a compressed archive (Archive.tar) of the directory

(Dir) and its contents.tar xf Archive.tar Dir Extract the tar file into the working directory.tar xzf Archive.tgz Dir Extract the compressed tar file into the working directory.tar tzf Archive.tgz Dir List the contents of the tar file.tar xf Archive.tgz FileofInterest List the contents of the tar file.

File mode and ownership:

chmod Change file modechown Change file ownership or group

7.1 The Linux Virtual Filesystem (VFS)All files in Linux are accessible from a single integrated filesystem called a Virtual Filesystem. It is atree structured filesystem with directories and files. The base of that system is called the root and oftendenoted by a slash (/). Users home directory is found in /home. User with user name joe will have/home/maria as The mc is a visual shell for use in a command line environment and it may be used tonavigate around the system files below:

The next figure outlines the root VFS root tree as set by the Filesystem Hierarchy Standard located athttp://www.pathname.com/fhs/

Among the most important of these are /, /etc, /usr, and /opt, /tmp and /home. Below is a sampledirectory tree that starts from root but lacks the /var subtree.


CHAPTER 7. FILESYSTEM ESSENTIALS

Figure 7.1: Linux VFS as viewed by mc

The root directorybin Essential command binariesboot Static files of the boot loaderdev Device filesetc Host−specific system configurationlib Essential shared libraries and kernel modulesmnt Mount point for mounting temporary filesystemsopt Add−on application softwaresbin Essential system binaries

usr Secondary hierarchvar Variable data

tmp Temporary files

Figure 7.2: VFS Root Tree

7.1.1 Exercises

1. The tree diagram above is not correct for a modern version of Linux. Find and compare thedifferences with your system and write down your results.

2. The /proc filesystem is a special on that has direct access to the system kernel. The files in /prochave no size, but what happens if you do the following:<bash>: cat /proc/cpuinfo

<bash>: cat /proc/pci


7.2. FILE ATTRIBUTES

/ bindevetchome mary

janejoelib

proc

tmpusr X11R6

bin

etc

include

lib

local

man

spool

bin

emacs

etc

lib

src linuxtmp

Figure 7.3: A Linux directory tree.

Discuss the possible uses of this.

7.2 File Attributes

Lets take a close look at a directory listing of mail. Enter the command ls -agl mail:

drwxrwxrwx 1 joe user users 33 Dec 20 10:49 mail-file

type

-user

read

-user

write

-user

execu

te-g

roup

read

-gro

up

write

-gro

up

execu

te-o

ther

read

-oth

erw

rite-o

ther

execu

te

link

num

ber

user

ID

gro

up

ID

Size

Month

Day

Tim

e

File

Nam

e

Figure 7.4: Unix File Attributes

Its entry gives us a lot of information. It has all of its access entirely open, so everyone can read, write,and execute. Also note that the first attribute listed is its type, in this case a “d” for directory. Fileattributes for restricted (plain type) files are listed as dashs (-).

The next 9 parameters are file permissions or access parameters In UNIX , files have 3 sets of 3 accessparameters or “permissions”. The first 3 are for the owner, the second 3 for the group, and the last 3 for“other” or the “world”. The permissions are read (r), write (w), and execute (x). When you look at a file



listing in long format you will see these attributes listed as (r,w,x) or omitted with a “-” in their place.If the attributes are present, access is granted to for that action, otherwise access is denied.

Other major file types include symbolic links (l), an internal reference to another file, and plain files (-).The following table summarizes native Linux file types as listed by the find command. Note that theplain file permission never lists by ls as (f) but as a dash (-).

b block special file

c character special file

d directory

l symbolic link

p named pipe (FIFO)

s socket

Table 7.1: File Types as Listed by find

7.3 Changing File Attributes with chmod

An important task for any Linux user is to manage file attributes (permissions) with the chmod command.The syntax for chmod is

chmod [-R] MODE FILE

where MODE can be a comma separated combinations of 3 sets of symbols:

[ugoa][+ - =][rwx]

u --> userg --> groupo --> othersa --> all

You can add (+), subtract (–), or exactly set (=) the mode. Examples:

chmod u+w mail # Add user (u) write (w) privilegeschmod u-w mail # Remove user (u) write privileges.chmod g+w mail # Add group (g) write privilegeschmod g=r mail # Allow only the group read privileges, nothing else.chmod o+x mail # Add execute (x) privileges for others.chmod a+x mail # Add execute (x) privileges for everyone..chmod a=xr mail # Allow read and execute only to everyone..chmod go-r mail # Remove group and others read privileges..chmod ugo+w mail # Same as chmod a+w mail

MODE can also be an octal representation which represents the absolute mode:


7.3. CHANGING FILE ATTRIBUTES WITH CHMOD

1 = others execute # o=x2 = others write # o=w4 = others read # o=r10 = group execute # g=x20 = group write # g=w40 = group read # g=r

100 = user execute # u=x200 = user write # u=w400 = user read # u=r

The trick with octal is that you can add up the different numbers to get all different combinations ofprivileges like:

1 = others execute # o=x+ 2 = others write # o=w+ 4 = others read # o=r-----------------------------= 7 = other read, write, and execute = o+rwx

10 = group execute # g=x20 = group write # g=w00 = no group read #

100 = user execute # u=x000 = no user write #400 = user read # u=r-------------------------- add these last 6 to get530 = u=rx, g=wx

chmod 750 mail => chmod u=rwx,g=rx,u= mailchmod 700 mail => chmod u=rwx,g=,u= mailchmod 751 mail => chmod u=rwx,g=rx,o=x mailchmod 741 mail => chmod u=rwx,g=r,o=x mail

7.3.1 Exercises

1. On directories, the execution privilege allows for access into the directory, so having just read andwrite (rw) privilege is not enough. Only execute privilege will allow any access to anything insidethe directory. Try this out by doing a

<bash>: chmod 600 mail

<bash>: cd mail

There are actually many other modes that can be set using the first (non-octal) method. The oneswe discussed are the essential ones. Please check out the man pages on chmod and ls.

2. Restore your mail directory back to a usable mode.

3. Write down the command you just used in both octal and symbolic modes.

4. Read the man pages and learn how to set and the function of the “set user ID on execution” (SUID)bit for chmod. Discuss this with your classmates and instructor. This concept is a requirement forthe LPI exams.



7.4 Changing File Ownership with chown and chgrp

Just as you can change the file permission, you can change the owner and group of a file. chown doesthis for you with the syntax:

chown [-R] USER[.[GROUP]] FILE

Note that chgrp can change the group at the same time as in the following examples.

<bash>: chown johnson mail # Sorry, only johnson or Root can do that.chown: mail: Operation not permitted

<Super>: chown joe user.users mail # Change owner to joe user and group to users.

<Super>: chown joe user.bozos mail # Change owner to joe user and group to bozos.chown: mail: Operation not permitted # You’re not a member of bozos group!

<Super>: chown .testers mail # Ok, you are in the testers group.

chgrp is less powerful and not really needed but should be noted. Its syntax is

chgrp [-R] GROUP FILE

Here is an example.

<bash>: chgrp testers mail # Same as ‘‘chown .testers mail’’

<bash>: chgrp users mail # Same as ‘‘chown .users mail’’

7.4.1 Questions

Why can’t you change ownership of someone elses files? Who can do this? Is this a good or bad feature?

7.4.2 Exercises

1. Log in as root and change the owner and group of some files in the /tmp directory. Make sure you arein /tmp and not /home/you/tmp or some other place.

7.5 Devices

In Unix (almost) all commands and hardware are represented by files. The hardware devices are fun-damentally different and live in the directory /dev. Device files allow access to the hardware directlythrough the kernel.

These special files have major and minor numbers that identify them in the Linux kernel. Major DeviceNumbers specify a particular driver for I/O redirection in the kernel, and Minor Device Numbers specifya device to be accessed by that driver. A typical example is the /dev/hda hard-drive:

brw-rw---- 1 root disk 3, 0 May 5 1998 /dev/hda


7.5. DEVICES

It has a major number 3 and minor number 0. The leading “b” indicates that this is a block device. It’sowner is root and its group is disk .

Do an ls -agl /dev to see the major and minor numbers of typical devices.

We list the most commonly used devices below. Note that N will represent an integer number.

Device Name Type Description

/dev/hda B first IDE hard disk

/dev/hdaN B Nth partition of /dev/hda

/dev/sda B first SCSI hard disk

/dev/sdaN B N th partition of /dev/sda

/dev/loop0 B First loopback device

/dev/console C System console

/dev/parport0 C first parallel port

/dev/lp0 C first printer port (LPT1)

/dev/null C System Trash Can (Black Hole)

/dev/ttyN C N th virtual console

/dev/pts/N C N th pseudo terminal (dynamically created)

/dev/ttySN C N th serial port

/dev/psaux C PS/2-style mouse port

/dev/mouse L Soft link to a serial mouse like psaux

/dev/printer S lpd local socket

Table 7.2: Linux Devices (See 7.1 for type info)

/dev/null is a funny device that swallows everything you feed it like a black hole, so don’t put anythingthere that you want back. It’s very convenient for swallowing up unwanted error messages like this:

<bash>: netscape >& /dev/null &

We will learn the meaning of the preceding example in the next section. The cat command prints a textfile to the console and the cp command copies one file to another. Try this example

<bash>: cat .cshrc

<bash>: cp .cshrc /dev/null

<bash>: cat /dev/null

The last command should produce nothing.



7.5.1 Exercise: Redirect the output of command top to file top.log

7.5.2 Exercise:

<bash>: ls -l /dev | grep ˆc

<bash>: ls -l /dev | grep ˆb

<bash>: ls -l /dev | grep ˆl

<bash>: ls -l /dev/hda*

<bash>: ls -l /dev/sda*

7.5.3 Discussion


Chapter 8

Managing Filesystems

Creating and mounting filesystems is essential for disk access. The following sections discussthe essential elements. Most of the commands we use are only accessible by the SuperUser(root), so you must be logged in as such.

Filesystem Managementdudfddfdiskmountsyncumountmkfsmkswapswapon

8.1 Creating Partitions with Fdisk

On any new hard disk, you must first create the partitions that will be mounted. This is done with thefdisk utility. The following is a typical fdisk application to add a 100MB partition to /dev/hdc:

8.1.1 Primary Partitions

<Super>: fdisk /dev/hdcCommand (m for help): n (create a new partition with n)First cylinder (768-788, default 768):Using default value 768Last cylinder or +size or +sizeM or +sizeK (768-788, default 788): +100MCommand (m for help): p (print the current partition table)

Disk /dev/hdc: 255 heads, 63 sectors, 788 cylindersUnits = cylinders of 16065 * 512 bytes


CHAPTER 8. MANAGING FILESYSTEMS

Device Boot Start End Blocks Id System/dev/hdc1 * 1 128 1028128+ 6 FAT16/dev/hdc4 742 788 377527+ 5 Extended/dev/hdc7 768 780 104391 83 Linux

8.1.2 Swap Partitions

A swap partition is just extra memory that lives on your hard drive. Most systems need swap if yoursystem has only a moderate amount of RAM.

To create a swap partition, toggle the Partition Id field to 82 with the “t” command while in fdisk.We assume that we are still in fdisk from the previous section:

Command (m for help): tPartition number(1-7): 7Hex code (type L to list codes):82

Command (m for help): pDisk /dev/hdc: 255 heads, 63 sectors, 788 cylindersUnits = cylinders of 16065 * 512 bytes

Device Boot Start End Blocks Id System/dev/hdc1 * 1 128 1028128+ 6 FAT16/dev/hdc4 742 788 377527+ 5 Extended/dev/hdc7 768 780 104391 82 Linux Swap

Command (m for help): w (write the new partition info and exit)The partition table has been altered!

8.2 Creating a Filesystem

After you have created a partition as in 8.1 you must first make the filesystem on it with mkfs (pronounced“make fs”).

For example,

<Super>: mkfs -t ext2 /dev/hda7

This will write the inodes and the superblocks that the filesystem requires. Inodes are points on thefilesystem that correspond to filenames and storage. Superblocks are blocks of data that have criticalinformation about the filesystem, which can be used to fix the filesystem in case of a crash.

Lets create an ext2 filesystem on a floppy for fun. First use fdformat -n /dev/fd0. Then use mkfsas ...

<Super>: mkfs -t ext2 /dev/fd0 1440


8.3. MOUNTING FILESYSTEMS BY HAND

8.3 Mounting Filesystems by Hand

A filesystem is unusable by Linux until you “attach” it to the operating system. This attachment iscalled mounting. You can mount a filesystem with the mount command.

In order to mount a filesystem, you need to know the partition name (as in /dev/fd0) and a mountpoint. A mount point is an empty directory where the filesystem will be located. It MUST be an emptydirectory. Then you execute:

<Super>: mount -t ext2 /dev/fd0 /alt

Where the -t ext2 tells mount that the filesystem type is an ext2 filesystem, the default Linux filetype./alt is the mount point and /dev/fd0 is the device to be mounted. Go into /alt and see if you canwrite on that partition.

To remove a filesystem, use the umount command:

<Super>: umount /alt

8.4 Swap Partitions

Swap partitions are an important source of virtual memory for the system. When you use all of yourRAM, Linux uses the Swap space for temporary storage.

8.4.1 Preparing Swap Space

Once your swap partitions are created, you need to prepare the swap partition with the mkswap com-mand, which formats the partition:

<Super>: mkswap [-c] [-vN] [-f ] device [size]

The device can be either a partition or a file. Consult the man page now for mkswap options. A swappartition is prepared in the following way:

<Super>: mkswap -v1 /dev/hda7

To create a swap file, you need to use the dd command as follows:

<Super>: dd -f=/dev/zero 0f=/swap bs=1024 count=8192

<Super>: mkswap -c /swap 8192

<Super>: sync

which will make a swap file of 8192 blocks (8MB) from /dev/zero and put it in the file /swap in yourroot directory.

We used the sync command after mkswap to force the format information to be written to the filesystem,otherwise the system may take its sweet time to do so. sync ensures that /swap gets written before acrash or system failure.


CHAPTER 8. MANAGING FILESYSTEMS

8.4.2 Activating Swap Space

To activate a swap partition, use the swapon command:

<Super>: swapon /dev/hda7

To activate a swap file, use the following syntax:

<Super>: swapon /swap

where /swap is the swap file created as above.

To remove swap from the system, swapoff is used:

<Super>: swapoff /swap

8.5 Automatic Mounting at Boot with /etc/fstab

The /etc/fstab contains mounting information that the system uses at boot time. The format for/etc/fstab is

# /etc/fstab# device Mount Point Type Options Freq Pass#/dev/hda1 / ext2 defaults 1 1/dev/hda2 /usr ext2 defaults 1 2/dev/hda3 /home ext2 defaults 1 2/dev/hda7 none swap defaults 0 0/dev/cdrom /mnt/cdrom iso9660 noauto,ro 0 0

Edit this file now and add an entry for our hand-mounted filesystem for /dev/fd0 above. Once this isdone, you can abbreviate the mount command for it by simply doing

<Super>: mount /alt

Mount will look inside of /etc/fstab and get all the required information that it needs to do its work.You can override /etc/fstab by specifying all the information as before.

Linux will automatically mount all the files listed correctly in /etc/fstab at boot time.


Chapter 9

Process Control

In this chapter we will learn how to control your processes both interactively and non-interactively. It is very important to maintain control of your system and manage unrulyprocesses.

The program design philosophy under Unix is to treat each program as a filter which takes itsinput from the standard input file and sends its output to the standard output. By default,the standard input file is the keyboard and the standard output file is the display screen.

Programs can be strung together using pipes. A pipe takes the output of one program andredirects it to become the input for another program.

IO Redirection, Pipes and Filterscmd < file Use file as input to cmd.

Example: mail user@RemoteHost< filecmd > file Create file containing the output of cmd.

Example: cat file1 > file2; equivalent to cp file1 file2cmd >> file Append output of cmd to file filecmd1 | cmd2 Connect two processes with a pipe. The output of the

first is piped to the input of the second

Processes and MultitaskingC-c Abort current taskC-z Suspend current taskbg Place suspened task in backgroundjobs List currentjobs processesfg [JobId] Bring background task or suspeneded to foregroundtask & Launch task into backgroundps aux Report process status for all processeskill -9 Pids Abort processes with Pidstop Display and update top CPU processesnice Run a task with a modified scheduling priorityrenice Alter priority of a running tasknohup cmd Run cmd as a background task immune to hangups.crontab -e user Edit user’s cron file to schedule background tasks


CHAPTER 9. PROCESS CONTROL

9.1 Creating Foreground and Background Processes

Process control refers to the ability to control processes both in the foreground and background of theshell. A foreground process is one that is running interactively in your shell, and include most processeswe have discussed so far. You can interrupt a foreground process by typing Ctrl-Z pronounced “controlz” (hold <Ctrl> and hit the z key). Now to put that process back into the foreground, you use the (bash)command fg.

A background process, by contrast, is one that does not interact with your shell directly. It keeps to itselfand runs silently. To place a process into the background you use the builtin command bg. From theshell point of view, all processes have a job number. From the system’s point of view, all processes havea process ID, often called a PID (see the ps command).

To actually be able to catch one of these processes, we need to find a command that lasts longer than aninstant (like date). Lets work with the xterm command which creates an X-terminal window:

<bash>: xterm # Start a fresh xtermCtrl-Z # You hit Ctrl-Z and suspended it![1]+ Stopped xterm

<bash>: fg # Restarted it with fgxtermCtrl-Z # You suspended it again.. good..[1]+ Stopped xterm

<bash>: jobs # Find the job number.[3]+ Stopped xterm<bash>:

<bash>: bg %3 # Put xterm in the background.[3] xterm &<bash>:

You should now have an xterm which behaves normally. The last [3] displayed indicates that its JobNumber is 3. You can use that number to refer to it using fg as in

<bash>: fg %3 # foreground it with fgCtrl-Z # Suspend it again.[3]+ Stopped xterm

<bash>: bg %3 # Put it in the background with bg[3] xterm &

<bash>: jobs[1] Running xterm -cr red -ms >&[2] Running xclock[3] Running xterm

<bash>: kill -9 %3

<bash>: jobs[1] Running xterm -cr red -ms >&[2] Running xclock[3]- Terminated xterm


9.2. KILLING PROCESSES WITH KILL

The jobs displays all jobs in the shell along with their number and status. The kill -9 %N kills jobnumber N . The flag (-9) is needed to force a “sure kill” which is sometimes needed. The ampersandsymbol (&) is what you would use to put a command directly into the background with xterm &.

Be careful because some commands want to write errors to the standard output stdout and that couldcause strange results on your screen. To avoid this, you can redirect stdout to /dev/null (or a log file)AND put it into the background in one gulp:

<bash>: xterm >& /dev/null &or<bash>: xterm >& /tmp/xterm-log &

This is advisable if you plan to script your commands, because stdout that has no place to go can stopyour script in its tracks. Experience has shown to always deal with the trash in advance.

9.2 Killing Processes With kill

Now we will display the process numbers with the ps command

<bash>: psPID TTY TIME CMD683 ttyp1 00:00:00 bash

31666 ttyp1 00:00:00 xterm32716 ttyp1 00:00:00 xclock

<bash>: ps -agl # Get a long list with psF UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND

000 501 683 672 13 0 2220 804 rt_sig S ttyp1 0:01 [bash]000 501 31666 683 0 0 3012 1900 wait4 S ttyp1 0:00 xterm -cr r000 501 32716 683 0 0 2572 1304 do_sel S ttyp1 0:00 xclock

<bash>: top42 processes: 40 sleeping, 2 running, 0 zombie, 0 stoppedCPU states: 2.3% user, 0.7% system, 0.0% nice, 96.8% idleMem: 63484K av, 52328K used, 11156K free, 18020K shrd, 2896K buffSwap: 220316K av, 4964K used, 215352K free 28804K cached

PID USER PRI NI SIZE RSS SHARE STAT LIB %CPU %MEM TIME COMMAND664 root 18 0 14824 14M 1540 R 0 1.1 22.7 18:21 X

31797 carinhas 9 0 7060 7060 3276 S 0 0.0 11.1 0:16 xemacs464 carinhas 0 0 6020 6020 3596 S 0 0.0 9.4 0:03 netscapex

31666 carinhas 0 0 1900 1900 1440 S 0 0.0 2.9 0:00 xterm32716 carinhas 0 0 1304 1304 1096 S 0 0.0 2.0 0:00 xclock... <Shift-M> Sorted by memory usage....Ctrl-C <quit>

Now I have located a renegade process called xclock which seems using way too much memory accordingto the top utility. For example, if you want to kill process 32716, do the following:

<bash>: kill -9 32716



[3] Terminated xclock

<bash>:

Study also the man pages for killall, pstree, and proc for the LPI exams.

Next is an abbreviated manual page for kill, but beware that most shells have their own built-in versionof kill.

KILL(1) Linux Programmer’s Manual KILL(1)NAME

kill - terminate a processSYNOPSIS

kill [ -s signal ] %job | pid ...

kill -l [ signal ]

OPTIONSpid .. Specify the list of processes.-s Specify the signal name or number to send.-l Print a list of signal names. See signal(7)

Here is the relevant and abbreviated man section of signal(7):

SIGNAL(7) Linux Programmer’s Manual SIGNAL(7)NAME signal - list of available signals

Signal Value Action Comment----------------------------------------------------------SIGHUP 1 A Hangup detected on controlling terminalSIGINT 2 A Interrupt from keyboardSIGQUIT 3 A Quit from keyboardSIGILL 4 A Illegal InstructionSIGABRT 6 C Abort signal from abort(3)SIGFPE 8 C Floating point exceptionSIGKILL 9 AEF Kill signalSIGSEGV 11 C Invalid memory referenceSIGPIPE 13 A Broken pipe: write to pipe with no readersSIGALRM 14 A Timer signal from alarm(2)SIGTERM 15 A Termination signalSIGUSR1 30,10,16 A User-defined signal 1SIGUSR2 31,12,17 A User-defined signal 2SIGCHLD 20,17,18 B Child stopped or terminatedSIGCONT 19,18,25 Continue if stoppedSIGSTOP 17,19,23 DEF Stop processSIGTSTP 18,20,24 D Stop typed at ttySIGTTIN 21,21,26 D tty input for background processSIGTTOU 22,22,27 D tty output for background process

The letters in the "Action" column have the following meanings:

A Default action is to terminate the process.B Default action is to ignore the signal.C Default action is to dump core.


9.3. MANAGING PROCESS PRIORITIES

D Default action is to stop the process.E Signal cannot be caught.F Signal cannot be ignored.G Not a POSIX.1 conformant signal.

9.3 Managing Process Priorities

In Linux and UNIX , all processes have an importance factor called a priority. The priority tells thesystem kernel how often to service that process. Obviously some processes are much more critical tothe system than others. Hardware access is more important than email. The smaller the priority value,the more importance the process is given. Priority values range from -20 to 20, where -20 is the highestpriority and 20 is the least.

You can see the priority of your processes by reading the PRI column in the ps -xl command:

<bash>: ps -xl

F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND000 501 617 612 0 0 2032 504 do_sel S ? 0:03 fvwm000 501 622 617 0 0 3144 872 do_sel S ? 0:01 xterm -cr r....etc....

There are two major tools for manipulating process priorities, nice and renice. nice lets you specifythe priority for a command initially, and renice allows you to change the priority after the process isstarted. Here’s an exercise for you:

<bash>: man nice----------------------------------------------------------------NICE(1) NICE(1)

nice - run a program with modified scheduling prioritySYNOPSIS

nice -n priority command [arguments]

----------------------------------------------------------------

9.3.1 Exercises

<bash>: xclock -rv &

<bash>: nice -10 xclock -rv &

<bash>: /bin/nice -n 10 xclock -rv & # Same as above

<bash>: ps -xl

F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND000 501 617 612 0 0 2032 504 do_sel S ? 0:03 fvwm000 501 622 617 0 0 3144 872 do_sel S ? 0:01 xterm -cr r000 501 2855 1321 8 4 2568 1300 do_sel SN ttyp2 0:00 xclock

<bash>: renice 20 2855 # Renice that xclock pid 2855.



2855: old priority 4, new priority 20 # Don’t worry if numbers don’t jive.

<bash>: renice 5 2855 # Please renice to LOWER # ?renice: 2855: setpriority: Permission denied # Only HIGHER #’s allowed!

<bash>: ps -xl

F UID PID PPID PRI NI VSZ RSS WCHAN STAT TTY TIME COMMAND000 501 617 612 0 0 2032 504 do_sel S ? 0:03 fvwm000 501 622 617 0 0 3144 872 do_sel S ? 0:01 xterm -cr r000 501 2937 1321 19 19 2568 1300 do_sel SN ttyp2 0:00 xclock

Note that you can only renice to a higher number (thus lower priority) so the system gives an errormessage. Also, the listed PRI takes time to adjust to the NI (nice) value.

LINUX WARNING! tcsh defines its OWN built-in version of nice:

TCSH(1) TCSH(1)Builtin commands

nice [+priority] [command]

9.4 Cron

Cron is a daemon that can automate tasks by periodically running them based on a time scheme. Its veryuseful for maintenance tasks. Cron takes a filename as a script for input that it incorporates internallyand checks periodically. The file is called a crontab file.

9.4.1 Making a Crontab File

The crontab file is by default a bash style script. It has extra parameters that tell cron when to startexecution. Here is the format:

mm hh dd MM D COMMAND–m

inute

–hour

–day–M

onth

–Weekday

–Executable

Figure 9.1: Crontab File Specification Format

Stars (*) are wildcards which always match, and rely on the other parameters to give cron timing queues.If you want to execute every day, just put a * in the day column.

A simple example of this would be to run a script above at 12 midnight every night:

# File: crontab-file. Execute me at 00.00 hours EVERYDAY!0 0 * * * /home/joe user/bin/backup >& /home/joe user/backup-log &

Notice that we have carefully routed stdout to a log file, otherwise cron could hang the job, or send usall the error by mail.

Another interesting property of the cron time spec is it’s ability to specify numbers in a modulo fashion.Just divide a * by a number to get cron to execute a periodic command:


9.4. CRON

# File: crontab-file. Execute backups every four hours.* */4 * * * /home/joe user/bin/backup >> /home/joe user/backup-log 2>&1

9.4.2 Submitting Cron Jobs

Once we have written our crontab file, we can submit it by doing a crontab crontab-file. crontab-l lists the current crontab entries for you.

<bash>: crontab crontab-file

<bash>: crontab -l# DO NOT EDIT THIS FILE - edit the master and reinstall.# (crontabs installed on Mon Feb 7 00:13:36 2000)# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)# Execute the job every four hours.# /home/joe user/crontab-file* 0 * * * /home/joe user/bin/backup >> /home/joe user/backup-log 2>&1

You may also edit the current crontab file with the crontab -e command, which will throw you intoyour default editor. When you exit, the crontab file is automatically updated.

9.4.3 Removing Cron Jobs

You can always remove your cron jobs by using the (-r) flags:

<bash>: crontab -r

<bash>: crontab -l

9.4.4 Exercises

Exercise 1:

Create a simple script named hello that prints out “hello joe user”. It should only have two lines:

#!/bin/bashecho Hello joe user you slacker...

This file should be placed in your personal ∼/bin directory, and should have the appropriate file modefor execution (hint: chmod). Make sure to test this script by hand before you go on.

Exercise 2:

Now create a crontab file named crontab-file that will execute hello every 2 minutes. Submit thiscrontab file as discussed above and check your mail on the even minutes to see if the scripts work.

Note: Cron’s default behavior is to send any unhandled output to your mail. If mail is not working forsome reason, you may send your output to a file or to your screen.




Chapter 10

Shell Scripting

There’s nothing more aggravating than having to type out long commands over and over again. Shellscripts are simple text files that list shell commands into a convenient “tool” which you can use moreeasily than typing in all those commands over and over.

Now that we know enough shell workings to be dangerous, we need to focus our attention to makingchanges permanent. It is essential to have easy to use environments and scripts to do complex tasks,because otherwise we are stuck repeating numerous small commands daily. The natural starting point isto customize our .bashrc and .cshrc files with a few aliases and a prompt or two.

10.1 Shell Initialization Files

We’re going to develop simple aliases and shell variables for our bash shell that will make life easier whenyou log into your Linux system.

10.1.1 Initialization Files for Bash

Start editing .bashrc now. Write in the following aliases and prompt definitions:

alias m=’less -EMsX ’alias h=’history | tail -40’alias d=’ls -l $* | less -EMsX’alias dt=’ls -blt $* |less -EMsX’alias Dt=’ls -aglt $* |less -EMsX’PS1=’[\s]:\u: ’

The $* represents all the remaining variables on the command line except for the calling command. It isequivalent to all subsequent command line variables, “$1 $2 ...”.

After you modify .bashrc you must do a “source .bashrc” to initialize your changes. Type aliasto see what the system’s definitions, and try out our new aliases.

Remember we can also have a .bash profile which is supposed to carry your environmental variables:

# .bash_profile# Get the aliases and functions


CHAPTER 10. SHELL SCRIPTING

if [ -f ˜/.bashrc ]; then. ˜/.bashrc

fi

# User specific environment and startup programsPATH=$PATH:/usr/local/bin:$HOME/binENV=$HOME/.bashrcUSERNAME=""

export USERNAME ENV PATH

Note the final export at the end of the file.

10.1.2 Initialization Files for Tcsh

Now lets make the analogous aliases in your .cshrc:

alias lester "less -EsX" # Use less as our pager..alias h ’history \!* | tail -40 | lester’alias m lester # Used to be more, now more is less.alias d ’ls -l \!* | lester’alias dt ’ls -lt \!* | lester’ # Sort by chronological order.alias Dt ’ls -laFt \!* | lester’ # Same as above, show dot (.xxx) files.set prompt="%n:%~: " # "user:/location/location/location: "

Try these out in tcsh after executing

<tcsh>: source .cshrc

Now submit these changes by typing source .cshrc. Type alias to see what the system’s definitions,and try some of them out. Both Tcsh and Csh use .cshrc and .login. The .login file is normallyused to store environment variables and path information, while .cshrc normally hold aliases and promptinformation.

10.2 Utility Shell Scripts

Our first goal is to create a script utility that will print files in a directory called/home/joe user/address. The syntax will be

phone.shell file1 file2 ....

Because of tcsh’s simple and clear syntax, we will first use the tcsh language to construct a solution.Afterwards, we show the equivalent bash solution.

10.2.1 Tcsh Solution

The first thing we want to do is to tell the script that it is a csh script, and what the acceptable numberof parameters are. In the following scripts make sure you ALWAYS remove the comment on the firstline, otherwise these examples will not function. In general, students should NOT write any commentsin these examples.

Start by editing a file named ∼/bin/phone.csh:


10.2. UTILITY SHELL SCRIPTS

#! /bin/csh -f # This is not a comment. Remove from 2nd #.#set a=$0 # $0 refers to the name of this script.if($#argv < 1 ) thenecho Usage: $a:t ’Name’exit(1)

endif

The term set a=$0 refers to the 0th component of the command line, which is the command file,phone.csh, in our case. The $a:t instructs csh to remove all the leading pathname components,since we are uninterested in seeing the full path. Test this by invoking the name of the script:

<bash>: phone.cshUsage: phone.csh Name

Did you remember to chmod our script so that it is executable?

So far the script does nothing but tell us if we have the right format. Next we will add a foreach linethat will give us control for each filename:

foreach name ($*)echo ~/address/$name

end

Now your script should print each filename you enter on the command-line, as a check in our progress.Again, test this script and check that the loop works:

<bash>: phone.csh red green blue

/home/joe user/address/red/home/joe user/address/green/home/joe user/address/blue

Now that we know it works, replace the echo command with the correct cat phrase. While we are atit, we will check for the existence of each file with the ( -e filename ) test. The previous foreach clauseshould NOW look like:

foreach name ($*)if (-e ~/address/$name) then # The (-e) flag checks for existence.cat ~/address/$name

elseecho "no such file $name"

endifecho "-------------------------" # Separate entries with a line.

end

The final script should resemble the following:

#! /bin/csh -f # This a csh script, omit this comment!# phone.csh # Comment to give the filename.



set a=$0 # $0 refers to the name of this script.if($#argv < 1 ) then # If right number of arguements,echo Usage: $a:t ’name’ # Echo the proper syntaxexit(1) # Exit with error code 1

endif # End if#foreach name ($*) # For each name on the command line.if (-e ~/address/$name) then # If file exists (-e),cat ~/address/$name # List the file

else # Elseecho "no such file $name" # Barf

endif # End ifecho "---------------------" # Separate entries with a line, cosmetic.

end # Bye-Bye

Lets check our program by creating an /address directory and several files there...

<bash>: mkdir ˜/address

<bash>: cd ˜/address

<bash>: echo hello1 says 123 > hello1 # Creates a file called hello1.



<bash>: phone.csh hello1 hello2 hello3

hello1 says 123------------------------hello2 says 456------------------------hello3 says 789------------------------

10.2.2 Bash Solution

The bash solution to the exercise is not as obvious as the csh one because of bash’s bizarre syntax.Call this file phone.bash:

#! /bin/bash # This is a bash script. Omit comment.if [ "$#" -lt 1 ] ; then # If number of arguments too small

echo Usage: $0 args # State the correct usageexit 1 # Exit with error status

fi # Note "fi" closes "if" statementsfor i in $@ ; do # Start a "do" loop on the $@ array

if [ -f ~/address/$i ];then # If the file exists.cat ~/address/$i ; # print it,

else # Elseecho There is no file $i # Error message if absent.

fi # End ifecho -------------- # Print a separator between filesdone # "done" closes the "do" loop


10.3. GENERAL SCRIPTING

10.2.3 Exercises

Exercise 1: Make the phone.bash version above work.

Exercise 2: Add an extra error message line if the file does not exist.

10.3 General Scripting

In this section we will create a several script programs named “doubletake” that will remove repeat entriesin a list. We will use several of our newly learned languages to accomplish that task.

10.3.1 awk version

Edit doubletake and put in the following lines:

#! /bin/csh -f# This script remove double in a sorted file. See the uniq manual.set file1 = $1 # $1 is going to be the filename.cat $file1 | awk ’{ if ($0 != last ) {print $0; }; }{ last = $0; }’

The script first stores the first command argument in $1’. It then cat’s the file into awk. Notice youneed the initial line of

#! /bin/csh -f

to tell the system that this is indeed a csh script.

Now the script that awk uses does 2 things. It first checks to see if the variable last was seen, if not,prints it. The second thing it does is to store the current line in last. Of course you could just type thewhole second line into the command line, but that’s not much fun if you have to do it 9 times a day.

10.3.2 Perl Version

#! /usr/bin/perl -wmy $line = ""; # declare this variable.foreach $filename (@ARGV) # cycle through each filename in the ARGV list.{

open (SF, $filename) or die "Can’t open $filename" ; # open the file and complain if not.while (<SF>) # For every line in the file{

chomp($_); # Remove tailing whitespace and/or linefeeds.if ( $line ne $_ ) # ne = not equal{

print "$line \n"; # Yes, print me.}$line = $_ ; # Reset $line.

}}



10.4 Script Automation

Shell scripts are great tools when you can use them, and can save a lot of time. If you need to runapplications or backups at odd hours or nightly, you really want to automate the task by using the crontools, which we have already covered in chapter 9.4.

The following exercise describes how to set up a script program to do your dirty work while you areplaying golf or sleeping.

10.4.1 Getting Scripts in Order

Before you submit a job for cron to handle, you must make sure you can run the job by hand. Thisinvolves making every command self sufficient by providing complete paths to the commands and settingthe right permissions. Lets take backups for example. You need to have privileges to write to the tapedevice as a normal user unless you plan to run your jobs as root .

Here is a typical script, /home/guest/bin/backup.sh that would back up your home directory to thefloppy device /dev/fd0. In this example, the floppy is being used as a character device just like a tapedrive would be used:

#! /bin/bash# This file is named backup.shcd /home/guest # Make sure you are at home./bin/tar cvfz /mnt/fd0 . # Back up your home directoryecho All done boss!

In this example, we have specified all paths to commands. This is required since your normal path/shellis not invoked in a script like this one. This script will create a lot of output which must be sent into alog file or /dev/null later when we make our crontab file.

10.4.2 Testing the Scripts

You must make sure you can run the scripts by hand before you try to invoke cron. This is typical forall programs that must be automated. Also make sure as mentioned before that all commands are givenby their full path names. BTW, why doesn’t the echo command need a path?

10.4.3 Exercises

Ex 1: Once you are confident that your script will run on its own two feet, you are ready to set it up withcron. Create a crontab file /home/guest/crontab.backup and submit it with

<bash>: crontab crontab.backup

as done in section 9.4. Ask for help if you get stuck.


Chapter 11

Networking and The Internet

Linux comes with all the TCP/IP networking goodies you would expect from a full-featuredUNIX box. It has telnet, ftp, mail, and many other services. Here’s a quick list of thebread-n-butter commands:

Networking TermsTCP/IP Transmission Control Protocol / Internet Protocoludp Best effort packet delivery protocoltcp Reliable byte stream protocolrpc Remote procedure callARP Address Resolution Protocolroute a network route to another machinehost Any machine on the networkclient The host that wants data or a serviceserver The host that provides data or a service

Remote access and file transferstelnet RemoteHost Remote login, often disabled for security reasonsrlogin RemoteHost Login to RemoteHost (insecure)rsh RemoteHostcommand Execute command on remote host. Used in conjunc-

tion with .rhosts file. Often disabled for security rea-sons.

ssh [user@]RemoteHost[command] Secure shell, replacement for telnetftp RemoteHost File Transfer Protocol transfers files between hostssftp RemoteHost Secure file transfer protocolrcp fromfile tofile Remote copy (insecure)scp file RemoteHost Secure copy file from here to RemoteHostscp Host1:f1 Host2:f2 Secure copy f1 from Host1 to Host2

Email.forward File in user’s home directory containing mail forward-

ing addressmail user[@RemoteHost] [¡ file ] Mail user agentpine Mail user agentelm Mail user agentmutt Mail user agent


CHAPTER 11. NETWORKING AND THE INTERNET

System Information:hostname -f Set or show the system’s host namehostid Report the numeric identifier of the local hostuname -a Report system informationuptime Report time since last rebootdu -h Estimate file space usage at and below the cwddf -h Report file system disk space usageps aux Report process statustop Report top processes

Superuser/Root:su [user] Switch to different user (user must be in /etc/passwd

file)sudo command Execute command as root (User must be in

/etc/sudoers file)chown owner : group file Change file/directory ownershiphalt This and the next two commands reboot the system

(must be superuser)rebootshutdownCtrl-Alt-Delete Reboot the sytem

TCP/IP Diagnostics:Some of these commands may require Superuser priviledges.

/etc/hosts Static table lookup for host names./etc/resolv.conf Static table with name of the DNS/etc/services Internet network services list.hostname -f Set or show the system’s host nameifconfig Display or configure local TCP/IP network interface con-

figurationping RemoteHost Send a “ping” to a remote host to check connectivitytraceroute RemoteHost Print the route packets take to network hostipmaddriptunnelarp Display or modify IP-to-Physical address translation ta-

bles used by ARPrarproute Print out the local routing table (check /sbin/route)dig RemoteHost Send domain name query to DNSnslookup RemoteHost Send domain name query to DNStcpdump -i ethX Listen to network traffic on ethXethereal Capture and analyze network trafficnetstat Display protocol statisticsnetwatch Monitor network connectionsnmap Network exploration and security monitoringwhois Client for the whois directory service

FTP examples

• ftp hostnameName userNamePassword userPassword


11.1. NETWORK COMMUNICATION

11.1 Network Communication

This section covers network communication from a user point of view. We won’t talk much about howto set up hardware or server configurations. This has already been done by your administrator.

11.1.1 Checking Network Status

Sometimes it is unclear if your network is working correctly. There are some easy commands that youcan use to check to see if a computer is “alive” on the network. The first and most important commandis ping which will send a test message to the computer who should respond:

[LocalHost]/home/joe user: ping grouchoPING groucho (192.168.1.3): 56 data bytes64 bytes from 192.168.1.3: icmp_seq=0 ttl=255 time=0.7 ms64 bytes from 192.168.1.3: icmp_seq=1 ttl=255 time=0.3 ms64 bytes from 192.168.1.3: icmp_seq=2 ttl=255 time=0.3 ms....

This shows that the host groucho is alive and well. It provides us with some timing information whichindicates the speed of the connection between LocalHost and RemoteHost.

If your DNS (see next section) is not working, you may not be able to ping by name, but perhaps bynumber. You can also ping by IP number like this:

[LocalHost]/home/joe user: ping 192.168.1.3PING 192.168.1.3 (192.168.1.3): 56 data bytes64 bytes from 192.168.1.3: icmp_seq=0 ttl=255 time=0.7 ms64 bytes from 192.168.1.3: icmp_seq=1 ttl=255 time=0.3 ms...etc...

If you don’t get any response, or do get an error, something is wrong with the network and you will notbe able to communicate with that computer.

11.1.2 DNS, Dig, and Nslookup

The Domain Name Service also known as DNS allows you, the users, to translate names like www.yahoo.cominto a number like 216.32.74.52 which is needed for your computer to communicate over the network.Networks are controlled by gremlins that only use numbers not names. Thus DNS is very important ifyou are going to use the network at all.

Who provides DNS service? Most of the time your ISP provides this service, especially if you are using amodem, cable modem, DSL, etc. If you are in a large corporate setting, your own company network willprovide DNS.

The main tool you need to check that your DNS is working is nslookup which can translate names tonumbers or vice-versa:

[LocalHost]/home/joe user: nslookup www.io.comServer: flure.pair.comAddress: 209.68.1.159



Name: www.io.comAddresses: 199.170.88.21, 199.170.88.41, 199.170.88.39

Nslookup tells us there are 3 names that go with www.io.com. Large sites like www.io.com andwww.yahoo.com often have many numbers because they maintain several servers to handle all the re-quests to their very busy sites. The server flure.pair.com is our DNS server as seen by our local DNSsetup.

DNS can also translate numbers into names:

[LocalHost]/home/joe user: nslookup 199.170.88.21Server: flure.pair.comAddress: 209.68.1.159

Name: www-02.io.comAddress: 199.170.88.21

This was one of the www.io.com sites listed in the first example. If DNS can’t find your name, there islittle chance you can connect through the internet to it:

[LocalHost]/home/joe user: nslookup belse.comServer: flure.pair.comAddress: 209.68.1.159

*** flure.pair.com can’t find belse.com: Non-existent host

Sorry, you are SOL. No can do.. (unless you know the number already) Maybe try www.belse.com.

Sometimes you can connect by IP number even when DNS doesn’t work. Try to connect to www.io.comwith one of the numbers we found. Note that Linux has a text browser called Lynx that is many-timesconvenient.

11.1.3 The “R” Commands

The so-called R-Commands are rlogin, rcp, rsh, and friends. These are powerful and dangerouscommands, and should be used with great care. Many systems exposed to the Internet disable, as theyshould, these commands because data is completely open to public scrutiny over that network. Now day,some of these commands have secure versions whose data is encrypted, or the commands are replaced byssh, the secure shell.

In order to execute remote commands on a remote machine without passwords, you set up a file called.rhosts (in your home dir) that looks like the following:

# .rhosts file for RemoteHostLocalHost joe user

and on the local machine a complimentary setup allows you to work from there.


11.2. USING X-WINDOWS OVER THE NETWORK

# .rhosts file for LocalHostRemoteHost joe user

Now try to copy a file from here to there:

rcp filename RemoteHost:rcp filename RemoteHost:. # Same as above.

Now copy a file from RemoteHost to LocalHost:

rcp RemoteHost:fileA . # Copy the file to . with same name.rcp RemoteHost:fileA fileB # Renames the file

Now user rlogin to log into RemoteHost:

rlogin RemoteHostLast login: Sat Jan 15 02:53:13 from beppoYou have mail.[RemoteHost]/home/joe user:ls -agltotal 3236drwxr-xr-x 20 joe user users 4096 Jan 14 09:45 .drwxr-xr-x 5 root root 4096 Jan 14 08:42 ..-rw------- 1 joe user users 1748 Jan 14 06:59 .ICEauthority-rw------- 1 joe user users 101 Jan 14 08:35 .Xauthority-rw-r--r-- 1 joe user users 1506 Jan 14 08:18 .Xdefaults-rw------- 1 joe user users 133 Jan 11 15:56 .bash_history....etc....

11.2 Using X-Windows Over the Network

One great thing about X is that it has networking built right into it. You can use any X application fromanywhere on your network. All you need to do is set a few parameters that allow communications.

11.2.1 Connecting X to another client

There are two things you must do to get X to work over a network. First you need to allow other machinesto connect to your workstation. One way to do this is to use xhost RemoteHost, where RemoteHostis the machine you want to communicate with.

Second, you must telnet (or ssh) to a another workstation, and set the DISPLAY environment variableto your workstation like this:

[LocalHost]/home/joe user:xhost + RemoteHost[LocalHost]/home/joe user:telnet RemoteHostRed Hat Linux release 6.1Kernel 2.2.12-20 on an i586login: joe userPassword: ******



Last login: Thu Jan 13 12:13:37 from bistro.rr.com[RemoteHost]/home/joe user:setenv DISPLAY LocalHost:0[LocalHost]/home/joe user:xcalc -title RemoteHostCtrl-C[RemoteHost]/home/joe user:exit[LocalHost]/home/joe user:

Remember that RemoteHost and LocalHost need to be changed to whatever your workstation is named.

What you should have now is a process that is run by the remote system! It should be clear that thisis powerful. In the next chapter we’ll talk about other networking features.. If you put a line like thefollowing in your .login or .bashrc, your DISPLAY variable will be set automatically for you. Play withthis for a second:

<tcsh>: setenv DISPLAY ‘who am i | perl -ne ’/$(.+)$/ ; print $1’‘”:0”or in bash,

<bash>: set DISPLAY= ‘who am i | perl -ne ’/$(.+)$/ ; print $1’‘”:0”

11.2.2 Securing Your X Connection

Above we talked about xhost and how it allows other machines to connect to your X-Server on yourworkstation. There is a security flaw that allows ALL people from the RemoteHost to connect to yourworkstation. This “hole” has been used to exploit many a computer.

Fortunately there is a secure alternative to using xhost called xauth. The way to use xauth is to firstexecute xauth list on the RemoteHost, grab the information it produces with your mouse (left clickand drag), and then paste it (middle click) on your workstation right after typing xauth add.

[RemoteHost]/home/joe user: xauth listbistro.marx:0 MIT-MAGIC-COOKIE-1 6c7d8ba30a

bistro/unix:0 MIT-MAGIC-COOKIE-1 7bf452301a230498localhost:0 MIT-MAGIC-COOKIE-1 a43f0989dblocalhost:10 MIT-MAGIC-COOKIE-1 3c08b9241d

On your (local) workstation:

<bash>: xauth add bistro/unix:0 MIT-MAGIC-COOKIE-1 7bf452301a230498

11.3 Network Security for Users

Network security from a user point of view is simple: Don’t use easy-to-guess passwords, and don’t leavethe “R” commands open to attack.

Hard to guess passwords don’t resemble any word and should have non alphabetic characters in them.Hello is not a good password, but h3l9Xz is. Hackers always use crack which uses a dictionary to guesspasswords, and Hello would be cracked in a few seconds.

As far as security is concerned, you need only remember a couple of things.


11.3. NETWORK SECURITY FOR USERS

• Always keep your home directory (˜) protected.<bash>: chmod -R 700 ∼/

• Don’t use R-commands over an insecure network, as hackers can see most everything. If youhave secure shell OpenSSH and secure versions of R-commands, you are better off. Check http://www.openssh.org for info on OpenSSH.

• Programs like telnet show your passwords over the Internet. Use OpenSSH from http://www.openssh.org

• Never tell anyone your password, and if you do, change it the next day.

• chmod 600 .rhosts

• Change your password every few months.

• Never write your password down or identify it as a password.

• Pick an account names and passwords that are difficult to guess. Passwords should have uppercase,lowercase, and numeral characters. Do not use a dictionary word as a password or even part of apassword. You can check your password tables agains crack at ftp://ftp.cerias.purdue.edu/pub/tools/unix/pwdutils/crack.

• Check permissions on files periodically. Most files should be 600 or 700.




Chapter 12

Program Development inUnix/Linux

Program Development Toolsgas GNU Assemblercc file.c Compile a C programgcc file.c Compile a C programg++ file.cpp Compile a C++ programgdb The GNU program debugger of C and C++make Maintain, update, and regenerate program and files based

on user defined dependencies found in the file makefilecat -A makefile Show non-printing characters (especially tabs and tailing

spaces) in makefilercs options file ... The revision control system RCSci Check a file into the RCS directoryco Check out a file from the RCS directorycvs [opts] cmd [opts] [args] The concurrent version system (CVS) which allows you

to keep old versions of files, a log of who, when, and whychanges occurred, etc., like RCS and SCCS

script Capture terminal interactions into a file called typescriptexit Exit a script session

Many compilers are available for Linux including versions of C++, C, Perl, Python, awk, and manyothers. It also has text formatting solutions of TEX and LATEX which is what we use to format thisdocument. There are many other freely available additions like Java, Fortran, and Lisp.

Here is what our systems spits out with man -k compiler:

cccp, cpp (1) - The GNU C-Compatible Compiler Preprocessor.f4rpcgen (1) - an RPC protocol compilerg++ (1) - GNU project C++ Compilergcc, g++ (1) - GNU project C and C++ Compiler (egcs - 1.1.2)gcc, g++ (1) - GNU project C and C++ Compiler (egcs-1.1.2)tic (1m) - the terminfo entry-description compilerzic (8) - time zone compilerB (3) - The Perl CompilerB::Bytecode (3) - Perl compiler’s bytecode backend


CHAPTER 12. PROGRAM DEVELOPMENT IN UNIX/LINUX

B::C (3) - Perl compiler’s C backendB::CC (3) - Perl compiler’s optimized C translation backendB::Deparse (3) - Perl compiler backend to produce perl codeO (3) - Generic interface to Perl Compiler backends

Here is what our systems gives for man -k language:

Tcl (n) - Summary of Tcl language syntax.bc (1) - An arbitrary precision calculator languagegawk (1) - pattern scanning and processing languageperl (1) - Practical Extraction and Report Languageperlxs (1) - XS language reference manualpython (1) - an interpreted, object-oriented programming language

12.1 C & C++ Compilers

12.1.1 C programs

Edit program try.c with favorite text editor.

#include <stdio.h>int main() {

printf("running my first program\n");return(0);

}

Compile program

<bash>: cc try.c

Run program

<bash>: a.out

Print program

<bash>: lpr try.c

Capture program and output

<bash>: script

<bash>: cat try.c

<bash>: cc try.c

<bash>: a.out

<bash>: exit

Print session

<bash>: lpr typescript

12.1.2 C++ programs

Edit source program in favorite editor then


12.2. DEBUGGING WITH GDB

<bash>: gpp YourSourceFile.cpp

<bash>: lsa.out YourSourceFile<bash>: a.outYour program in action

12.1.3 Command-Line Arguments

12.1.4 Standard C Libraries

12.1.5 Libraries and Archives

12.1.6 Error Handling in C

12.1.7 Error Recovery

12.2 Debugging with gdb

• Interactive debugging

• Basic dbx commands

– trace

12.2.1 Examining Core Dumps

Sample C session

<bash>: cc

Sample C++ session

<bash>: g++

12.2.2 The Debugger

Sample debugging session

<bash>: cc -

<bash>: gdb

12.3 The Unix make Facility

make is a command generator which uses a description file to create a sequence of commands for executionby the Unix shell. It is commonly used as a tool for maintaining component modules in a software systemand for building an executable from group of files.

The make command performs actions defined in a description file usually named makefile or Makefile.The makefile is placed in the root directory of the collection of files comprising the application. The makeutility determines which files to update by using the list of dependences described in the makefile and a



file’s time stamp field (last-modified time). The makefile consists of suffix definitions, macro definitions,dependencies, and commands. Each command behaves as if it were invoked within a shell.

Entries in a makefile have the form:

target : prerequisites(tab) commands

where both the prerequisites and commands are optional. The commands are executed only if the targetis not upto data with respect to the prerequisites. The target is dependent on one or more files calledprerequisites each of which may have other files as prerequisites. The description file defines all of theintermediate steps necessary to accomplish the task. The first character of a command line must be atab and no other line may begin with a tab. The following command may be used to check the tabs inthe description file.

<bash>: cat -v -t -e makefile

or simply

<bash>: cat -A makefile

The makefile is run by entering the command

<bash>: make

or

<bash>: make target

The make utility determines what parts of a program need to be rebuilt and then carries out the appro-priate set of actions producing the latest version of the target.

For more details than are provided in this section, see [4] or visit the GNU Project web site server:www.gnu.org/software/make/make.html.

The example in Subsection 12.3.3 is a portion of the makefile used in the preparation of this text.

12.3.1 The Description File

The makefile consists of one or more lines each of which describes either a dependency rule, a command,a macro, or a suffix rule. A dependency is a target followed by a list of items on which the target isdependent. A command is an action that following a dependency describes how the target file is createdfrom the dependencies. The line containing a command always begins with a tab character. The targetfiles are intermediate files and the ultimate executable file that results from linking the object filestogether. The dependent files are source files ( .c, .cpp or .h files) or object files (.o). A macro is a namefor a set of files. The syntax of the makefile rules is described in Table 12.1.

Commands

Commands have the form:

TAB command [; command]

Additional commands may be placed on separate lines or separated with semicolons if placed on the sameline.


12.3. THE UNIX MAKE FACILITY

makefile ::= suffix rule | dependency rule | command | macro definition

suffix rule ::= .suffix[.suffix] :

dependency rule ::= target : [prerequisites] [; [command]]

command ::= tab Shell command

macro definition ::= name = string

Table 12.1: Makefile syntax

Dependence Rules

Dependence rules have the form:

target : prerequisites(tab) command

The prerequisites may be placed on separate lines provided the previous line ends in a backslash. Thelist of prerequisites may be followed by a command which is either preceded by a semicolon or enteredon a new line.

Macros

A macro definition has the form:

NAME = string

It is customary to use all caps in the name of the macro. A tab may not precede a macro definition and itis terminated by a new line. A macro must be defined before is it used and it is referenced by ${NAME}.

Suffix Rules

Suffix rules have the form:

.suffix.suffix :(tab) command

Files ending with the first suffix can be prerequisites of files that have the same name but end with thesecond suffix. No spaces are allowed between the suffixes. The command used to transform a file withthe first suffix to one of the second suffix may be placed on the next line. See the example in subsection12.3.3.

The suffix dependences for C and C++ are built into make so need not be used.

12.3.2 Short Example

12.3.3 Larger Example

The following example is the make file used in preparing this text and it illustrates these concepts.



# Macro definitionsSHELL = /bin/sh # Limit commands to Bourne shell

# Suffixes significant for typeset documents.SUFFIXES : .aux .dvi .idx .log .tex .toc .pdf .ps .tgz

# Suffix relationships and build commands# Create .dvi files.tex.dvi: #latex must be run twice to resolve forward references

latex $< ; latex $<

# Create .ps (postscript) files from the .dvi file.dvi.ps :

dvips $<# Create .pdf files from the .tex file.tex.pdf :

pdflatex $<

# SubdirectoriesCOMP = CompilerDOC = DocPrepFDIR = FundamentalsPROG = Programming

# File dependencies (correspond to target input and includes)# Touch command necessary to force rebuild

# Lines deleted that refer to files use in other texts

# Fundamentalsfundamentals.tex : definitions.tex defsAaby.tex\

${FDIR}/X-Windows.tex\${FDIR}/bibliography.tex\${FDIR}/commandline.tex\${FDIR}/copyright.tex\${FDIR}/environment.tex\${FDIR}/filesystem-management.tex\${FDIR}/filesystem.tex\${FDIR}/glossary.tex\${FDIR}/introduction.tex\${FDIR}/process.tex\${FDIR}/scripting.tex\${FDIR}/security.tex\${FDIR}/software.tex\${FDIR}/tcpip.tex\${FDIR}/terms.tex\${FDIR}/text-tools.tex\${FDIR}/unixCommands.textouch fundamentals.tex

# Build all of the documents.


12.4. CONCURRENT VERSION CONTROL (CVS)

# This make file serves to build four texts

all : compiler docprep fundamentals programming

# Build individual documents

compiler : compiler.tex compiler.dvi compiler.ps compiler.pdfdocprep : docprep.tex docprep.dvi docprep.ps docprep.pdffundamentals : fundamentals.tex fundamentals.dvi fundamentals.ps fundamentals.pdfprogramming : programming.tex programming.dvi programming.ps programming.pdf

# Create src package - run make clean firstrelease :

cd ..; /bin/tar cfz linux.tgz Linux; mv linux.tgz Linux/.

# Remove all non-source files.clean :

/bin/rm -f *.aux *.dvi *.idx *.log *.toc *.pdf *.ps *.tgz

12.4 Concurrent Version Control (CVS)

CVS (Concurrent Versions System) is a tool that is used to track changes to any thing that can bestored as a set of files. It permits multiple developers to coordinate changes, track versions, and permitsimultaneous development on different versions of the same code.

For more details than presented here, see [5] or visit www.cvshome.org.

Concurrent Version Control (CVS)cvs The Concurrent Versioning System (CVS) pro-

vides versioncontrol in a multi-developer, multi-directory environ-ment.

cvs [*] -H [cmd] CVS help optionCVSAREA= sanbox .profile or .bashrc environmental variable.CVSROOT= repository for the sandbox and source repository respectively

<tr><td colspan="2">CVSAREA={\it sandbox} CVSROOT={\it sourceRepository}</td>

<td>.profile or .bashrc file environmental variable entries for sandboxand source repository respectively.</td>

</tr>



<tr><td colspan="2">setenv CVSAREA {\it sandbox} setenv CVSROOT {\it sourceRepository}</td>

<td>.cshrc file environmental entry for sandbox and source respositoryrespectively.</td>

</tr><tr><td>cvs</td><td>[-d {\it sourceRepository}] init</td><td>Create the source repository</td>

</tr><tr><td>cd cvs</td>

<td>$CVSAREA/{\it projectDirectory} import -m {\it msg repository vendor release}</td>

<td>Check project files into repository from the sandbox.</td></tr><tr><td>cd cvs</td>

<td>$CVSAREA/{\it projectDirectory} checkout {\it projectDirectory}</td>

<td>Check project files out of the repository and into the sandbox.</td></tr><tr><td>cvs</td><td>commit -m msg file</td><td></td>

</tr>

12.5 Packaging Your Software - Tar

The file compression utilities are used to decrease storage requirements and to reduce the time it takesto transmit files.

compress file compressedFile.Z The compress utility. Compressed files should have a .Z suffix.

uncompress file.Z The uncompress utility.

gzip file zippedFile.gz The GNU zip utility. Zipped files should have a .gz suffix

gunzip file.gz The GNU unzip utility.

The tar (tape archive) utility is used to construct a linear representation of the unix hierarchical filestructure and restore the hierarchical structure from the linear representation. In addition, it may beused to compress and decompress the files. It was originally designed to save file systems on tapes as abackup.

tar cf theArchive.tar theDirectory - creates a tar file of theDirectory and its contents anplaces it in the working directory.


12.6. SESSION CAPTURE

tar cfz theArchive.tgz theDirectory - creates a gziped tar file of theDirectory and its contentsan places it in the working directory.

tar xf theTarFile.tar - extracts the tar file into the working directory, duplicating the tarreddirectory structure.

tar xfz theTarFile.tgz - gunzips and extracts the tar file into the working directory

tar tzf theTarFile.tgz - List of the contexts of the tar file

tar xf theTarFile FileOfInterest - extract the file of interest from the tar file.

12.6 Session Capture

The following illustrates how to capture and print a terminal session to provides a program listing andevidence of compilation and execution.

<bash>: script

<bash>: cat yourProgram.c...<bash>: cc yourProgram.c

<bash>: a.out...<bash>: exit

<bash>: lpr typescript



Consult the following websites for current software information:

http://www.freshmeat.net for Linux software.http://www.sourceforge.net for OpenSource software.http://www.newsforge.com Open Source News.http://www.seul.org Linux software for education and science.http://www.slashdot.org Don’t forget Slashdot.

12.7 Exercises

1. Create makefile for a small C or C++ program. The program should consist of several files. Yourmakefile should insure that only the necessary files are recompiled after changes.

2. Package the program and makefile of the first exercise in a gzipped tar file.


Appendix A

Glossary

alias . . . . . . . . . . . . . . . . . . . . A shortcut name for another commandASCII . . . . . . . . . . . . . . . . . . American Standard Code for Information Interchange = textbackground . . . . . . . . . . . . Processes work silently here and dont bother youbash . . . . . . . . . . . . . . . . . . . . The GNU Bourne-Again SHell.bin . . . . . . . . . . . . . . . . . . . . . Refers to an executable ”binary”, usually a directorycwd . . . . . . . . . . . . . . . . . . . . . Your Current Working Directorycursor . . . . . . . . . . . . . . . . . . The active point for editing, usually a highlighted rectangledaemon . . . . . . . . . . . . . . . . A program in the background and provides a service, like HTTPDdevice . . . . . . . . . . . . . . . . . . A physical or logical component of a computerdirectory . . . . . . . . . . . . . . . A folder in the computerDNS . . . . . . . . . . . . . . . . . . . . Domain Name Service: domain names for IP numbers & vice-versaenvironment . . . . . . . . . . . A set of parameters your shell works inexecute . . . . . . . . . . . . . . . . To enter a commandext2 . . . . . . . . . . . . . . . . . . . . The native Linux filesystemfile permissions . . . . . . . They control who reads, writes, and executes filesforeground . . . . . . . . . . . . . Where processes with your consoleFTP . . . . . . . . . . . . . . . . . . . . File Transfer Protocol, used to move files over networksGID . . . . . . . . . . . . . . . . . . . . Group ID of a file or processGUI . . . . . . . . . . . . . . . . . . . . Graphical User Interface. Think Windows, but better!$home . . . . . . . . . . . . . . . . . . Your home directoryhost . . . . . . . . . . . . . . . . . . . . A computer on a networkjob . . . . . . . . . . . . . . . . . . . . . A process running in your shellkernel . . . . . . . . . . . . . . . . . . The core software that runs Linuxman . . . . . . . . . . . . . . . . . . . . The MANual pagesmount . . . . . . . . . . . . . . . . . . Attach a disk partition to Linuxmount point . . . . . . . . . . . An empty directory you attach (mount) a partition tonetwork . . . . . . . . . . . . . . . . A series of communication channels for computerspartition . . . . . . . . . . . . . . . A subdivision of a hard disk. A slice of pie.path . . . . . . . . . . . . . . . . . . . . /This/is/where/you/are/ in the filesystem (think folder)PID . . . . . . . . . . . . . . . . . . . . Process IDpriority . . . . . . . . . . . . . . . . Number indicates how important a process is to the kernelprocess id . . . . . . . . . . . . . . Every running program has such a numberprompt . . . . . . . . . . . . . . . . . That piece of text just before you type commands. Customizable.regular expression . . . . A pattern used for searching text.route . . . . . . . . . . . . . . . . . . . The path data takes through a nework.


APPENDIX A. GLOSSARY

script . . . . . . . . . . . . . . . . . . . An shell program much like a ”batch file” in MS-DOS.shell . . . . . . . . . . . . . . . . . . . . A program that reads/executes command from users.server . . . . . . . . . . . . . . . . . . A computer that provides data and/or servicesstderr . . . . . . . . . . . . . . . . . . Standard Errorstdin . . . . . . . . . . . . . . . . . . . Standard Inputstdout . . . . . . . . . . . . . . . . . Standard Outputstring . . . . . . . . . . . . . . . . . . Just a text wordswap . . . . . . . . . . . . . . . . . . . Memory on a disktcsh . . . . . . . . . . . . . . . . . . . . The C-Shell, very friendlyUID . . . . . . . . . . . . . . . . . . . . User ID. Each user has one and files get tagged with it.username . . . . . . . . . . . . . . Your login nameuser . . . . . . . . . . . . . . . . . . . . A computer userxterm . . . . . . . . . . . . . . . . . . A console program for X-Windows


Appendix B

Unix/Linux Commands by Location

Commands are found in several different locations as specified by the filesystem hierachy standards FSH2.2; see www.pathname.com/fhs. Not all commands in the standard are listed here. Not all commandslisted here are covered in the text.

Location of Commands in the Filesystem Hierarchy/bin Essential user commands (for use by all users)/sbin Essential system commands for use by system administra-

tor/usr/bin Primary directory of user commands/usr/sbin Non-essential standard system commands used by system

administrator/usr/local/bin Host specific commands (for use by all users)/usr/local/sbin Host specific system commands used by system adminis-

trator/home/userid/bin User’s frequently used binaries and scripts.

The following are an abbreviated listing of popularly used commands found in the directories.

/bin: Commonly used essential commandsarch Print machine architectureawk Pattern scanning and processing languagebash GNU Bourne-Again Shellcat Concatenate files and print on the standard outputchgrp Change group ownership of fileschmod Change access permissions (mode) of fileschown Change ownership of filescp Copy files and directoriesdate Print or set system date and timedd Convert and copy a filedf Report filesystem disk space usagedir List directory contents (also ls and vdir)du Estimate file space usageecho Display line of textegrep Print lines matching a pattern (grep -E)fgrep Print lines matching a pattern (grep -F)grep Print lines matching a pattern


APPENDIX B. UNIX/LINUX COMMANDS BY LOCATION

groups Print the groups a user is ingunzip compress or expand filesgzcat compress or expand filesgzip compress or expand fileshostname Set or print the name of the current hostkill Terminate a processlast Show listing of last logged in usersln Make links between filesls List directory contents (also dir and vdir)mail Send and receive mailmesg Write control access to terminalmkdir Make directoriesmore file View file contents a page at a timemv A B Move (rename) file A to Bnetstat Print network informaiton and statisticsping Send ICMP ECHO REQUEST to network hostspstree Display a tree of processesps Report process statuspwd Print name of current/working directoryrmdir Remove empty directoriesrm Remove files or directoriessed Stream editorsh Execute following script in the bash shellsu Change user Id or become super-usersync Flush filesystem bufferstar Tape archiving utilitytcsh C shell with file name completion and command line edit-

inguname -a Print system informationwall Send a message to everybody’s terminalwc Print the number of bytes, words, and lines in fileszcat Compress or expand fileszmore File rerusal filter for viewing compressed text

/sbin: Commonly used essential system commandsarp Maninpulate system ARP tablearping Send ARP REQUEST to a neighbor hosthalt Stop the system (also reboot, poweroffifconfig Configure a network interfaceinit Process control initialization (also telinit)rarp Manipulate system RARP tablereboot Stop the system (also halt and poweroffroute Show/manipulate the IP routing tablerunlevel Find the current and previous system runlevelshutdown Bring the system downtelinit Process control initialization (also init)

/usr/bin: Commonly used commandsCommands available depend on the distribution.

aspell Spell checker (also ispell)


autoconf Generate configuration scriptsautomake Creates Makefiles from template filesawk Pattern scanning and processing language (also gawk)bison GNU parser generator (yacc replacement)c++ C and C++ compiler (also cc, gcc and g++)cancel Cancel print jobscat file(s) Concatinate files, result to standard outputcc C and C++ compiler (also c++, gcc, and g++)chfn Change user name and informationchgrp Change group ownership of fileschown Change ownership of fileschsh Change user’s login shellgunzip compress or expand filescompress compress filescp Copy files and directoriescrontab Maintain crontab files for individual userscvs Concurrent versions systemdc An arbitrary precision calculatordd Convert and copy a filedf Report free space remaining in the file systemdiff Find differences between two filesdig DNS lookup utilitydir List directory contents (also ls and vdir)du Report disk space used in the current directory and all

subdirectoriesemacs Emacs text editor and system interfaceethereal Interactively browse network trafficfind Search for files in a directory hierarchyflex Fast lexical analyzer generatorfortune Print a random adageg++ C and C++ compiler (also c++, cc, and gcc)gawk Pattern scanning and processing language (also awk)gcc C and C++ compiler (also c++, cc, and g++)gdb The GNU debuggerggv Display postscript and pdf fileshostid Print the numeric identifier for the current hosthostname Set or print the name of the current hosthost DNS lookup utilityinfo Read Info documentskill Terminate a processlastlog Examine lastlog fileless Similar to more but allows backward movementlex Lexical analyser generator (see Flex)ln Make links between fileslocate Index and search for files on your systemlogname Print user’s login namelook Display lines beginning with a given stringlpq Show printer queue statuslprm Cancel print jobslpr Print file using default printerlpstat Print printing system status information



lp Print filesls List directory contents (also dir and vdir)lynx Text terminal browser for the WWWm4 Unix macro processormake Utility to maintain groups of programsman Format and disply the on-line manual pagesmkdir Make directoriesmv Move (rename) filenasm Portable 80x86 assemblernice Change execution level of tasknohup Run a command immune to hangups, with output to a

non-ttynslookup Deprecated. Use dig insteadpasswd Change user passwordperl Practical extraction and report languagepg Browse pagewise through text filespr Convert text files for printingpstree Display a tree of processespython Interpreted, interactive, O-O programming languagercs Revision control systemrenice Alter priority of running processesrmdir Remove empty directoriesrm Remove files or directoriesscp Secure copyscript Make a typescript of a terminal sessionsftp Secure file transfer programsort Sort lines of text filesssh RemoteHost Secure remote loginsudo cmd Execute command as Superuser (requires a password)sync Flush filesystem bufferstail file View last lines of a filetracepath Trace path to a network host discovering MTU along this

pathtraceroute RemoteHost Find the route to RemoteHostuncompress expand compressed filesunzip List, test, and extract compressed files in a ZIP archiveuptime Tell how long a system has been runningvim Text editor (also vi)vi Text editor (also vim)wc Print the number of bytes, words, and lines in fileswhatis Search the whatis database for complete wordswhereis Locate the binary, source, and manual page files for a

commandwhich Show full path of shell commandswhoami Print effective useridwho Show who is logged onwhois Search the WHOIS databasew Show who is logged on and what they are doingzip Package and compress files

/usr/sbin: Commonly used system commands


adduserlpcnfsstattcpdump

Headingkeyboard

definition key

key definition

Term . . . . . . . . . . . . . . . . . . . Glossary definition

command options description

<Super>: command # Superusr command

/home/joe user: command # User level command




Appendix C

Unix/Linux Commands by Category

The Filesystem/ Root directory. Current directory.. Parent directory

Home directoryuser Home directory

* Wild card: Match any sequence of characters? Wild card: Match any single characterls [D] List the contents of the working directory or directory Dls -l List the contents of the working directory in long formatln [opts] Existing New Create a link from New to Existingcd Change working directory to home directorycd .. Change working directory up one levelcd pathname Change working directory to pathnamepwd Path to the working directorycp A B Create a copy of A named Bmv A B Move (rename) file A to Bmkdir A Make a directory named Arm A Remove file Arm -r * Remove (recursively) all files in working directoryrmdir D Remove directory Ddu Report disk space used in the current directory and all

subdirectories

Compression and Archives:compress file file.Z Compress a fileuncompress file.Z file Uncompress a filegzip file file.gz Compress a filegunzip file.gz file Compress a filetar Tape archive untilitytar cf Archive.tar Dir Create an archive (Archive.tar) of the directory (Dir) and

its contents.tar cfz Archive.tar Dir Create a compressed archive (Archive.tar) of the directory

(Dir) and its contents.tar xf Archive.tar Dir Extract the tar file into the working directory.


APPENDIX C. UNIX/LINUX COMMANDS BY CATEGORY

tar xzf Archive.tgz Dir Extract the compressed tar file into the working directory.tar tzf Archive.tgz Dir List the contents of the tar file.tar xf Archive.tgz FileofInterest List the contents of the tar file.

Filesystem Managementdudfddfdiskmountsyncumountmkfsmkswapswapon




Processes and MultitaskingC-c Abort current taskC-z Suspend current taskbg Place suspened task in backgroundjobs List currentjobs processesfg [JobId] Bring background task or suspeneded to foregroundtask & Launch task into backgroundps aux Report process status for all processeskill -9 Pids Abort processes with Pids


top Display and update top CPU processesnice Run a task with a modified scheduling priorityrenice Alter priority of a running tasknohup cmd Run cmd as a background task immune to hangups.crontab -e user Edit user’s cron file to schedule background tasks

Protection and Security.nofinger File in users home directory created using the touch com-

mand limits finger to local machinepasswd Change user passwordumask Set default file access permissions (permissions taken awaychmod [ugo][+|-][rwx] file Change file mode (permissions)chmod octalmode Change file mode (permissions)ssh [user@]RemoteHost[command] Secure shell, replacement for telnetsftp RemoteHost Secure file transport, replacement for ftpvim -x file Create encrypted file. You will be promped for an

encryption key

Networking TermsTCP/IP Transmission Control Protocol / Internet Protocoludp Best effort packet delivery protocoltcp Reliable byte stream protocolrpc Remote procedure callARP Address Resolution Protocolroute a network route to another machinehost Any machine on the networkclient The host that wants data or a serviceserver The host that provides data or a service

Remote access and file transferstelnet RemoteHost Remote login, often disabled for security reasonsrlogin RemoteHost Login to RemoteHost (insecure)rsh RemoteHostcommand Execute command on remote host. Used in conjunc-

tion with .rhosts file. Often disabled for security rea-sons.

ssh [user@]RemoteHost[command] Secure shell, replacement for telnetftp RemoteHost File Transfer Protocol transfers files between hostssftp RemoteHost Secure file transfer protocolrcp fromfile tofile Remote copy (insecure)scp file RemoteHost Secure copy file from here to RemoteHostscp Host1:f1 Host2:f2 Secure copy f1 from Host1 to Host2

Email.forward File in user’s home directory containing mail forward-

ing addressmail user[@RemoteHost] [¡ file ] Mail user agentpine Mail user agentelm Mail user agentmutt Mail user agent

System Information:



hostname -f Set or show the system’s host namehostid Report the numeric identifier of the local hostuname -a Report system informationuptime Report time since last rebootdu -h Estimate file space usage at and below the cwddf -h Report file system disk space usageps aux Report process statustop Report top processes

Superuser/Root:su [user] Switch to different user (user must be in /etc/passwd

file)sudo command Execute command as root (User must be in

/etc/sudoers file)chown owner : group file Change file/directory ownershiphalt This and the next two commands reboot the system

(must be superuser)rebootshutdownCtrl-Alt-Delete Reboot the sytem

TCP/IP Diagnostics:Some of these commands may require Superuser priviledges.

/etc/hosts Static table lookup for host names./etc/resolv.conf Static table with name of the DNS/etc/services Internet network services list.hostname -f Set or show the system’s host nameifconfig Display or configure local TCP/IP network interface con-

figurationping RemoteHost Send a “ping” to a remote host to check connectivitytraceroute RemoteHost Print the route packets take to network hostipmaddriptunnelarp Display or modify IP-to-Physical address translation ta-

bles used by ARPrarproute Print out the local routing table (check /sbin/route)dig RemoteHost Send domain name query to DNSnslookup RemoteHost Send domain name query to DNStcpdump -i ethX Listen to network traffic on ethXethereal Capture and analyze network trafficnetstat Display protocol statisticsnetwatch Monitor network connectionsnmap Network exploration and security monitoringwhois Client for the whois directory service

Personal Information and Environment.login File in home directory which Personalize the csh and tcsh environment. Read upon login..cshrc Files in home directory which personalize the csh, tcsh environment. Read upon login..profile File in home directory which personalizes the sh, bash environment.forward File in user home directory containing email forwarding information


.signature Automantically added to your email

.plan File in user home directory containing public plans

.project File in user home directory

.nofinger If present denies existence of user outside of local host. Created by touch .nofinger

finger user[@RemoteHost] Lookup user information contained in password file andthe following files.

chsh Change user’s login shellchfn Change use name and information in /etc/passwd filesource file Update environmental information from file (.cshrc).

Systemsu user Switch user (requires a password)sudo cmd Execute command as Superuser (requires a password)chown user:group file Change file ownership (must be Superuser)df Report free space remaining in the file systemhalt System shutdownreboot System rebootCtl-Alt-Del System reboot

IO Redirection, Pipes and Filterscmd < file Use file as input to cmd.

Example: mail user@RemoteHost< filecmd > file Create file containing the output of cmd.

Example: cat file1 > file2; equivalent to cp file1 file2cmd >> file Append output of cmd to file filecmd1 | cmd2 Connect two processes with a pipe. The output of the

first is piped to the input of the second

Colaboration and Productivity Toolscal [[month] year] Display calendar for month or yeardate Display or set system date and timew Display information on users currently logged onwho Display information on users currently logged onwhoami Display userid of current usertalk user@hostname Initiate a talk session with another userwrite user@hostname Send a message to another usermsg y | n Enable, disable reception of talk and message from other

usersmail user@hostname ¡ file Send file as an email message to user@hostnameispell file Look for misspellings in file and offer suggestionsaspell file Aspell (replacement for Ispell) looks for misspellings and

offers suggestionslook prefix Display words beginning with prefixsort [rules] file Sort file according to ruleswc file List the number of characters, workds, and lines in filediff file1 file2 List the differences between file1 and file2cat list of files Concatentate files and sent to standard outputmore file Display file one page at a timeless file Display file one page at a time and allow paging backwordshead file Display the first few lines of a filetail file Display the last few lines of file







Bibliography

[1] Welsh, Dalheimer, and Kaufman, Running Linux, 3rd Edition. O’Reily & Associates, 1999.

[2] Ellen Siever, Linux in a Nutshell, 3rd Edition. O’Reilly & Associates, 2000.

[3] M. Sobell, A Practical Guild to Linux. Addison-Wesley, 1997.

[4] Oram, A and Talbott, S. Managing Projects with Make. O’Reilly & Associates, 1991.

[5] Vesperman, Jennifer, Essential CVS. O’Reilly & Associates, 2003.

[6] http://www.linuxdot.org/nlm/ Linux Newbie Guide.

[7] http://sunsite.auc.dk/linux-newbie/ Linux Newbie Administrator Guide.

[8] The AfterStep Homepage at http://www.afterstep.org

Documents

STUDY NOTES GNU/Linux: the command line interfaceharinathrokesh.weebly.com/uploads/4/7/6/1/4761172/fundamentals.pdf · 1.3 Why GNU/Linux? Linux can operate as a web, ﬁle, smb (WinNT),