Introduction to UNIX / Linux - 5

Summer 2015SILICON VALLEY UNIVERSITY

CONFIDENTIAL 1

Introduction to UNIX / Linux - 5

Dr. Jerry Shiao, Silicon Valley University

SILICON VALLEY UNIVERSITY CONFIDENTIAL 2Summer 2015

Introduction UNIX/Linux Course

Section 5 Basic File Processing

UNIX Commands Viewing Files Copying, Moving and Removing Files Appending to Files Comparing Files Removing Repeated Lines Print Files and Controlling Print Jobs

SILICON VALLEY UNIVERSITY CONFIDENTIAL 3

Copyright @2005 Pearson Addison-Wesley.

Introduction UNIX/Linux Course

Basic File Processing Viewing Contents of Text Files

Viewing Permissions View File Need Read Permission View (Searching) Directory Need Execute

Permission Portion of File contents:

Complete Contents. cat: Display text in continuous stream.

Initial / Middle / Last. head, tail: Display initial or tail portion of a file.

One Screen or One Page. more, less: Display text in pages.



Introduction UNIX/Linux Course Basic File Processing Viewing Content of Texts Files cat [ options ] [ file-list ]

Concatenate / Display files in “file-list”. - e: Display $ at end of each line. - n: Put line numbers with the line. - t: Display tabs as “Î” and formfeeds as “^L”. - v: Display nonprintable characters, except for lines from options “-n”,

“-t”.

$ cat -e text.txt Hello Class,$ Tab1 Tab2 Tab3$ $ cat -n text.txt 1 Hello Class, 2 Tab1 Tab2 Tab3 $ cat -t text.txt Hello Class, ÎTab1ÎTab2ÎTab3 ÎÎ



Introduction UNIX/Linux Course Basic File Processing Viewing Contents of Text Files $ cat ~/class/text.txt ~/class/text2.txt

Displays contents of text.txt and text2.txt in ~/class, right after each other without pausing.

$ nl text.txt Displays the lines, similar to “cat - n”.

$ cat Take input from stdin (keyboard) output to stdout (display). Take input from stdin (keyboard) output to stdout(display). $ pr text.txt 2012-10-08 23:08 text.txt Page 1 Hello Class, Tab1 Tab2 Tab3 Line3 . . . 2012-10-08 23:08 text.txt Page 2 Line67 . . .



Introduction UNIX/Linux Course Basic File Processing Viewing Files By Page more [ options ] [ file-list ]

Contents of files in “file-list” displayed page by page. + / str : Start two lines before line with “str”. $ more +/Line20 text.txt ...skipping Line18 Line19 Line20 Line21

less [ options ] [ file-list ] Contents of files in “file-list” displayed page by page. + cmd : “cmd” executed when file is viewed. : Move up. : Move down. $ less +dLine20 text.txt Line20 Line21



Introduction UNIX/Linux Course Basic File Processing Viewing Head or Tail of File

Identify data in file by special header used by applications. Inspect updated information at tail of file ( i.e. Log or Error File ).

head [ option ] [ file-list ] Display the first 10 lines (default) of files in “file-list”. - N : Display first “N” lines. $ head text.txt

Hello Class, Tab1 Tab2 Tab3 Line3 . . .

$ $ head -2 text.txt

Hello Class, Tab1 Tab2 Tab3

$

Display 10 Lines (Default)

Display 2 Lines



Introduction UNIX/Linux Course Basic File Processing Viewing Head or Tail of File tail [ option ] [ file-list ]

Display the last 10 lines (default) of files in “file-list”. - N : Display last “N” lines. - f : Wait and display lines as they are appended to the “file-list”.

<ctrl> <c> terminates wait. $ tail text.txt

. . . Line68 Line69 Line70

$ tail -2 text.txt Line69 Line70

$ tail - 2r text.txt Line70 Line69

Display Last 10 Lines (Default)

Display Last 2 Lines

Display Last 2 Lines In Reverse Order



Introduction UNIX/Linux Course Basic File Processing Viewing Head or Tail of File tail [ option ] [ file-list ]

$ tail –f text.txt Line62 Line63 Line64 Line65 Line66 Line67 Line68 Line69 Line70

NewLine1NewLine2NewLine3^C

$ cat >> text.txtNewLine1NewLine2NewLine3^C

XtermXterm



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files cp [ options ] file1 file2

Copy “file1” to “file2”. If “file2” is a directory, “file1” is copied to the directory.

File1 : Source file. Must exist and must have READ permission.

File2 : Destination file. If exist, must have WRITE permission. Directory must have WRITE and EXECUTE permission.

- f : Force copy, if “file2” does not have WRITE permission. Directory must have WRITE and EXECUTE permission.

- i : Prompt before writing. - p : Preserve “file1” attributes (user:group:other and mod time). - r : Recursively copy files and subdirectories (duplicate a directory for backup).



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files $ ls –lt file1 file2 -r-xr--r-- 1 sau users 0 2012-10-09 18:02 file2 -rw-r--r-- 1 sau users 6 2012-10-09 17:59 file1

$ cp file1 file2 cp: cannot create regular file `file2': Permission denied

$ cp -f file1 file2

$ ls –lt file1 file2 -rw-r--r-- 1 sau users 6 2012-10-09 18:03 file2 -rw-r--r-- 1 sau users 6 2012-10-09 17:59 file1

$ cp -i file1 file2 cp: overwrite `file2'? Yes

$ cp –r file1_dir file2_dir

$

The “-f” or “force” option. If the existing destination file cannot be opened, remove it and try again.

The “-r” or “recursive” option. The destination must be a directory. If source is a directory, copy the contents of the source directory and its subdirectories and contents into the destination directory.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files mv [ options ] file1 file2 mv [ options ] file-list directory

File1 : Source file. Must exist, but does not need any permissions. Directory must have WRITE and EXECUTE permission.

File2 : Destination file. If exist, must have WRITE permission. Directory must have WRITE and EXECUTE permission.

- f : Force copy, if “file2” does not have WRITE permission. Directory must have WRITE and EXECUTE permission.

- i : Prompt before writing. - u : Move only with “file1” is newer than “file2” or when “file2” does not exist.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files $ ls -lt drwxr-xr-x 2 sau users 4096 2012-10-10 12:12 file_dir -rw-r--r-- 1 sau users 39 2012-10-10 12:08 file2 lrwxrwxrwx 1 sau users 5 2012-10-09 17:59 file1_link -> file1 $ ls -lt file_dir/ -rw-r--r-- 1 sau users 74 2012-10-10 12:12 file2 $ mv –u file2 file_dir/ $ ls -lt drwxr-xr-x 2 sau users 4096 2012-10-10 12:12 file_dir -rw-r--r-- 1 sau users 39 2012-10-10 12:08 file2 lrwxrwxrwx 1 sau users 5 2012-10-09 17:59 file1_link -> file1 $ ls -lt file_dir/ -rw-r--r-- 1 sau users 74 2012-10-10 12:12 file2

. . . $ ls -lt dr-xr-xr-x 2 sau users 4096 2012-10-10 13:46 file_dir lrwxrwxrwx 1 sau users 5 2012-10-09 17:59 file1_link -> file1 $ mv file1_link file_dir/ mv: cannot move `file1_link' to `file_dir/file1_link': Permission denied

File older. Do not move.If overwrite an existing file, compare the modification times of the source and target files.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files rm [ options ] file-list

file-list : List of files to be removed from disk. Must exist, but does not need READ and WRITE permissions. Directory must have WRITE and EXECUTE permission.

- f : Force copy, if “file” does not have WRITE permission. Directory must have WRITE and EXECUTE permission.

- i : Prompt before writing. - r : Recursively remove the files in the directory.

shred [ options ] file-list file-list : List of files to be overwritten.

Must exist and have WRITE permission. - u : Remove file after overwriting.

Directory must have WRITE and EXECUTE permission. - f : Force overwrite, if “file-list” files does not have WRITE permission.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files $ ls -lt dr-xr-xr-x 2 sau users 4096 2012-10-10 17:04 file_dir -r--r--r-- 1 sau users 5 2012-10-10 17:04 file2 $ rm file2 file_dir/file3 rm: remove write-protected regular file `file2'? yes rm: cannot remove `file_dir/file3': Permission denied $ ls -lt dr-xr-xr-x 3 sau users 4096 2012-10-10 17:15 file_dir $ ls -lt file_dir drwxr-xr-x 2 sau users 4096 2012-10-10 17:18 file_dir2 -r--r--r-- 1 sau users 5 2012-10-10 17:04 file3 $ rm -rf file_dir rm: cannot remove directory `file_dir/file_dir2': Permission denied rm: cannot remove `file_dir/file3': Permission denied $ chmod 755 file_dir $ rm -rf file_dir $

Current directory file2 without WRITE permission will be prompt. Directory in path without WRITE has access permission.

Recursive Force requires WRITE/EXECUTE in all directories.Files does not need WRITE permission with Force Option.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files $ ls –lt drwxr-xr-x 2 sau users 4096 2012-10-10 16:19 file_dir -r--r--r-- 1 sau users 36 2012-10-10 15:11 file2 $ ls -lt file_dir -r--r--r-- 1 sau users 18 2012-10-10 16:19 file2 $ shred -u file_dir/file2 shred: file_dir/file2: failed to open for writing: Permission denied $ shred -uf file_dir/file2 $ ls -lt file_dir total 0 $ shred file2 shred: file2: failed to open for writing: Permission denied $ shred -f file2 $ ls -lt --w------- 1 sau users 4096 2012-10-10 16:24 file2 drwxr-xr-x 2 sau users 4096 2012-10-10 16:20 file_dir

Shredded file overwritten default 25 times to prevent data recover.



Introduction UNIX/Linux Course Basic File Processing Copying, Moving and Removing Files File Sizes

wc [ options ] file-list Display sizes of the files in “file-list”

- c : Display only the number of characters. - l : Display only the number of lines. - w : Display only the number of words.



Introduction UNIX/Linux Course Basic File Processing File Sizes wc [ options ] file-list $ wc file1 11 30 151 file1

$ wc -c file1 151 file1 $ wc -l file1 11 file1 $ wc -w file1 30 file1 $

Line Word Byte FileCount Count Count Name



Introduction UNIX/Linux Course Basic File Processing Appending to Files cat [ file-list ] >> destination-file Append the files in “file-list” to end of “destination-file”.

$ cat file1 File1 line1. $ cat file2 File2 line1. $ cat file3 File3 line1. $ cat file1 file2 file3 >> fileall $ cat fileall File1 line1. File2 line1. File3 line1. $ cat >> file_keyboard Typing from keyboard. More from keyboard. < Ctrl > < D > $ cat file_keyboard Typing from keyboard. More from keyboard.



Introduction UNIX/Linux Course Basic File Processing Combining Files cat [ file-list ] > destination-file Combine the files in “file-list” and put them in “destination-file”.

$ wc file? 2 4 26 file1 1 2 13 file2 1 2 13 file3 4 8 52 total $ cat file1 file2 file3 > fileall $ wc fileall 4 8 52 fileall $ cat file1 > file1_repeat $ cat file1 >> file1_repeat $ cat file1 >> file1_repeat $ wc file1_repeat 6 12 78 file1_repeat $ cat > file_keyboard Enter from keyboard. Enter again from keyboard. $ cat file_keyboard Enter from keyboard. Enter again from keyboard.

Initially overwrite or create destination-file. Then destination-file appended with file-list.

Without optional input file, cat command can be used to put keyboard input (stdin) into destination file.



Introduction UNIX/Linux Course Basic File Processing Comparing Files diff [ options ] [ file1 ] [ file2 ]

Compare “file1” with “file2” line-by-line and display differences between them as a series of commands that can be used to convert “file1” to “file2”.

Generates a (add), c (change), d (delete) commands. - b : Ignore trailing white spaces. - e : Generate script for “ed” editor to change “file1” to “file2”. - h : Fast comparision.

SILICON VALLEY UNIVERSITY CONFIDENTIAL 22Spring 2014

Introduction UNIX/Linux Course Basic File Processing

patch [options] [original [patchfile]] Takes a patchfile containing a difference listing produced by the diff command

and applies those differences to original file to produce the patched version. Options:

Type of diff listing (-c = context, -e = ed editor, -n = normal, -u = unified). -p [num]: How much of preceding pathname to strip.

Apply patches to current Linux code. Patch file created from “diff” command used in “patch” utility.

diff –Naur <original> <modified> >> patchfile.patch -N: Mismatched files treated as empty. -a: Treat all files as text and compare line-by-line. -u: Use unified output format. -r: Recursively compare any subdirectories.

patch –p1 < patchfile.patch -p1: Strip off first level in path designator in patch file.patchfile.patch:

diff –Nru gcc-3.4.4.ori/gcc/config/arm/arm.c gcc-3.4.4.patched/gcc/config/arm/arm.c

--- gcc-3.4.4.ori/gcc/config/arm/arm.c 2005-02-01 16:07:02.000000000 +0100

. . .

No option uses the patchfile to determine the type of diff listing.



Introduction UNIX/Linux Course Basic File Processing Comparing Files sau@buildbed-vm: cat -n < class > file1

1 File1 line1. 2 File1 line2. 3 File2 line1. 4 File2 line2.

sau@buildbed-vm: cat -n < class2 > file2 1 File2 line1. Change this line. 2 File2 line1.5 Insert this line. 3 File2 line2. 4 File2 line3. 5 File2 line4.

sau@buildbed-vm: diff –e file1 file2 > diff_1_2 sau@buildbed-vm: cat >> diff_1_2

wq

sau@buildbed-vm: sau@buildbed-vm: ed file1 < diff_1_2

80137

$ diff file1 file21,3c1,2< File1 line1.< File1 line2.< File2 line1.---> File2 line1. Change this line.> File2 line1.5 Insert this line.4a4,5> File2 line3.> File2 line4.

$ cat file1File2 line1. Change this line.File2 line1.5 Insert this line.File2 line2.File2 line3.File2 line4.



Introduction UNIX/Linux Course Basic File Processing Removing Repeated Lines uniq [ options ] [ +N ] [ input-file ] [output-file ] Remove consecutive repetitious lines from the sorted “input-file” and

write non-repetitious lines to “output-file”. + N : Number of characters skipped before checking for uniqueness. - c : Precede each output line by the number of times it occurs. - d : Display the repeated lines. - u : Display the lines that are not repeated.

sort [ options ] file-list Write concatenation of files in file-list to stdout. - o : Write result to file instead of stdout. - k : Start key at column. - n : Compare according to string numerical value.



Introduction UNIX/Linux Course Basic File Processing Removing Repeated Lines (Unsorted File) $ uniq file1 This is Line1. This is Line2. This is Line3. This is Line2. This is Line4. $ uniq -c file1 1 This is Line1. 3 This is Line2. 1 This is Line3. 2 This is Line2. 1 This is Line4. $ uniq -u file1 This is Line1. This is Line3. This is Line4. $ uniq -d file1 file_out $ cat file_out This is Line2. This is Line2.

$ cat file1This is Line1.This is Line2.This is Line2.This is Line2.This is Line3.This is Line2.This is Line2.This is Line4.

Output non-repeating lines.

There are two groups of “This is Line2”.



Introduction UNIX/Linux Course Basic File Processing Removing Repeated Lines (Sorted File) $ sort file1 > file1_sort $ uniq file1_sort This is Line1. This is Line2. This is Line3. This is Line4 $ uniq -c file1_sort 1 This is Line1. 5 This is Line2. 1 This is Line3. 1 This is Line4. $ uniq -u file1_sort This is Line1. This is Line3. This is Line4. $ uniq -d file1_sort file_out $ cat file_out This is Line2.

$ cat file1This is Line1.This is Line2.This is Line2.This is Line2.This is Line3.This is Line2.This is Line2.This is Line4.

Output non-repeating lines.

There is one group of “This is Line2”.

$ cat file1_sortThis is Line1.This is Line2.This is Line2.This is Line2.This is Line2.This is Line2.This is Line3.This is Line4.



Introduction UNIX/Linux Course Basic File Processing Printing Files

Output Device is Printer, instead of Console. Printer shared, First Come First Serve queuing.

Print Requests Placed in Print Queue. Each Request Assigned Job ID.

Printer daemon, lpd, handles print queue.

SILICON VALLEY UNIVERSITY CONFIDENTIAL 28www.linuxjournal.com

Introduction UNIX/Linux Course Basic File Processing CUPS ( Common UNIX Printing System )

Replacement for lpd Printing System, cupsd printer daemon. Printer drivers for parallel port, USB, and serial connections. $ ls -l /usr/bin/lpr

lrwxrwxrwx. 1 root root 23 Apr 5 2013 /usr/bin/lpr -> /etc/alternatives/print $ ls -l /etc/alternatives/print

lrwxrwxrwx. 1 root root 17 Apr 5 2013 /etc/alternatives/print -> /usr/bin/lpr.cups

Configuration file, /etc/cups/cupsd.conf. Common printing interface across local network. Provides interactivity with SMB (Server Message Block) and

Windows printers. Dynamic Printer detection and grouping.

Broadcast Printers available on local computer. Class is group of printers that appears as a single printer.

GPL Licensed ( Easy Software Products ).

SILICON VALLEY UNIVERSITY CONFIDENTIAL 29www.linuxjournal.com

Introduction UNIX/Linux Course Basic File Processing Samba UNIX Print and File Servers provides Interactivity with

Windows Clients using SMB/CIFS (Common Internet File System) Protocol. Developed from IBM NetBIOS (LAN Protocol) SMB (File

Sharing Protocol) CIFS CIFS Services:

File and Print Services ( smbd daemon ) Authentication and Authorization (smbd daemon ) Name Resolution ( nmb daemon ) Service Announcement ( Browsing) (nmb daemon )

Open Source Implementation (http://samba.org/) Ported to non-UNIX Operating Systems: VMS, Netware.



Introduction UNIX/Linux Course Basic File Processing Printing Files lp [ options ] file-list Submit a Print Request to Print the Files in “file-list”

- P page-list : Print pages specified in ‘page-list’. - d ptr : Submit the print request for the ‘ptr’ printer. - m : Send mail after printing is complete. - n N : Print ‘N’ copies. - t title : Print ‘title’ on a banner page. - w : Write to user’s terminal after printing is complete.

lpr [ options ] file-list - # N : Print ‘N’ copies. - P ptr : Submit the print request for the ‘ptr’ printer. - T title : Print ‘title’ on a banner page. - m : Send mail after printing is complete. - p : Format the output by using the pr command.



Introduction UNIX/Linux Course Basic File Processing Printing Files lpstat [ options ] Display the Status of Print Jobs on a Printer.

- d : Display the status of print jobs on the default printer. - o “job-ID-list” : Display status of the print jobs. - p printer-list : Display the status of print jobs on the printer specified in ‘printer-list’. - u : Display the status of print jobs from the users in ‘user-list’.

lpq [ options ] Display the Status of Print Jobs on a Printer.

- P printer-list : Display the status of print jobs on the printers specified in “printer-list”. - l : Display status of print jobs in a long format.









Introduction UNIX/Linux Course Basic File Processing Canceling Print Jobs

cancel [ options ] [ printers ] Cancel Print Requests – Remove Print Jobs From Print Queue.

- jobID-list : Cancel print jobs specified in “jobID-list”. - u login : Cancel all jobs issued by user “login”.

lprm [ options ] [ jobID-list ] [ user(s) ] Cancel Print Requests – Remove Print Jobs From Print Queue. Remove from “user(s)”, if specified.

- P ptr : Specify the print queue for the “ptr” printer.



Introduction UNIX/Linux Course Basic File Processing Canceling Print Jobs

Documents

Introduction to UNIX / Linux - 5