CS162B: PipesJacob T. Chan

Pipes

These allow output of one process to be the input of another process

One of the oldest and most basic forms of Inter-Process Communications We shall discuss this first before going on to semaphores. Both actually deal with

IPC, but piping will be discussed so that the basics will be tackled

This is useful when two processes run different programs and there is a need for both of them to communicate

This is simple enough to understand, but it looks harder than it seems Process management issues

Inter-Process Communication (IPC)

Exchange of data among multiple threads in one or more processes (according to Wikipedia)

Issues include Read-write to shared data Synchronization Memory sharing Remote Procedure Calls (RPC) aka execution of subroutine in another address space

In other words, it’s how processes communicate with one another (hence, the name)

Processes need to communicate too! Example: ALL processes depend on the mother of all processes known as the SCHEDULER

Pipes in Unix

In bash, what we do is to run a piped process. Example:

ls | wc –l

Running this command will result into the output of the first command being the input of the second command

So, ls will run first. Then whatever its output, it will be the input of the second command wc –l

Invoking Pipe

#include <unistd.h> (This will allow the use of libraries like pipe() )

int pipe(int filedes[2]) Creates data pipe filedes[2] are your file descriptors, with filedes[0] opened for READING

ONLY and filedes[1] opened for WRITING ONLY Note: reading data on filedes[0] will access data written by filedes[1]

Output: 0 if successful -1 if not successful

The variable errno will indicate what type of error that caused the pipe to fail

Example of pipe posted in Moodle (Piping Basic Example)

Piping Features

When two processes are piped, they are unaware of it They continue writing to and reading from standard file descriptors Whatever the output of the first process, the second process will use that

REGARDLESS of conditions (in short, second process of pipe is dependent on first one)

Pipes are byte streams! Process reading from pipes is free to read any size of data blocks Implication: not everything written by the writing process is read by the reading

process

You cannot lseek pipes! Because data is read in FIFO fashion

Piping Features

If the file descriptor to the write end is closed (as a result of EOF), all reads will return 0 or EOF once everything is read

PIPES ARE UNIDIRECTIONAL (one end for reading, one end for writing ONLY)

Each pipe write is atomic Writing to the same pipe will not be mixed if they write at most X bytes at a time

(that’s the function of having a specific file descriptor)

When writing to full pipe (meaning max space for writing is reached), operation will block UNTIL space becomes available on pipe

Reading from empty pipe will be blocked until new data is written to the pipe

Piping Features

Analysis: Producer-Consumer (which will be like your lab) Producer (writing side) will keep on producing the characters until there is nothing

left to write Consumer will then read what the producer produced until everything is read In cases where there is nothing to read, the consumer “sleeps” (or gets BLOCKED)

until there is something to read In cases where pipe is full, producer will stop producing in the pipe!

Will wait until consumer consumes something. Only then will producer will write output to the pipe

Pipe Sample Code Snippet (more examples at Moodle)

int filedes[2];

if(pipe(filedes) == -1)

exit(1);

switch(fork()) { case 0:

if(close(filedes[1]) == -1) errExit(“close”);

break; default:

if(close(filedes[0]) == -1) errExit(“close”)

break;

}

Piping Issues

Rarely do two or more processes read from the same pipe If both processes read at the same time, there will be RACE CONDITIONS (and there

will be no way to guarantee will be reading data over the other)

Any two processes (not just parents and children) that are “related” to one another should communicate!

Running the previous command ls | wc –l should work because they communicate via shell (which is their parent that created the pipe)

If the processes being piped belong to different parents, then the pipe will not work (in most cases, since they do not share similar data)

Piping Issues

Implementing parent-child communication is simple enough, but connecting two programs reading from stdin and writing to stdout might be trickier

This is an issue of shared memory, semaphores, socketing (which will be discussed in the coming weeks)

Redirection of stdout to a pipe and stdin being pointed to the same pipe (which is technically socketing)

In piping, we can use the function dup2()

Pipes: int dup2(int oldfd, int newfd)

This duplicates of file descriptor oldfd and assigns it the new number newfd

If newfd corresponds to open descriptor at time of calling, the file descriptor is CLOSED first (to ensure that no blocking will occur in this time)

This allows read and write descriptors to get their corresponding descriptor numbers

Returns the new file descriptor number; otherwise returns -1 on error

Example posted in Moodle

Pipes: FILE* popen(char* command, char* type) A simpler way to creating shell-like piping

Pipes are usually used to read from output or write to input in the context of shell commands

This command creates a pipe and forks child process that execs a shell

This creates child process to execute shell command (which is first parameter)

Type is a string that indicates whether the process will write to (noted by “w”) or read from (noted by “r”) the pipe

For using popen, instead of fclose, use pclose(FILE *stream) instead to close the pipe itself

Example posted in Moodle

Pipes: FILE* popen(char* command, char* type)

Pipes

There are many more techniques to piping, and what was shown were just one of them

We only handled the basic ones.

If you want more examples on piping, I posted some on Moodle for your reference

Or, you can consult your man pages in your terminal.

Lab 9: pcpipe.c

Create a program pcpipe.c that run as follows: Create a pipe in your program Fork a child (this will be the producer named “prod”) Fork another child in the PARENT part (this will be the consumer named “cons”) Make output of prod the same as the read end of pipe using dup2 Make input of cons the same as write end of pipe using dup2 In prod, execve() the producer program, including arguments In cons, execve() the consumer program, including arguments

Note: popen() is NOT allowed. You have execve()

Lab 9: pcpipe.c

The program runs the same way as executing bash commands. For example:

./pcpipe producerprog prodargs … -consumer consumerprog consumerargs…

should produce the same command as running its bash version:

producerprog prodargs … | consumerprog consumerargs …

Example:

./pcpipe ls –l –consumer grep “2013-2-2”

This program runs similarly as doing the following:

ls –l | grep “2013-2-2”

Lab 9: pcpipe.c

Other sample input you can use:

./pcpipe cat /etc/passwd -consumer cut -d\: -f1,3,6

./pcpipe ls -l /home -consumer grep "$LOGNAME“

This should produce the same expected results as its bash version

Lab 9: pcpipe.c

Legal stuff Don’t forget Certificate of Authorship (with COMPLETE details), as well as your

pcpipe.c program. Filename: CS162B_Lab9_<Section>_<Surname>_<ID Number>.tar Deadline: Next week

Tuesday for Section A Thursday for Section B