Computing Surveys, Vol. 5, No.

4,

December 1973

Concurrent Programming Concepts

Per Brinch Hansen

Information

Science,

California Institute of Technology, Pasadena, California 91109

This paper describes the evolution of language features for multiprogrammingfrom event queues and semaphores to critical regions and monitors. It suggeststhat the choice of language concepts shouldbe guided by two simple principles :First, it should be possible to understand a concurrent program in time-independent terms by an effort proportional to its size; secondly, it should bepossible to state assumptions about invariant relationships among programcomponents and have these assumptions checked automatically. The centralproblems of multiprogramming are illustrated by annotated algorithms writtenin a well-structuredprogramming language.

Key words and phrases: structured multiprogramming, programming languages,operating systems, programming errors, resource protection, compile-timechecking, correctness proofs, sequential and concurrent processes, synchronizingevents, semaphores, shared data, mutual exclusion, critical regions, monitors.

CR categories: 4.22, 4.31, 4.32, 5.24

Somewhere there are decisions made that arenot rational in any sense, that are subject tonothing more than the personal bias of thedecision maker. Logical methods, at best, re-arrange the way in which personal bias is tobe introduced into a problem. Of course, this"at best" is rather important. Present intuitivemethods unhappily introduce personal bias insuch a way that it makes problems impossibleto solve correctly. Our purpose must be to re-pattern the bias, so that it no longer interferesin this destructive way with the process ofdesign, and no longer inhibits clarity of form.Chistopher Alexander, Notes on the Synthesis

ofForm

1. INTRODUCTIONThis paper describes an attitude towardmultiprogramming—the programming tech-niques used to control concurrent activitiesby computers.* It tries to identify abstractproperties of programming languages thatfacilitate the design of large,reliable multi-* In this paper, the term multiprogramming is usedconsistently to denote all programming techniquesused to control concurrent processes on single-processor aswell as multiprocessorsystems.

programming systems. The techniques usedto implement these language concepts, whichapply both to single-processor and multi-processor systems, are described elsewhere.[1]

When a new programming technique, suchas multiprogramming, is invented, program-mers will initially use it in a completelyun-restricted manner to discover its potential.At a later stage, when experience with itspractical limitations has been gained, de-signers begin to recognize the benefits of amore restrictive language notation whichclarifies their own understanding of pro-grams and enables compilers to detect seri-ous errors.

This survey describes the evolution of lan-guage features for multiprogramming duringthe past decade. I view this developmentmainly as a gradual shift from concepts thathave a strong resemblance to assembly lan-guage features toward a notation that en-courages hierarchical structuring of pro-grams. Several useful proposals are notmentioned here, and those that are included

Concurrent Programming Concepts224 Per Brinch Hansen

225

CONTENTS

1. Introduction 2232. SequentialProcesses 2243. Unrestricted Concurrency2264. Structured Concurrency2285. Disjoint Processes 2306. Time-DependentErrors 2317. TimingSignals2338. Critical Regions 2369.

Conditional

Critical Regions 24110. Conclusion 244Acknowledgements244Bibliography244

Copyright © 1973, Association for ComputingMachinery, Inc. General permission to republish,but not for profit, all or part of this material isgranted,provided that ACM's copyright notice isgiven and that reference is made to this publica-tion, to its date of issue, and to the fact that re-printing privileges were grantedby permission ofthe Association for Computing Machinery.

are only meant to illustrate a general atti-tude toward the subject.

The central problems of multiprogram-ming are illustrated by algorithms writtenin a high-level language.I have included in-formal assertions in the algorithms and out-lined arguments of their correctness to showthat multiprogramming concepts can be de-fined as concisely as sequential program-ming concepts. In many ways, this surveycan be regarded as an expanded version ofan earlier paper [2]. We will begin with abrief characterization of sequential pro-grams.

2. SEQUENTIAL PROCESSESAlthough most programmers have an intui-tive understanding of what a sequentialprocess is, it may be useful to illustrate thisconcept by an example.Algorithm 1 definesa sequential process that inputs a fixednumber of records RI, R2, . . . ,Rn fromone sequential file, updates these records,and outputs them on another sequential file.

The language notation is borrowed fromPascal. [3] The files are declared as twovariables, reader and printer, each consist-ing of a sequence of records of some type T(declaredelsewhere). A record of type T canbe appended to or removed from a sequenceby means of two standard procedures, putand get. The current record is kept in a vari-able, this, of type T. The variable i countsthe number of records processed; it can as-sume integer values from 0 to n.

Algorithm 1 Sequential file updating

var reader, printer: sequence of T;this: T; i: 0. . n;

begin(1) i:=0;(2) while i<n do(3) get (this, reader) ;(4) update (this) ;(5) put (this, printer) ;(6) i: = i + l;(7) end(8) end

The program defines a sequential processin terms of data (represented by constants

and variables) and operations on data (rep-resented by statements). During the execu-tion of the program, the operations are car-ried out strictly one at a time, and eachoperation is completed within a finite time.The result of an operation can be defined byassertions about the relationships of databefore and after its execution.In Algorithm 1, we can make the followingassertions about the state of the variablesbefore and after the execution of the inputstatement in line 3:

comment 3 i out of n records processed(where 0 i < n) ;get (this, reader) ;

comment 4 i out of n records processed(where 0 i < n), and this = inputrecord Ri + 1;

Sequential programs have two vital prop-erties :

1) The effect of a sequential program isindependent of its speed of execution. Allthat matters is that operations are carriedout one at a time with positive speed, andthat certain relationships hold before andafter their execution. The time-independentbehavior of sequential programsenables theuser to ignore details over which he has nocontrol (such as the schedulingpolicy of theoperating system and the precise timing ofoperations carried out by processors and pe-ripherals).

2) A sequentialprogram delivers the sameresult each time it is executed with a givenset of input data. The reproducible behaviorof sequential programs under any circum-stances is particularly important for pro-gram validation. It enables one to isolateand correct program errors by systematictesting.

These properties of sequential programsare so well known that we tend to take themfor granted. But as we shall see, these prop-erties can only be maintained for multipro-gramming systems by a careful selection oflanguage concepts.

Algorithm 1 is a well-structured programthat can be analyzed in a step-wise manneras a set of nested operations. At the mostdetailed level, the programcan be describedin terms of assertions about each statement.

The following assertions will hold beforethe execution of lines 1-8, respectively.

comment 1 0 out of n records processed(where 0 n) ;

comment 2 i out of n records processed(where 0 i n) ;

comment 3 i out of n records processed(where 0 i < n) ;

comment 4 i out of n records processed(where 0 i < n), and this = inputrecord Ri + 1 ;

comment 5 i out of n records processed(where 0 i < n) , and this = updated

record Ri + 1 ;comment 6 i + 1 out of n records proc-

essed (where0 i < n) ;comment 7 i out of n records processed

(where 0< i n) ;comment 8 n out of n records processed

(where 0 s^ n) ;Once the individual- statements are under-

stood, one can define the effect of a sequenceof statements by a single pair of assertions.As an example, the body of the while state-ment in Algorithm 1 can be defined as asingle operation by means of the followingassertions:

comment 3 i out of n records processed(where 0 i < n) ;process next record and increment i byone;

comment 7 i out of n records processed(where 0< i n) ;

The next step of simplification is to re-duce the while statement to a single opera-tion:

comment 2 i out of n records processed(where 0 i n) ;process the remaining n-i records;

comment 8 n out of n records processed(where 0 n) ;

Finally, the whole programcan be definedas a single operation:

comment 1 0 out of n records processed(where 0 n) ;process n records;

comment 8 n out of n records processed(where0 n) ;

At this point, only what theprogram does asa whole, is relevant, but the details of howit is done are irrelevant. The possibility ofreplacing complex descriptions by partial

Computing Surveys, Vol. 5, No.

4,

December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

226 Per Brinch Hansen Concurrent Programming Concepts227

I-

J

Fig. 1 Flowchartof Algorithm 1.

descriptions, called abstractions, is essentialto enable us to understand large programs.

Figure 1 is a flowchart of Algorithm 1. Itshows that the program consists of nestedstatements, each with a single starting pointand a single completion point. Algorithm 1can be analyzed by associating an assertionwith each statement, and gradually replac-ing these assertions by fewer and fewer as-sertions. It is most important that aprogramwith a hierarchical structure can be builtand understood in detail by an intellectualeffort proportional to its size, and that cer-

tain functional aspects of it can be describedin even more economical terms. This is thewhole purpose of structured programming!

It is also essential to use hierarchicalstructuring of programs that control concur-rent processes, and choose language con-structs for multiprogramming that can beunderstood in time-independent abstractterms; otherwise, we shall find ourselvesunable to apply systematic methods of pro-gram analysis and verification.

3. UNRESTRICTED CONCURRENCY

Processes are called concurrent if their exe-cution overlap in time. More precisely, twoprocesses are concurrent if the first opera-tion of one process starts before the lastoperation of the other process ends. Theprogramming techniques used to controlconcurrent processes are called multipro-gramming.

No assumptions will be made about thespeed at which concurrent processesare exe-cuted (except that it be positive). We willbenefit from this weak assumption in twoways: 1) It will encourage us to try tounderstand multiprogramming systems intime-independent terms; and 2) It will per-mit the underlying implementation to usea dynamic scheduling algorithm to achieveefficient resource sharing.

One of the earliest ideas of multiprogram-ming was to specify the start and comple-tion of processes by two operations whichI will call start and complete (In the litera-ture, they are often called fork and join).The meaning of these operations can best beillustrated by an example:

var task: response;begin

start task do SI ;S2;complete task;S3;

end

Initially, this program will be executed aaa single, sequential process. The statement

start task do SI

will start the execution of statement SI asa new process. As soon as the new processstarts to run, the old process continues toexecute the next statement S2. When state-ment SI has been executed, the new processindicates its completion by means of a vari-able, task, of type response. Following this,the new process ceases to exist. After theexecution of statement S2, the old processwaits until the response variable indicatesthat statement SI has terminated. The pro-gramthen continues to execute statementS3as a single, sequential process. (We will notbe concerned with how the process and re-sponse concepts can be implemented.) Fig-ure 2 shows a flowchart of this program.

Algorithm 2 is a concurrent version of thefile updating program (Algorithm 1) thatuses the start and complete statements. Thepurpose of introducing concurrent processesis to permit input/output operations to pro-ceed simultaneously with updating opera-

Fig. 2 Flowchart of program to illustrate startand complete operations.

tions. Algorithm 2 is deliberately written inan unstructured manner that makes it diffi-cult to understand. A well-structured algo-rithm will be presented later. (I recommendthat thereader not spend too much time try-ing to understand Algorithm 2 at this point.)

Algorithm 2 Concurrent file updating (un-structured version)

var reader, printer: sequence of T;reading, printing: response; next,this, last: T; i: 2 . . n;

beginif n 1 then

get (last, reader) ;if n Js 2 then

update (last) ;start printing do put (last,

printer) ;get (this, reader)

;

i:= 2;while i < n do

start reading do get (next,reader) ;

update (this) ;complete printing;last:— this;start printing do put (last,

printer) ;complete reading;this:= next;

endcomplete printing;last:= this;

endupdate (last) ;put (last, printer) ;

endendThe program inputs and updates the first

record RI. It then starts the output of thefirst record and inputs the second recordR2 at the same time. The program now re-peats the execution of a cycle that, in gen-eral, inputs the next record Ri + 1, updatesthis record Ri, and outputs the last recordRi-1 simultaneously. When all records havebeen input (i = n) , the program completesthe output of the second last record Rn-1.Finally, it updates and outputs the lastrecord Rn.

Computing Surveys, Vol. 5, No.

4,

December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

i:= i + l;

«t

Concurrent Programming Concepts

229

228 Per Brinch Hansen

In spite of this explanation, the details ofAlgorithm 2 are still hard to follow. Thecause of this obscurity becomes clear whenwe examine the flowchart of the body of thewhile statement (Fig. 3). This flowchartcannot be decomposed into a set of nestedoperations. The program is highly confusingbecause a process started outside the whilestatement (output) may be completed eitherinside or outside the while statement (de-pendingon whether or not n > 2) .

It is certainly possible to write a simplerprogram using the start and complete state-ments (just as it is possible to structure pro-grams written in assembly language) . Butthe concepts of a high-level language shouldbe chosen such that they naturally encour-age the programmerto write well-structuredprograms. The start and complete state-ments clearly have the flavor of go to state-ments. They make it very difficult to deter-mine from the program text where concur-rent processes begin and end, and whichvariables they operate on.

4. STRUCTURED CONCURRENCY

A well-structured representation of concur-rent processes in a high-level programminglanguage was suggested by Dijkstra in 1965.[4] The concurrent statement

cobegin Si; S2; . . . ; Sn coendindicates that the statements Si, S2, . . . ,Sncan be executed concurrently; when all ofthem are completed, the following statementin theprogram (not shown here) is executed.

Algorithm 3 defines file updating bymeans of concurrent statements. We willanalyze this program, statement by state-ment, in terms of assertions that hold beforethe execution of lines 1-12.

The initial assertion is the following:comment 1 0 out of n records processed

(where0 n) ;If the input file is non-empty (n 1) , the

program inputs the first record RI:comment 2 0 out of n records processed

(where 1 n), and this = input recordRI;

If the input contains more than one record(n 2), the program inputs the secondrecord R2 and updates the first record RIsimultaneously:

comment 3 0 out of n records processed(where 2

s$

n) , next — input record R2,and this = updated record RI ;

Algorithm 3 Concurrent file updating(structured version)

var reader, printer: sequence of T;next, this, last: T; i: 2 . . n;

begin(1) if n 1 then

get (this, reader) ;(2) if n S3 2 then

cobeginget (next, reader)

;

update (this) ;

(4) while i < n do

update (this) ;

coend(6) i.-= i + l;(7) end(8) put (this,printer) ;

this : = next;(9) end

(10) update (this) ; put (this, printer) ;(11) end(12) end

The program now enters a main loop thatrepeatedly inputs a record Ri + 1, updatesa record Ri, and outputs a record Ri — 1simultaneously. The loop is initialized bysetting a counter i equal to 2. This leads tothe following program state:

comment 4 i — 2 out of n records proc-cessed (where 2 i n) , next = inputrecord Ri, and this = updated recordRi-1;

Within the loop, theprogramgoes throughthe following sequence of states:

comment 5 i — 2 out of n records proc-essed (where 2 < i < n) , this — input

record Ri, and last = updated recordRi- 1;

comment 6i — 1 out ofnrecord processed(where 2 i < n) , next — input recordRi + 1, and this = updated record Ri;

comment 7 i — 2 out of n records proc-essed (where 2< i n) , next — inputrecord Ri, and this — updated recordRi-1;

A comparison of assertions 4 and 7 showsthat assertion 4 is a relation that remainsinvariant after each execution of the loop.When the loop terminates, assertion 4 stillholds and i is equal to n:

comment 8 n — 2 out of n records proc-essed (where 2 n) , next = input rec-ord Rn, and this = updated recordRn - 1;

The program now outputs the second lastrecord Rn — 1 :

comment 9 n — 1 out of n records proc-essed (where 2 n) , and this — inputrecord Rn;

The effect of the innermost if statementment can be summarized as follows:

comment 2 0 out of n records processed(where 1 n) , and this — input recordRI;process n — 1 records;

comment 10n — 1 out of n records proc-essed (where 1 n), and this = inputrecord Rn;

If n = 1 then assertions 2 and 10 are equiv-alent, and if n 2 then assertions 9 and10 are equivalent. This covers the two casesin which the if statement is either skippedor executed.

Finally, the last record Rn is updated andoutput:

comment 11 n out of n records processed(where 1 n) ;

The whole program can be understood asa single operation defined as follows:

comment 1 0 out of n records processed(where 0 n) ;process n records;

comment 12 n out of n records processed(where 0 < n) ;

This concurrent program has precisely thesame characteristics as a well-structured se-quential program. It can be analyzed in

INPUT UPDATE OUTPUT

Fig. 3 Flowchart of the body of the while state-ment in Algorithm 2.

terms of time-independent assertions by aneffort proportional to its size, and assertionsabout simple statements can gradually bereplaced by assertions about structuredstatementsuntil the whole programhas beenreduced to a single statement defined by twoassertions. This abstract description of theprogram evenenables the programmer to ig-nore the concurrent nature of the solution.To see this, it is sufficient to observe thatthe initial and final assertions 0 and 12 forAlgorithm 3 are identical to the assertionsmade at the beginning and end of the se-quential version (Algorithm 1).

In the previous discussion, the concurrentstatement was defined and used informallyto give you an intuitive understanding of itsmeaning. We will now explicitly discuss theassumptions that led to the simplicity ofAlgorithm 3.

Computing Surveys, Vol. 5. No. 4, December 1973Computing Surveys, Vol. 5, No. 4, December 1973

coend(3) i:= 2;

last := this; this := next;(5) cobegin

get (next, reader)

;

put (last, printer) ;

s»

230 Per Brinch Hansen Concurrent Programming Concepts

231

1r

—i II |I II |I I

■

r-

| LI I

-J I

_ll_

Fig. 4 A flowchart of the body of the while state-ment in Algorithm 3.

5. DISJOINT PROCESSESThere is one striking difference between Al-gorithms 2 and 3: When concurrent state-ments are used, the program text directlyshows where a given process begins andends. This simplification is achieved by re-stricting the freedom of scheduling. Thestart and complete statements enable theprogrammerto initiate and terminate proc-esses in any order he pleases. A concurrentstatement defines processes that are startedand completed at the same time. Completegenerality of programming implies completeabsence of structure. As a compromise, onehas to look for programming tools that areintellectually manageable and at the sametime practical for most (but not necessarilyall) applications. The concurrent statementis one such compromise.

Since a concurrent statement has a singlestarting point and a single completion point,it is well-suited to structured programming.Figure 4 shows a flowchart of the body ofthe while statement in Algorithm 3. In con-trast to Fig. 3, the present flowchart can bedecomposed into nested operations. This ex-plains why we were able to reduce assertionsabout simple statements to assertions about

structured statements, and then reduce as-sertions about structured statements to as-sertions about the whole algorithm.

It may seem surprising that the use ofconcurrent statements in Algorithm 3 in noway complicated the formulation of asser-tions. The informal arguments made in fa-vor of its correctness are quite similar to thearguments on Algorithm 1 in Section 2. Itturns out that we were able to apply se-quential methods of program analysis to aconcurrent program simply because theprocesses defined by the concurrent state-ments are completely independent of oneanother. These processes operate on disjointsets of variables:

process variablesget next, readerupdate thisput last, printer

They are called disjoint or non-interactingprocesses. The input, update, and outputprocesses are carried out simultaneouslyonly to utilize the computer more efficiently.But conceptually, these processes could justas well be carried out strictly sequentiallyas defined by Algorithm 1.

Since the processes in Algorithm 3 haveno variables in common, they can be ana-lyzed one at a time as unrelated sequentialprocesses. Consider, for example, the con-current statement inside the loop. It can beanalyzed as three independent operationsdefined by the following assertions:

comment 5a i — 2 out of n records proc-essed (where 2 i < n) ;get (next, reader) ;

comment 6a next = input record Ri + 1(where 2 i < n) ;

comment 5b i — 2 out of n records proc-essed (where2 i < n) , and this = in-put record Ri;update (this) ;

comment 6b this = updated record Ri(where 2 i < n)

;

comment 5c i — 2 out of n records proc-essed (where 2 i < n) , and last =updated record Ri — 1 ;put (last printer) ;

comment 6c i — 1 out of n records proc-essed (where 2 i < n) ;

Here assertions 5 and 6 have been split intothree pairs of assertions, one for each processstatement. Each pair of assertions definesour assumptions about a variable i that re-mains constant during the execution of theconcurrent statement and our assumptionsabout the variables that are private to thecorresponding process and may be changedby it.

Since the processes are disjoint, the con-junction of assertions 6a, 6b, and 6c holdsafter the execution of the concurrent state-ment.

In general, we can define the properties ofa concurrent statement by the followingrule of disjointness: The statement

cobegin SI ; S2 ; .. . Sn coend

defines statements SI, S2, . . . ,Sn that canbe executed concurrently as disjoint proc-esses. The disjointness implies that a varia-ble vi changed by a statement Si cannot bereferenced by another statement Sj (wherej 9^ \). In other words, a variable subjectto change by a process must be strictly pri-vate to that process; but disjoint processescan refer to common variables not changedby any of them.

The general rule that we intuitively fol-lowed in analyzing Algorithm 3 can bestated as follows: Suppose that we knowthe following about statements SI, S2, . . . ,Sn that operate on disjoint variables:

Statement SI will terminate with a re-suit RI if a precondition PI holds beforeits execution.

Statement S2 will terminate with a re-suits R2 if a precondition P2 holds beforeits execution.

Statement Sn will terminate with a re-suit Rn if a precondition Pn holds beforeits execution.Then we can conclude that a concurrent

execution of SI, S2, . . . ,Sn will terminatewith the result RI & R2 ... & Rn if thepreconditionPI &P2 & . . . &Pn holds be-fore its execution. (It should be added thatthe assertions Pi and Ri made about state-

ment Si must only refer to variables thatare accessible to Si according to the rule ofdisjointness.)

6. TIME-DEPENDENT ERRORS

An error in a sequential program can be lo-cated by repeating the execution of the pro-gram several times with the data that re-vealed the error. In each of these experi-ments, the values of selected variables arerecorded to determine whether or not a givenprogram component works. This process ofelimination continues until the error hasbeen located.

When a given program component hasbeen found to behave correctly in one test,we can ignore that component in subsequenttests because it will continue to behave inexactly the same manner each time the pro-gram is executed with the given data. Inother words, our ability to test a large se-quential program in a step-wise manner de-pends fundamentally on the reproduciblebehavior of the program.

A careful programmerwho writes a well-structured concurrent program, such as Al-gorithm 3, and outlines an informal proofof its correctness can still make mistakeswhen he types the final program. And hemay not find all these errors during a proof-reading of the program text. One possiblemistake would be to type the concurrentstatement within the loop of Algorithm 3 asfollows :

cobeginget (next, reader)

;

update (this) ;put (this, printer) ;

coendIn this case, the concurrentstatement will

input the next record Ri + 1 correctly, butwill update and output the current recordRi simultaneously. So the output record willnormally be only partially updated. In amultiprocessor system with a common store,a record occupying, say 256 machine words,may be output with x words updated and256-x words unchanged (where 0 x256). The processing of a single record cantherefore produce 257 different results. If

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

232 Per Brinch Hansen Concurrent Programming Concepts 233

we update a given file of 10,000 records, theprogram can give of the order of 257 10'000

different results.In a multiprogramming system, the execu-

tion of concurrent statements can be inter-leaved and overlapped in arbitrary order.The system uses this freedom to multiplexfast computer resources (processors andstores) among several user computations toachieve short response times at areasonablecost.

The result of the erroneous version ofAlgorithm 3 dependson the relative rates atwhich updating and output of the samerecord take place. The rates will be influ-enced by the presence of other (unrelated)computations, and by the absolute speeds ofperipheral devices and operators interactingwith the computations. It is therefore veryunlikely that the erroneous program willeverdeliver the same result twice for a giveninput file. The error will be particularlyhard to find if the updated file is not in-spected by humans, but just retained forlater processing by other programs.

Such unpredictable program behaviormakes it impossible to locate an error bysystematic testing. It can only be found bystudying the programtext in detail. This canbe very frustrating (if not impossible) whenit consists of thousands of lines and one hasno clues about where to look. // we wish tosucceed in designing large, reliable multipro-gramming systems, we must use program-ming tools that are so well-structured thatmost time- dependent errors can be caughtat compile time.

A closer look at the incorrect version ofAlgorithm 3 reveals a clear violation of therule of disjointness: Within the erroneousconcurrent statement the output process re-fers to a variable, this, which is changed bythe updating process. It is therefore inevita-ble that theresult of the output depends onthe time at which updating takes place.

To make it simple for a compiler to checkthe disjointness it should be possible by scan-ning the program text to recognize concur-rent statements and variables accessed bythem. The compiler must be able to dis-tinguish between variables that can be

changed by a statement and variables thatcan be referenced by a statement but notchanged by it. These two kinds of variablesare called the variable parameters and con-stantparameters of a statement.

When start and complete statements areused, one cannot, in general, recognize con-current statements from the syntax alone.This recognition is, however, trivial whenconcurrent statements areenclosed in brack-ets, cobegin and coend. To make the check-ing of disjointness manageable, it is neces-sary to restrict the use of pointer variablesand procedure parameters far more thanpresent programming languages do.

Although the problem has not yet beenanalyzed completely, it seems certain thatthe necessity of compile-time checking ofdisjointness will have a profound influenceon language design. An example will makethis clear. In sequential programming lan-guages a pointer variable may be bound toother variables of a given type, for example:

var p: pointer to integer;This declaration indicates that variable p

is apointer to any integervariable. The no-tation enables a compiler and its run-timesystem to check that p always points to avariable of a well-defined type (in this case,an integer) . This kind of pointer variable isfar better than one that can point to anarbitrary store location containing data ofunknown type (or even code). But the dec-laration pointer to integer still does notenable a compiler to recognize which integerthe variable p will point to during programexecution. So, unless theprogrammeris will-ing to make all integers private to a singleprocess, this pointer concept is inadequatefor multiprogramming. We need to bind apointer p to a particular set of integers, forexample:var i, j, k: integer; p: pointer to i or j;

Some of the languagerequirements neededto make compile-time checking of disjoint-ness practical are discussed in more detail in

The concurrent statement has not yetbeen implemented and used for practical

programming, but I would expect it to be aconvenient tool for the design of small, dedi-cated multiprogramming systems involvinga predictable number of processes. I havesome reservations about its usefulness inlarger multiprogramming systems (for ex-ample, operating systems) as discussed laterin this paper.

The rule of disjointness is introduced tohelp theprogrammer ; it enables him to stateexplicitly that certain processes should beindependent of one another and to dependon automatic detection of violations of thisassumption. To make multiprogrammingintellectually manageable and reasonablyefficient, disjoint processes should be usedwherever possible. But, as we shall see, allmultiprogramming systems must occasion-ally permit concurrentprocesses to exchangedata in a well-defined manner. The com-ponents of a concurrent statement must, forexample, be able to indicate their termina-tion in a common (anonymous) variable;otherwise, it would be impossible to deter-mine when a concurrent statement is termi-nated as a whole. The cobegin coend nota-tion hides this communication problem fromthe user, but it has to be solved at someother level of programming (in this case bythe code generated by a compiler) . The fol-lowing sections describe language featuresused to control interactions among processes.

7. TIMING SIGNALS

Concurrent processes that access commonvariables are called interacting or communi-cating processes. When processes competefor the use of shared resources, commonvariables are necessary to keep track of therequests for service. And when processescooperateon common tasks, common varia-bles are necessary to enable processes to askone another to carry out subtasks and reporton their results.

We will study systems in which oneProcess produces and sends a sequence ofdata items to another process that receivesand consumes them. It is an obvious con-straint that these data items cannot be re-

ceived faster than they are sent. To satisfythis requirement it is sometimes necessaryto delay further execution of the receivingprocess until the sending process producesanother data item. Synchronization is a gen-eral term for timing constraints of this typeimposed on interactions between concurrentprocesses.

The simplest form of interaction is an ex-change of timing signals between two proc-esses. A well-known example is the use ofinterrupts to signal the completion ofasynchronous peripheral operations to acentral processor. Another kind of timingsignals, called events, was used in earlymultiprogramming systems to synchronizeconcurrent processes. When a process de-cides to wait for an event, the execution ofits next operation is delayed until anotherprocess causes the event. An eventoccurringat a time when no processes "are waiting forone has no effect.

The following program illustrates thetransmission of timing signals from oneprocess to another by means of a variable cof type event. Both processes are assumed tobe cyclical:

var c: event;cobegin

cycle "sender". . . cause event(c)

;

endcycle "receiver". . . await event(c) ; . .end

coendA relationship of this type exists in a real-time system in which one process schedulesconcurrent tasksregularly by sending timingsignals to other processes that carry outthese tasks.

in Section 6 we recognized that simultane-ous operations on the same variable can leadto a large number of different results. Inthat context, the problem was caused by aprogram error. Now we have the sameprob-lem again: Theconcurrentoperations, awaitand cause, both access the same variable c.But we can no longer regard this as a viola-tion of the rule of disjointness since our in-

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

234 Per Brinch Hansen Concurrent Programming Concepts

235

tention is that the processes should exchangedata. So we must relax the rule of disjoint-ness and permit concurrent processes to ac-cess shared variables by means of well-de-fined synchronizing operations. Our nexttask is to determine under which conditionssynchronizing operationsare "well-defined".

To analyze the effect of an interactionbetween the receiver and the sender in theprevious program, we must consider all thepossible ways in which the execution ofawait and cause operations can be inter-leaved and overlapped in time. In a continu-ous time scale there are infinitely manypossibilitiesto consider. A drastic simplifica-tion is clearly needed to reduce this infinityto a finite (small) number of cases. The onlypractical solution is to assume that synchro-nizing operationson a given variable cannotbe executed at the same time. In otherwords, await and cause operationson a givenevent variable can be arbitrarily interleaved(but not overlapped) in time. If a processtries to operate on an event variable whileanother process is operating on it, the systemmust delay the former process until the lat-ter process has completed its operation onthe eventvariable.

If this requirement is satisfied, there areonly two cases to consider in the previousprogram: either an await operation is exe-cuted before a cause operation or after it. Ifthe receiver waits before the sender causesthe next event, an interaction between thetwo processes is defined by the followingsequence of operations:

comment receiver not waiting;await event(c)

;

comment receiver waiting;cause event (c) ;comment receiver not waiting;But, if the sender causes the event before

thereceiver waits for it, the receiver will re-main delayeduntil the next event is caused :

comment receiver not waiting;cause event(c)

;

comment receiver not waiting;await event (c) ;comment receiver waiting;The most important result of this analysis

is the general observation that mutual ex-clusion of all operations on a shared variableenables the programmer to analyze the pos-sible effects of a process interaction in finite,sequential terms. For the special case ofevent variables, we have also discoveredthat the net effect of await and cause opera-tions depends on the order in which theseoperations are carried out. Or, to put it morestrongly: event operations force the pro-grammer to be aware of the relative speedsof the sending and receiving processes.

The programmer does not control theorder in which concurrent statements areexecuted; he is therefore unable to predictthe effect of a process interaction involvingevents. In a real-time system, the program-mer has essentially lost control of processscheduling—he cannot define a process thatwill schedule other processes in a predictablemanner. Event variables are only meaning-ful to use when one can assume that aprocess never is asked to carry out anothertask until it has completed its previous task.Although this assumption may be satisfiedin some applications, it will still complicateprogramming tremendously if one dependson it. The programmermust then be awareof the relative speeds of processes under allcircumstances. If a multiprogramming sys-tem is so large that no single person under-stands its dynamic behavior in detail, theindividual programmer cannotmakereliableestimates of the relative speeds of processesunder all circumstances. In particular, it willbe an intolerable burden to verify that thespeed assumptions are uninfluenced bymodifications or extensions of a large sys-tem.

We must therefore conclude that eventvariables of the previous type are impracti-cal for system design. The effect of an inter-action between two processes must be inde-pendent of the speed at which it is carriedout.

A far more attractive synchronizing tool,the semaphore, was invented by Dijkstra in1965. [4, 5] A semaphore is a variable usedto exchange timing signals amongconcurrentprocesses by means of two operations, waitand signal (originally called P and V). All

operations on a semaphore exclude oneanother in time. Associated with a sema-phore v are two integers defining the num-ber of signals sent and received through vand a queue in which receiving processescan await the sending of further timingsignals by other processes. Initially, thenumber of signals sent and received are zeroand the queue is empty.

Signals cannot be received faster thanthey aresent. This semaphore invariant:

0 received sent

is satisfied by using the following synchro-nization rules:

1) If a wait operation on a semaphore vis executed at a time when received < sentthen received is increased by one and thereceiving process continues; but if received= sent, the receiver is delayed in the queueassociated with v.

2) A signal operation on a semaphore vincreases sent by one; if one or more proc-esses are waiting in the queue associatedwith v, one of these processes is enabled tocontinue its execution and received is in-creased by one.

We assume that all processes waiting toreceive signals eventually will be able tocontinue their execution (provided a suffi-cient number of signals are sent by otherprocesses). The scheduling algorithm usedfor a semaphore queue must not delay anyprocess indefinitely in favor of more urgentprocesses. But, apart from this requirementof fair scheduling, no assumptions are madeabout the specific order in which waitingprocesses are allowed to continue. The weakassumption of finite progress (rather thanabsolute speed) for any process is a recur-rent theme of programming. We have madethis assumption for sequential and disjointprocesses, and now we make it again forinteracting processes to achieve simplicityof program analysis and flexibilityof imple-mentation.

Algorithm 4 defines a transmission oftiming signals from one process to anotherby means of a semaphore v.

Algorithm 4 Exchange of timing signalsby means of a semaphore

varv: semaphore;cobegin

cycle "sender". . . signal (v) ;endcycle "receiver"

wait (v);end

coend

Since wait and signal operations on thesemaphore v exclude each other in time, asignal can be sent either before or after thereceiver decides to wait for it. In the firstcase, we can make the following assertionsabout the sequence in which an interactiontakesplace:

comment receiver not waiting and 0received sent;signal (v)

;

comment receiver not waiting and 0received < sent;wait (v)

;

comment receiver not waiting and 0 <received sent;In the second case, the wait operation

may or may not delaythe receiver (depend-ing on whether received = sent or received< sent). But, in any case, the subsequentsignal operation will ensure that the receivercontinues its execution:

comment receiver not waiting and 0received sent;wait (v)

;

comment receiver waiting and 0 re-ceived — sent, or receiver not waitingand 0 < received sent;signal (v)

;

comment receiver not waiting and 0 <received sent;

The effect of an interaction is independ-ent of the order in which the wait and signaloperations are carried out. The commutativeproperty of semaphore operations enablesthe programmer to ignore the precise mo-ment at which a timing signal is produced.This is certainly the most important contri-bution of semaphores to program clarity.

Other uses of semaphores will be describedlater when we have clarified the funda-mental role of mutual exclusion in multipro-gramming.

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

236 Per Brinch Hansen Concurrent Programming Concepts 2378. CRITICAL REGIONS variable is defined in terms of actual and

implicit variables. An actual variable (suchas v) is declared in the program and repre-sented by a store location during its execu-tion. An implicit variable (such as x or r)refers to a property of the system that is notrepresented in the computer during programexecution.

comment x — r = v;print (v) ; [r:= r + v]comment x — r = 0;v:= 0;

same variable in one of its intermediatestates, the result will normally be incorrect.In the previous example, process P starts tooperate on variable v at a time when x —r = 0 (instead of x — r = v) ; this leads toa final state in whichx — r = v + 1 (insteadof x — r = v).

We will now consider concurrent processesthat exchange data of arbitrary type (andnot just timing signals). As an example wechoose a multiprogramming system in whichjob statistics are collected by a process Pand printed by another process Q. When auser job has been completed in this system,process P increases an integer v by one. Atregular intervals process Q prints the valueof v and resets it to zero. To discover theproblems of data sharing we will begin witha naive "solution":

comment x — r = v;v:= v + 1; [x:= x + 1]comment x — r = v;But in the third case, the "invariant" no

longer holds after the process interaction:In Section 7 we found that the effect of

synchronizing operations on event variablesand semaphores can be predicted only ifthey exclude one another in time. The pres-ent example shows that one must also beable to achieve mutual exclusion of arbi-trary concurrent statements referring to ashared variable of an arbitrary type.

Concurrent statements that refer to thesame shared variable are called criticalregions. The previous program contains twocritical regions:

The statements executed by processes Pand Q can be arbitrarily overlapped andinterleaved in time. We will, however, onlyanalyze the effects of an arbitrary inter-leaving of the concurrent statements thatoperate on the shared variable v. The in-crease of v by process P can occur eitherbefore, after, or in the middle of the print-ing and resetting of v by process Q. So wehave three cases to consider:

comment x — r — v;print (v) ; [ r:= r+ v]commentx — r = 0 ;v:= v + 1; [x:= x + 1]commentx — r = 1;

var v: integer;begin comment x — r = v + 1 ;

Whether or not the invariant is main-tamed depends on the ordering of the con-current statements in time. This is, ofcourse, unacceptable to theprogrammer whohas no control over the scheduling of con-current processes. and. . .print (v) ; v:— 0; . . . The correctness criterion of the possible

en^ interactions between processes P and Q iscoend defined by an invariant (v — x —r) that

en<* relates an actual variable v to two implicitThis program violates the rule of disjoint- variables x and r- To analyze the results ofness since processes P and Q both refer to the three PoSSlble cases > we m"st extend theand change the same variable v. Although ProSram Wlth implicit statements referringthis violation suggests a weakness of the to x and r (even though these statementspresent approach to the problem we will ig- Wlll never be executed). Conceptually, xisnore it for the time being increased by one when v is increased, and r

The state of the system accumulating job ! s increased by the value of v when the latterstatistics can be defined by two integers

1S

Pnnted- Assuming that the desired in-(initiallyequal to zero) "

variant holds before an interaction takesx the number of jobs executed place' we can make the foll °wing assertionsr the number of jobs reported about the first case : _

The variable v should represent the number comment x-r - v;of jobs executed but not yet reported; that v:~ v+ 1; [x:= +1]

Notice that in the first two cases, in whichthe invariant continues to be satisfied, thetwo processes have exclusive access to thevariable v while they areoperating on it:

print (v)

Critical regions referring to the same vari-able exclude one another in time. They can,however, be arbitrarily interleaved in time.We make three assumptions about criticalregions that operate on the same sharedvariable:

P h: = v + 1; Q I print (v);

Q print (v);

Mutual exclusion: At most, one processat a time can be inside a critical region.Termination: A process will always com-plete a critical region within a finite time.Fair scheduling: A process can alwaysenter a critical region within a finite time.Dekker has shown that mutual exclusion

The conflict in the third case is caused bymixing the sequence of operations of processQ on variable v with an operation of processP on the samevariable:

of arbitrary statements can be implementedby means of load and store operations only(provided that load and store operations ona given variable exclude one another intime). His solution is far too complicatedand inefficient to be of practical value [4],but it does illustrate the subtlety of the mu-tual exclusion problem.

The inadequacy of the load-and-store ap-proach to mutual exclusion inspired Dijk-stra and Scholten to invent semaphores.Algorithm 5 defines an implementation ofcritical regions by means of a semaphoremutex.

is, the relationship v = x - r should remain comment x - r = v;invariant after each sequence of operations Print > \-r:= r+ v]on the shared variable v. comment x — r = 0 ;

Theexample illustrates two general char- v:~ °;acteristics of multiprogramming systems: commentx — r = v;

1) The correctness criterion for concur- The implicit statements are enclosed in Process Q performs a sequence of opera-tions on the shared variable v. It is assumedthat the invariant holds before and afterthis sequence of operations. But while theoperations are carried out, the variable vmay be in various intermediate states inwhich the invariant is not satisfied. If otherconcurrent statements are carriedout on the

rent operations on a shared variable is de- square brackets to distinguish them fromfined by an invariant—& relationship that actual statements. In this case, the in-must be true after initialization of the vari- variant still holds after the process inter-able and continue to hold before and after action.subsequent operations on the variable. In the second case, the invariant is also2) The invariant property of a shared maintained by the interaction :

Computing Surveys, Vol. 5, No. 4, December 1973Computing Surveys, Vol. 5, No.

4,

December 1973

v:= 0;

v:= 0; ing and resetting of v by process Q. So

\

cobegin have three cases to consider:cycle "P"

V= V + 1- v:= v + 1; print(v); prinl(v);__j" ' '"' print{v); v:=0; v:= v+ 1;cTcW v:=0; ,: - +ij -= o; v:= v + 1

v:= 0

v: = 0;

v: =0; | P \v: = v+ 1;

Q print (v);

P v: = v + 1;

Q v: =Q;

*

238 Per Brinch Hansen Concurrent Programming Concepts

239

L

Algorithm 5 Critical regions implementedby means of a semaphore

var v: integer; mutex: semaphored);begin

cobegincycle "P"

...wait (mutex); v.— v + 1;signal (mutex) ;

endcycle "Q"

...wait (mutex); print (?;);

coendend

The declaration of a semaphore mutexhas been extended with an integer constantdefining the initial number of available sig-nals:

varmutex: semaphore (initial)A semaphore is now characterized by threeinteger components :initial the number of signals initially

availablesent the number of signal operations

completedreceived the number of wait operations

completedThe semaphore invariant must therefore berevised slightly:

0 received sent + initialFor the semaphores used in Section 7, initialis zero. In Algorithm 5, initial is equal toone.

To make a sequence of statements SI, S2,. . . , Sn a critical region, we enclose it by apair of wait and signal operations on asemaphore mutex initialized to one:

var mutex: semaphore(1)

;

. . wait (mutex) ; Si; S2; . . . Sn;signal (mutex);. . .

The initial signal allows precisely one proc-ess to enter its critical region. Once a proc-ess has consumed the available signal andentered its critical region, no other processcan enter a critical region associated withthe same semaphore until the former proc-

ess leaves its critical region and producesanother signal.

A more rigorous proof of the mutual ex-clusion depends on the following observa-tions:

1) Independent of how wait and signaloperations are used they maintain the sem-aphore invariant:

0 received sent + 12) When a semaphore is used to achieve

mutual exclusion, a process always executesa wait operation followed by a signal opera-tion. At any given time, some processes mayhave executed a wait operation, but not yetthe corresponding signal operation. So thestructure of the program shows that the in-variant:

0 sent receivedis also maintained.

3) Finally, it is clear that the number ofprocesses that are inside their critical re-gions at any given time are those processesthat have completed a wait operation butnot yet the following signal operation, thatis:

inside = received — sentBy combining these three invariants, we

find that the first assumption about criticalregions is satisfied:

0 Sj inside 1At most, one process at a time can be insidea critical region.

Assuming that processes are scheduledfairly when they are inside their critical re-gions, we can also conclude that the state-ments executedwithin thecritical regions ofAlgorithm 5 will terminate within a finitetime. And if the scheduling of processeswaiting for timing signals in the semaphorequeue is also fair, then aprocess can only bedelayed a finite number of times whileother processes enter critical regions aheadof it ; so a process will always be able even-tually to enter its critical region. The im-plementation of critical regions in Algo-rithm 5 is therefore correct. Notice that theanalysis of this concurrent program is statedin terms of the implicit variables, received,sent, and inside.

A semaphore is an elegant synchronizingtool for an ideal programmer who nevermakes mistakes. But unfortunately the con-sequences of using semaphores incorrectlycan be quite serious. A programmer mightby mistake write the wait and signal opera-tions in reverse order for a particular criti-cal region:

signal(mutex) ; . . . wait (mutex) ;

In that case, the system will sometimes per-mit three processes to be inside their "criti-cal regions" at the same time. This is atime-dependent error that only reveals itselfif other processes enter critical regions whilethe erroneous critical region is being exe-cuted.

Another serious error would be the fol-lowing:

wait (mutex) ; . . . wait(mutex) ;This one causes the process executing thecritical region to wait forever at the end ofit for a timing signal that will neverbe pro-duced. Since the incorrect process is unableto leave its critical region, other processestrying to enter their critical regions willalso be delayedforever. Notice, that the be-havior of the other processes is only in-fluenced by the erroneous process after thelatter has entered its incorrect region. So theerror is clearly time-dependent. Such a sit-uation in which two or more processes arewaiting indefinitely for synchronizing con-ditions that will never be satisfied is calleda deadlock.

These examples show how easy it is tocause a time-dependent error by means of asemaphore. Even if a semaphore is usedcorrectly it still does not provide us with asatisfactory notation to indicate that theviolation of the rule of disjointness withrespect to the variable v in Algorithm 5 isdeliberate.

A semaphore is a general programmingtool that can be used to solve arbitrary syn-chronizing problems. Hence a compiler can-not always assume that a pair of wait andsignal operations on a semaphore initializedto one delimits a critical region. In particu-lar, a compiler cannot recognize the fol-

lowing errors: if a pair of wait and signaloperations are exchanged, if one or both ofthem are missing, or if a semaphore is ini-tialized incorrectly. A compiler is also un-aware of the correspondence between ashared variable v and the semaphore mutexused to gain exclusive access to v. Conse-quently, the compiler cannot protest if acritical region implemented by a semaphoremutex by mistake refers to another sharedvariable w (instead of v). Indeed, a com-piler cannot give the programmer any assis-tance whatsoever in establishing critical re-gions correctly by means of semaphores.

Since semaphores alone do not enable aprogrammerto indicate whether a variablev should be private to a single process orshared by several processes, a compiler musteither forbid or permit any process inter-action involving that variable. To forbidprocess interaction is unrealistic (since itprevents us from building interactive multi-programming systems consisting of co-operating processes) ; to permit arbitraryprocess interaction is disastrous (becauseof the danger of irreproducible program-ming errors). We must therefore concludethat semaphores do not enable a compilerto give a programmer the effective assist-ance in error detection that he should ex-pect from an implementation of a high-levellanguage.

To improve this situation, I have sug-gested a structured notation for shared vari-ables and critical regions [2]. A shared vari-able v of type T is declared as follows:

var v: shared TConcurrent processes can only refer to andchange a shared variable v inside a struc-tured statement of the form:

region vdo Si ; S2 . . . Sn endThis notation indicates that the sequence ofstatements SI, S2, . . Sn should have ex-clusive access to the shared variable v. Byexplicitly associating a critical region withthe shared variable on which it operates theprogrammertells thecompiler that the shar-ing of this variable among concurrent proc-esses is a deliberate exception to the rule ofdisjointness; at the same time, the compiler

Computing Surveys, Vol.

S,

No.

4,

December 1973 Computing Surveys, Vol. 5, No.

4,

December 1973

v:=0;

v.— 0; signal (mutex);end

*

Per Brinch Hansen240 I imcurrent Programming Concepts241

can check that a shared variable is usedonly inside critical regions and can generatecode that implements mutual exclusion cor-rectly. It is perfectly reasonably to use sem-aphores in the underlying implementationof this language feature, but at higher levelsof programming the explicit use of sema-phores to achieve mutual exclusion is de-batable. A similar notation for critical re-gions was developed independently byHoare [6].

Algorithm 6 shows the use of a sharedinteger v and two critical regions to solvethe previous problem (See also Algorithm5).

Algorithm 6 Critical regions representedby a structured notation

var v: shared integer;begin

cycle "Q". . . region vdo print (v) ; v:= 0end. .

endcoend

end

It has been our persistent goal to look formultiprogramming features that can be un-derstood in time-independent terms. Sincethe precise ordering of critical regions intime is unknown, a time-independent as-sertion of their net effect can only be an as-sertion about a propertly of the associatedshared variable that remains constant—inshort, an invariant I that must be true afterinitialization of the variable and before andafter each critical region operating it.

A relationship 7 that remains true at alltimes must in some way reflect the entirehistory of the processes referring to theshared variable. So we find that the in-variant for Algorithms 5 and 6 is expressedin terms of the total number of jobs exe-cuted and reported throughout the existenceof the multiprogramming system. However,since the range of such implicit variables is

unbounded, they cannot be represented in acomputer with a finite word length. On theother hand, actual variables being bound toa finite range can only represent the mostrecent past of a system's history. We musttherefore find a function f(x, r) of the im-plicit variables x and r with a finite rangethat can be represented by an actual vari-able v. In Algorithms 5 and 6 the function is

For a semaphore, the invariant is a rela-tionship among the implicit variables rep-resenting all signals sent and receivedthroughout the lifetime of a semaphore:

0 received sent + initialIn a computer, these implicit variables canbe represented by a single integer (equal tosent + initial — received) that must remainnon-negative.

People with a strong interest in correct-ness proofs may well find it helpful to de-clare implicit variables and implicit state-ments referring to them explicity in theprogram text. Implicit quantities have noeffect on the execution of a program; theirsole function is to facilitate program veri-fication by making assumptions about thesystem's history explicit.

Implicit variables and statements shouldbe subject to the following restrictions:

1) Assertions about a program may referto actual as well as implicit variables.

2) Expressions involving actual and im-plicit variables may formally be "evalu-ated" and "assigned" to implicit variables.

3) Expressions evaluated and assigned toactual variables may only refer to actualvariables.

Critical regions referring to differentshared variables can be executed simulta-neously. The use of nested critical regionscan, however, lead to a deadlock unlessprecautions are taken. Consider, for exam-ple, the following program with two sharedvariables v and w of types T and T':

cobegin"P" region v do region w do . . . end

end

"Q" region w do region v do . . . endend

coend

Process P can enter its region v at the sametime that process Q enters its region w.When process P tries to enter its region w,it will be delayed because Q is alreadyinside its region w. And process Q will bedelayed trying to enter its region v becauseP is already inside its region v.

The deadlock occurs because the twoprocesses enter their critical regions in op-posite order and create a situation in whicheach process is waiting indefinitely for thecompletion of a region within the otherprocess. It can be proved that a deadlockcannot occur if all processes enter nestedregions in the same (hierarchical) order[I]. A compiler might prevent such dead-locks simply by checking that nested criticalregions refer to shared variables in the (lin-ear) order in which these variables are de-clared in the program.

It is an amusing paradox of critical re-gions that to implement one, we must ap-peal to the existence of simpler criticalregions (called wait and signal) . The im-plementation of wait and signal operations,in turn, requires the use of an arbiter—ahardware implementation of still simplercritical regions that guarantee exclusiveaccess to a semaphore by a single processorin a multiprocessor system. This use ofnested critical regions continues at all levelsof machine design until we reach the atomiclevel, at which nuclear states are known tobe discrete and mutually exclusive.

The main conclusion of this section mustbe that it is impossible to make useful as-sertions about the effect of concurrent state-ments unless operations on shared variablesexclude one another in time. Mutual exclu-sion is necessary to reduce a virtual infinityof possible time-dependent results to asmall number of possibilities. The use ofinvariant relationships simplifies the pro-gram analysis further. Together, these men-tal tools enable us to study concurrent pro-grams in time-independent terms by aneffort proportional to the number of criticalregions used. So in the end, our understand-

ing of concurrent processes is based on ourability to execute their interactions strictlysequentially; Only disjoint processes canproceed truly simultaneously.

9. CONDITIONAL CRITICAL REGIONS

We will now consider multiprogrammingsystems in which processes can wait untilcertain conditions are satisfied by otherprocesses. The classic example is the ex-change of messages between two processesby means of a buffer of finite capacity asdefined by Algorithm 7. Here the sendermust be able to wait while the buffer is full,and the receiver must able to wait while thebuffer is empty.

The message buffer v is declared as asharedrecord consisting of two components:a sequence s of messages of some type T,and an integer full defining the number ofmessagescurrently stored in the sequence.

Initially, the buffer is empty (full = 0) .Since the buffer has a finite capacity, theoperations used to send and receive a mes-sage must maintain the following buffer in-variant :

0 full capacityAlgorithm 7 Message buffervarv : sharedrecord

s: sequenceof T;full: integer;

endm, n: T;

comment send message m;regionv whenfull < capacity do

put(m,s) ;full:- full + 1;

end

comment receive message n;region v when full > 0 do

get(n,s);full:= full — 1;

end

To indicate that the sending of a messagemust be postponed until full < capacity, wewill use the conditional critical region pro-posed by Hoare [6] :

Computing Surveys, Vol. 5, No.

4,

December 1973

v:= 0;cobegin

cycle "P". . . regionudou.'=v + l end . . .end

f(x, r) — x — r — v

var v: shared T; w: shared T' ;

Computing Surveys, Vol. 5, No.

4,

December 1973

242 Per Brinch Hansen Concurrent Programming Concepts

243

regionv whenfull < capacity do . . . endWhen the sender enters this conditionalcritical region, the Boolean expression full< capacity is evaluated. If the expression istrue, the sender completes the execution ofthe critical region by putting a message mof type T into the sequence s and increasingfull by one. But if the expression is false, thesender leaves the critical region temporar-ily and enters an anonymous queue associ-ated with the shared variable v. The senderwill be allowed to reenter and complete thecritical region as soon as the receiver hasremoved a message from the buffer (thusmaking full < capacity).

Another conditional critical region is usedto postpone the receiving of a message untilfull > 0. If full = 0, the receiver leaves thecritical region temporarily and joins theanonymous queue. In that case, the criticalregionwill be continued when the sender hasput another message into the buffer andmade full > 0. At this point, the receiverwill take a message n of type T from thesequence s and decrease full by one.

In general, a conditional critical regionregion v when Bdo Si ; S2 ; . . . ;Sn end

is used to delay the completion of a criticalregion until a shared variable v satisfies aa specific condition B (in addition to an in-variant 7) .

When a process enters a conditional criti-cal region, a Boolean expression B is evalu-ated. If B is true, the critical region is com-pleted by executing the statements SI, S2,.. Sn; otherwise, the process leaves itscritical region temporarily and enters aqueue associated with the shared variable v.

All processes waiting for one condition oranother on the variable v enter the samequeue. When a process completes a criticalregion on v, the synchronizing conditions ofthe waiting processes are reevaluated. Ifone of these conditions is satisfied, the cor-responding process is allowed to reenter andcomplete its critical region.

The scheduling of waiting processes mustbe fair in the following sense: If a processis waiting for a condition B that is repeat-edly made true by one or more "producers"

Computing Surveys, Vol. 5, No.

4,

December 1973

and false by one or more "consumers," thecompletion of the given critical region canonly be delayed a finite number of times byother critical regions.

We will use conditional critical regions tosolve the following problem [8]: A streamof data elements of type T produced by aprocess PO passes through a sequence ofprocesses PI, P2, . . . , Pn that operate onthe data elements in that order:

Each pair of processes (Pi'-l and Pi,where 1 i n) is connected by a sequences(i) that can hold one or more data elementsof type T. The sequences 5(1),..., s(n)are kept in a common store with a finitecapacity. Algorithm 8 gives an overview ofthis pipeline system.

The common store is declared as a vari-able vof type pipeline (to be defined later) . jA process Pi receives a message ti of type T |from its predecessor and sends an updated jmessage to its successor by means of twoprocedures

receive(ti,v,i) send(ti,v,i+l)Algorithm 9 defines the data type pipe-

line and the procedures send and receive. Apipeline is a shared record consisting offour components: an array of sequences s;an array of integers defining the number offull locations ("messages) in each sequence;an array of integers defining the minimumnumber of store locations reserved perma-nently for transmissionof messages througheach sequence; and an integer defining the jnumber of store locations that are generally javailable for transmission of messagesthrough all sequences (when they have usedup their reserved locations).

Initially, all sequences are empty and thepipeline has been divided into reserved andgenerally available storage. A sequence s(i)can always hold at least reserved(i) mes-sages. When this amount has been used, thesequence must compete with other sequencesfor use of the rest of the available store. Sothe condition for sending a message throughsequence s(i) is

Algorithm 8 Pipeline systemvar v:pipeline; tO, t1, . . . , tn: T;begin

initialize (v) ;cobegin

cycle "PO"produce(tO)

;

send(tO, v, 1)end

cycle "Pn"receive (tn, v, n) ;consume (tn) ;

endcoend

endAlgorithm 9 Pipeline system (cont.)typepipeline = shared record

s: array 1 . . nofsequenceof T ;

full, reserved:array 1 . . nof integer;

available: integer;end

procedure send(t: T; var v: pipeline; i:l-.n);

if full (i) > reserved (i) thenavailable:^ available — 1;

endendprocedurereceive (var t: T; varv: pipe-

line; i: 1 . . n)

;

The condition for receiving a messagethrough sequence s (i) is

full(i) > 0A sequence s(i) may temporarily contain

more than reserved (i) messages. But thetotal number of messages stored in all se-quences cannot exceed the capacity of thepipeline. So the system must maintain thefollowing invariant:

It can be shown formally that Algorithm 9maintains this invariant, but the proof istedious and adds nothing to one's informalunderstanding of the pipeline.

In Hoare's conditional critical regions,processes can only be delayed at the begin-ning of a critical region. In practice, onemust be able to place synchronizing condi-tions anywhere within critical regions asshown in [2].

The conditional critical region is an im-portant description tool for multiprogram-ming systems. It remains to be seen whetherit also is a practical programming tool. Themain difficulty of achieving an efficient im-plementation is the reevaluation of syn-chronizing conditions each time a criticalregion is completed.

In simple cases, the reevaluation can bereduced (but seldom eliminated) by the useof semaphores. To do this, one must asso-ciate a semaphore with each synchronizingcondition. When a process makes a condi-tion B true, the process must check whetherother processes are waiting for that condi-tion, and, if so, produce a signal that willenable one (and only one!) of them to con-tinue. As I have pointed out elsewhere, [7]the efficiency of this scheme is bought at theexpense of increased program complexity.

Concurrent statements and critical re-gions seem well-suited to the design ofsmall multiprogramming systems dedicatedto user applications (but remember thatthey have not yet been implemented andused in practice). Although these concepts

Computing Surveys, Vol. 5, No.

4,

December 1973

PO -» PI -> P2 >Pn

full(i) < received (i) or available > 0

cycle "Pi"receive (ti,v,i) ;update(ti)

;

send(ti, v, i + 1 )

;

end

region vwhenfull(i) < reserved(i) or available

> Odoput(t,s(i));full(i):=full(i) + 1;

region vwhenfull(i) > 0 doget(t,s(i));it full (i) > reserved (i) then

available := available + 1 ;endfull(i):=full(i) -1;

end

full(i) 0 for 0 isg n

available + 23 max(full(i), reserved (i))i

= capacity

'*

244 Per Brinch Hansen < 'oncurrent Programming Concepts 245

'

are simple and well-structured, they do notseem to be adequate for the design of largemultiprogramming systems (such as oper-ating systems). The main problem is thatthe use of critical regions scattered through-out a program makes it difficult to keeptrack of how a shared variable is used byconcurrent processes. It has therefore re-cently been suggested that one should com-bine a shared variable and the possibleoperations on it in a single, syntactic con-struct called a monitor [1, 10, 11]. It is,however, too early to speculate about whatthis approach may lead to.

10. CONCLUSIONI have tried to show that the design of reli-able multiprogramming systems should beguided by two simple principles that areequally valid for sequential programming:

1) It should be possible to understand aprogram in time-independent terms by aneffort proportional to its size.

2) It should be possible to state assump-tions about invariant relationships amongprogram components, and have themchecked automatically.

I have also discussed some specific lan-guage features for multiprogramming. Toavoid misunderstanding, I ask you to re-gard these not as definite proposals butmerely as illustrations of a common theme.Better language concepts for multiprogram-ming will undoubtedly be proposed byothers. But I would expect any realisticproposal to:

1) distinguish clearly between disjointand interacting processes;

2) associate shared data explicitly withoperations defined on them;

3) ensure mutual exclusion of these op-erations in time ; and

4) include synchronizing primitives thatpermit partial or complete programmercon-trol of process scheduling.

ACKNOWLEDGEMENTS

I am indebted to my students Ram Rao, David

Smith,

and Sankaran Srinivas for many helpful

comments on this paper. I am also grateful for theconstructivecriticism of Coen Bron, Peter Denning,Brian Wichmann, Mike Woodger, and thereferees.

BIBLIOGRAPHY1. Brinch Hansen, P. Operating system princi-

ples. Prentice-Hall, Englewood

Cliffs,

NewJersey (July 1973).

An introductionto operating systems coveringsequential and concurrent processes, processorand store management, scheduling algorithmsand resource protection. Describes the RC4000 multiprogramming system in detail. In-troduces a programming language notationfor monitors based on the class concept ofSimula 67.

2. Brinch Hansen, P. "Structured multipro-gramming." Comm ACM 15, 7 (July 1972),574-578.

A condensed presentation of the viewpointsdescribed in the present paper. It suggests theuse of event queues within critical regions asa means of reducing the overhead of processscheduling.

3. Wirth, N. "The programming language Pas-cal." Acta Informatica 1, 1 (1971), 35-63.

A highly readable definition of a sequentialprogramming language that combines the al-gorithmic notation of Algol 60 with far moregeneral datastructures.

4. Dijkstra, E. W. "Cooperating sequential proc-esses." in Programming Languages, F. Genuys,(Ed.) Academic Press, New York, New York,1968.

The classical monograph that introduced con-current statements, semaphores, and criticalregions. It also contains Dekker's solution tothe mutual exclusion problem.

5. Habermann, A. N. "Synchronization of com-municating processes." Comm ACM 15, 3(March 1972), 171-176.

An axiomatic definition of the semaphoreoperations, waitand signal.

6. Hoare, C. A. R. "Towards a theory of parallelprogramming." in Operating Systems Tech-niques, C. A. R. Hoare and R. H. Perrott,(Eds), Academic Press, New York, New York,1973.

The original proposal for conditional criticalregions which includes an axiomatic definitionof disjoint processes and critical regions.

7. Brinch Hansen, P. "A comparison of two syn- Ichronizing concepts." Acta Informatica 1, 3 |(1972), 190-199. .

A comparison of the use of semaphores andconditional critical regions to solve a schedul-ing problem.

8. Dijkstra, E. W. "Information streams sharinga finite buffer." Information Processing Letters1, (1972), 179-180.

A solution to the "pipeline"problem by meansof conditional critical regions.

9. Dijkstra, E. W. "The structure of The multi-programming system." Comm ACM 11, 5(May 1968), 341-346.

A brief description of the hierarchical struc-ture of The multiprogramming system.

10. Dijkstra, E. W. "Hierarchical ordering ofsequential processes." Acta Informatica 1, 2(1970,115-138.

A more extensive motivation of the basic

design decisions made in The multiprogram-ming system (See also 9). Recommends theuse of monitors (called "secretaries") for fu-ture operating system design.

11. Hoare, C. A. R. "A structured paging system."Computer Journal 16, 3 (August 1973), 209---J 14.

Illustrates the use of the monitor conceptpresented in (1) for the design of a demandpaging system.

Computing Surveys, Vol. 5, No. 4, December 1973 Computing Surveys, Vol.

S,

No. 4, December 1973