PEI: A language and its refinement calculus for parallel programming

Parallel Computing 18 (1992) 1167-1184 North-Holland

1167

PARCO 704

PEI: A language and its refinement calculus for parallel programming *

E. V i o l a r d and G. -R. Pe r r in

University of Franche-Comt~, Laboratoire d'lnformatique, F-25030 Besant~on cedex, France

Received 11 February 1992

Abstract

Violard, E. and G.-R. Perrin, PEI: A language and its refinement calculus for parallel programming. Parallel Computing 18 (1992) 1167-1184.

The aim of this paper is to introduce a new modeling, using symbols and functional notations, called the language PEI (as Parallel Equations Interpretor), which could unify the classical approaches of program parallelization, according to the sorts of problems and the target computation schemes. Due to its fundamental structure, disconnected from concrete drawings, this modeling offers a straightforward generalization to adress convex or non-convex computation domains, synchronous or asynchronous computations. This sort of programming formalism allows a powerful structuration of statements and a stepwise transformation technique based on a semantical equivalence definition and a refinement calculus. From initial problem statements, transformations of expressions lead to various definition structures, which can match given computation models.

Keywords. Recurrence equations; functional language; systolic algorithms; parallel architectures; program equivalence; refinement.

0. Introduction

It is now admitted that reliable parallel programming techniques should be founded on formal transformations of programs, defined from data dependencies analysis and taking architectural-related computing models into account. To manage these transformations safely, a convenient formalism has to be defined, to express the successive statements, and the transformation rules themselves (cf. the works of Chandy and Misra [2]).

Due to the results of Karp, Miller and Winograd [8], a convenient abstract expression of programs c~n be a set of recurrence equations. Indeed, prefiguring architecture evolutions, the introduction of the uniform recurrence equations did define a formal frame where the classical notions of data dependencies, potential parallelism and computation scheduling were presented.

Since this presentation many proposals have generalized this abstract modeling. Affine recurrences on integral convex domains are mainly studied for systolic synthesis (see for example [5,9,14,11], etc., and ALPHA [10] or CRYSTAL [3] for language aspects). Other

* This work has been supported by the French Coordinated Research Program C 3 of the CNRS. Correspondence to: E. Violard, University of Franche-Comt~, La~ratoire d'lnformatique, F-25030 Besan~on cedex, France, email: [email protected]

0i67-8191/92/$05.00 © 1992 - Elsevier Science Publishers B.V. All rights reserved

1168 E. Violard and G.-R. Perrin

modelings are based on explicit data flow expressions: see for example the design of the language LUSTRE for reactive systems ([13]). All these formal frames, each of them relative to some tar.get computation scheme, allow - fruitful results due to their strict mathematical semantics~ b~t they also find their limit according to more general sorts of problems and to new architecture proposals. Some other languages are based on non-deterministic transitions on a multiset of values, from which an efficient execution model has to be defined (see e.g. :[11).

Our aim is to introduce a new modeling called the language PEI (as Parallel Equations Interpretor), which could unify the classical approaches of program parallelization, according to the sorts of problems and the target computation schemes. A program will be presented as a set of equations on symbols and functions, in such a way that they imply a very simple semantics substitution rule. Many syntactical transformations can then operate on equations, as folding, unfolding, application, abstraction, etc.

So, a problem and its solutions can be expressed with this single language PEI. The design of the solutions consists in transforming statements and maintaining the global semantics of the problem. The transformations are expressed in PEI too, by defining operators and equivalence rules. They are particular steps of a general refinement process going from abstract programs, satisfying some specifications, to an executable coding.

Section 1 is devoted to a definition of the language. After a short presentation of its main notation aspects, a mathematical semantics of the language is introduced. It is based on the concept of multiset and data field. Operators on data fields define the semantics of the language constructions. In Section 2, we present transformation rules. A refinement calculus of PEI programs is presented in Section 3, which introduces the operational semantics of the language. Section 4 details the classical example of the Gaussian elimination algorithm. Last, related works are compared in Section 5.

1. Presentation of the language

1.1. A short presentation

Classical languages founded on the recurrence equations concept (ALPHA [10] or CRYS- TAL [3] for example) consider variables or expressions as functions which map index sets to a set of values. These function domains are built from integral convex polyhedra in Z n. The domain and the values of the variables are defined from equations which define the data dependencies.

Example. The mat~rix-vector product. Let 3 - ( a , , ~ ) an m × p matrix, V = ( v k) a p-dimensional vector and X = A . V . Each

component x i of X is defined by: p

xi~- ~ a i , k × V k, l<_i<_m. kffil

In a recurrence equation form, it can

Si,o ffi 0 1 _<i (1)

Si ,k- 'Si ,k_l + a i , k X V k l <i (2) xi - si,p 1 < i (3)

where s~, k are intermediate results for the scalar products. The recurrence equation (2) emphazises uniform dependencies (0, 1) for the calculation of s~, k.

be calculated on the domain {1..p} of 7/by:

_<m

_<m, 1 < k < p

~ m ,

PEI: A language and its refinement 1169

Other languages such as GAMMA [1] or LINDA [6] are founded on non-deterministic transitions on a multiset of values. The main problem for these expressions is to define an efficient execution model. Nevertheless they express in a nice abstract way a very large class of problems.

A program in PEI benefits of such an abstraction of the variables domains, but fundamen- tally PEI is a recurrence equation language. Variables are defined as sets of values on abstract domains which are in fact the set of all their possible geometrical drawings in 7/n. They denote structures called data fields.

A program in PEI is of the form

W: [[ PRE; POST 1]

where W is the list of output variables, PRE and POST are lists of equations E = E ' , where E and E ' are expressions built from operators on variables. An equation in the PRE part generally defines the geometrical drawing of an input variable domain. Equations in the POST part define output or intermediate variables.

Operators either define the computations of the values of a data field (summation and product on the previous example), or express the data dependencies (the uniform dependencies (0, 1) hereabove), or refer any constructive aspect of data fields.

Example. The matrix-vector product (continued). In PEI, the previous problem can be expressed in the following way. Matrices A and S are

real-data fields whose geometrical drawings in 7/2 are {(i, k)l 1 < i < m A 1 < k < p}. Vectors V and X are real-data fields whose drawings are {k I 1 _< k < p}. This is the PRE part of the program defining the drawing of the input data fields:

A =A(i , k ) l (1 <_i<_m ̂ 1 <_k <_p)A,

V f A k l ( l <_k <_p)V;

In the right part of each equation, an operator is applied on a data field. Such an equation defines the drawing of A or V, for this operator which filters a given drawing, does not change these data field.~ Notice th.~t such operators could be associated with symbols as following:

A = matrix A, V - vector V;

where symbols are supposed to be defined in the context of the program as:

matrix ffi A( i, k ) l ( 1 < i < m ^ 1 < k < p ) ,

vector = A k I(1 < k < p) ,

The POST part of the program is defined by considering the recurrence equation system. To emphazise calculations and dependencies, this system can be re-written as:

Si,k=O 1 <i <m, k = 0 (1)

Si.k=f(Si.k_l, ai,k, Wi,k) 1 <i <m, 1 <_k <p (2) wi, k -~u k 1 <i <m, 1 <k <p (3)

xi--si, p 1 <i <m (4)

where f is the function from R 3 to R defined by f (x , y, z ) = x + y x z. Equation (1) is expressed in PEI as:

A(i, k ) l ( l < _ i A m ^ k = O ) S = A ( i , k ) l ( l _ < i A m ^ k = 6 ) O .

1170 E. I/iolard and G.-R. Perrin

This means that a sub-data field of S whose drawing is {(i, k ) l l (A( i , k) . ( i , k - 1)S, A, W).

This means that the complementary sub-data field of S whose drawing is {(i, k)!1 < i < m A 1 _< k _<p} is equal to the application of the operator A(x, y, z).x + y × z on a nuple of data fields. This nuple is composed of three data fields. The first one is the result of the application of the operator A(i, k).(i, k - 1) on S which expresses the dependency (0, 1). This operator can be associated with a symbol pre in the context of the progr~m.

Equation (3) is expressed as:

W=A(i , k) l (1 < i_<m). (1 , k ) (V 'Ak . (1 , k ) ) .

This means that the data field W is the result of the application of the operator a(i, k)l(1 _<i <m).(1, k) which broadcasts the values of the data field resulting of the application of the operator Ak.(1, k) on V. The operator a(i, k)l(1 < i <m).(1, k) can be associated with a symbol broadcast in the context of the program.

Last, equation (4) can be expressed as:

X:Ai . ( i , p ) = a ( i , k) l (1 <i <_m ^ k f p ) S .

This means that the data field X is the result of the application of the o0erator A,.(~. p) on the sub-data field of S whose drawing is {(i, k) I 1 < i < m A k = p}.

The resulting program PEI is the following: x : [[

A ffi matrix A, V ffi vector V; A(i, k)l(1 (pre S, A, broadcast (V: Ak.(1, k))), ~ X: Ai.(i, p)ffiA(i, k)/(1 <_i <m ^'k •p)S

]] where the symbols are defined in the context of the program as:

matrix ffi A(i, k) / (1 _<i < m ^ 1 < k < p ) , vector ffi Ak/(1 S k _< p), pre ffi A(i, k).(i, k - 1), broadcast ffi A(i, k) / (1 < i < m).(1, k)

1.Z Data field definition

In the literature about recurrence equations, variables domains are built from integral convex polyhedra in 7/n. These variables associate a value with any point in their domain. To abstract this concept we work backwards by considering the set of values resulting of some computation.

As defined in new languages as GAMMA or LINDA, this set is in fact a multiset, also called a bag, i.e. a collection of elements in which an element can appear more than once (cf. Fig. 1). This abstract data structure defines the semantics of a variable. It is a foundation for PEI.

In order to address the values, this multiset has to be described as a set, by using a numerable index set. Moreover, to define a computation, this index set is partially ordered.

PEI: A language and its refinement

Fig. 1. A multiset of values.

1171

So, any element v of ,~ multiset M is described by a pair (v, ref), where ref distinguishes clones of v in M. To organize the index set in a partial computation ordering, ref is defined as a pair or:z, where z denotes a geometrical coordinate in Z n and or is any bijection from Z ~ to 7/m. This bijection or means all the possible transformations on these coordinates, in order to yield a convenient computation ordering on Z m. We say that M is described as a set, in a relative way, within the bijection or.

Definition 1. We call data field any set of the form {(v, or:z)/v ~ M}, where M is a multiset, z denotes elements of 7/n and or is a bijection from 7/n to 7/m.

Definition 2. Let X a data field, We call drawing of X the subset {z / (v , or:z) ~X} of Z n (Fig. 2).

The values in the multiset associated with a given data field have the same type, say v. Such data fields are called v-data fields.

Examples - Let c an element of v and R a predicate. {(c, or:z)/R(z)} is a constant data field, whose

drawing is the set of points z satisfying R. - Let P=-<v(m, p)/l<_m, l<_p<_m>-, where v: {(m, p)/l<_m, l<_p<_m}--,• is de-

fined as

v ( m , p ) - v ( m - l , p - 1 ) + v ( m - l , p ) i f m # l , p # l a n d p # m , v(m, p ) = 1 else.

For any bijection or, {(v(m, p), or:(m, p ) ) / 1 Am, 1 _<p <m} is a data field associated with the multiset P. Its drawing is the classical Pascal's triangle.

Notation. In PEI, an equation E - E ' denotes the equality of two data fields.

1.3. Operators

In a semantical point of view, operators in PEI are functions. These functions are applied on data fields and return data fields. According to the way they are applied, operators are called functional, routing or change of basis operators.

Notations - The operator denoted by Ak/P(x) . f (x) in PEI, is defined from the fm~ction f .P is a

predicate which defines the domain Df of f. - The composition of operators is defined from the composition of functions. It is denoted by

- If f is a bijection, Ax/P(x) . f (x ) ~ denotes the operator defined from the inverse of f. - The operator (Ax/P(x). , f(x) , Ax/P' (x) . f ' (x) ) is defined from the pair (f , f ' ) of func-

tions. This concept is generalized to nuples.


• • • • • • •

2 2 0

1 ~ 1 ~ 1 • • • • • •

• • • • •

Fig. 2. An example of data field associated with the multiset on Fig. 1: (a) an initial drawing, (b) a drawing resulting of some bijection o- from 7/2 to 7/2, (c) an associated partial order.

Particular notations - hx. f (x) denotes hx/true.f(x), - hx /P(x) denotes Ax/P(x).x.

Examples - The expression h x / ( x > 0) . l /x denotes an operator in PEI defined from the function

which inverses any positive number. - Let us consider the operator denoted by Ax/(x > 0).2 × x in PEI. The inverse of this

operator is denoted either by h x / ( x > 0).2 Xx ̂ or by h x / ( x > 0).x/2.

1.3.1. Functional operators A functional operator defines comrmtations or extractions of some values of a data field.

Definition 3. Let f be any function from v to w, and X a v-data field. The application of the function f on X defines the w-data field {(f(v), cr:z)/(v, ~r: z) ~ X ^ v ~Df} (Fig. 3).

Notation. The application of a functional operator hv/P(v) . f (v) on an expression of type data field E is denoted in PEI by hv/P(v).f(v)t> E.

Examples - Let X a variable whose values are pairs of numbers:

A(a~,.b).a + b ~ X d e n o t e s a data field whose values are the sums of the pairs of numbers in X.

- Let X a variable, whose values arc numbers: Aa/a > O~ X denotes a data field whose values are the positive values in X.

2 2

3 4 4 3 G

2 5 5

2 2

Q • ,

G •

• @

(a) (b) (c) Fig. 3. Data field resulting of the application of the functional operator h x / x mod 2 = 0 . x / 2 on the previous data

field on Fig.2.


2 2 CI

3 3 " • " 6 •

2 2 ~ • " b o

Fig. 4. Data field resulting of the application of the routing operator a(i, j) / i > j .( i- 1, j - 1) on the data field defined on Fig. 2.

1.3.2. Routing operators A routing operator defines computations or extractions of indices of a data field drawing.

Definition 4. Let p any .function from Z n to Z n and X a v-data field. The application of the function p on X defines the v-data field {(v, or:z)/(v, or:p(z)) ~ X Az ~Dp} (Fig. 4).

Notation. The application of a routing operator Az/Q(z) .p(z)on an expression of type data field E is denoted in PEI by Az/Q(z).p(z)E.

Examples - Let X be a variable, whose drawing is a subset of Z e:

A(i, k).(i, k - 1 ) X denotes a data field whose values are moved along the k axis. Its expresses the uniform dependency (0, 1) presented in the example, Section 1.1.

- For the same variable X: A(i, k) / (1 <m A k =O)X denotes a sub-data field of X whose drawing is a line in 7/2.

1.3.3. Change of basis operators A change of basis operator is ,Jefined from a bijection which allows to determine

reindexings of some values of a data field. Such re-indexings redefine the bijection or, in order to yield a convenient partial order on the drawing. For any bijection h, the resulting data field is defined within the new bijection or o h-n. Its new drawing is the set of h(z), where z runs over the preceding drawing•

Definition $. Let h be any bijection from Z n to Z p and X a v-data field. The application of the function h on X defines the v-data field {(v, or., h- t :h(z ) ) / (v , or:v) e X Az EDh}.

Notation. The application of a change of basis operator A z/R(z) .h(z) on an expression of type data field E is denoted in PEI by E:az/R(z).h(z).

Examples - The previous example, in Section 1.1, defines the change of basis operator AiM, p). - In this previous example generalized operator Z can be reduced in several ways. All these

way can be express by different change of basis operators. For example, we can modify the program, by introducing a change of basis operator A ( i , k ) . ( i , p - k + 1 ) associated with the symbol merge, in the following way: x : [[

A = matrix A, V - vector V; A(i, k) / (1 (A, broadcast(V: Ak.(l, k)), X: Ai.(i, p) = a(i, k) / (1 _<i _<m Ak =p)S

11

1174 b,. Violard and G.-R. Perrin

2 • o

4 3 4 3 ~ . • • o oh "1 • " • • " •

~ Q 2 2

Fig. 5 Data field resulting of the application of the change of basis operator A(i, j ) . ( 3 - i, j ) on the data field defined on Fig. 2.

- The packing function. Considering any variable X, whose drawing D is a subset of Z, we introduce a canonical change of basis operator which packs this drawing in an interval of ~, whose length is the cardinality of D. Associated with any variable X, this operator is defined as the result of a predefined function tilda, called the packing function, applied on X. More formally, for any variable X, whose drawing D is a subset of Z, tilda(X) denotes the increasing bijection from D to N, whose image is [O..card(D)[. For example, if X is a variable whose drawing is D - {1, 3, 5, 7, . . . }, the drawing of the expression X: tilda(X) is N. In this case, the symbol tilda(X) is associated with the change of basis operator M.( i - 1)/2.

- Figure 5 presents the data field resulting of the application of a change of basis operator on the data field defined on Fig. 2. In this particular case the bijection or was equal to this change of basis. This means that the induced order is considered as a convenient computation ordering.

1.4. Nuples of data fields

Because all operators on data fields are unary operators, we define nuples of data fields. They are data fields whose values are nuples of values.

Definition 6. A nuple of data fields is a data field {((vl , . . . ,v , ) , or:z) / (vl , or: z ) ~ X ~ ^ "'" ^ (v, , or: z) ¢ X,}, where Xi, X2, . . . , X n are data fields.

Notation. The nuples of expressions E~, E2,..., E, are denoted in PEI by (E l, E2,..., En).

1.5. Symbols and context

A program can be defined within some context which associates names with operators. The structure of the program is then more clearly reaveled. Symbols and contexts are defined in the following way:

with INVeW:[[ Pe ;POSr l]

where INV is a list of declarations of the form: name = A-expression. The A-expression denotes a function built from composition, inversion and nuples of operators or names•

Example. In the previous example, in Section 1.3., the symbol merge can be associated with another change of basis operator for some other reduction of the generalized operator ,Y,, as for example A(i, k).(i, (k mod p ) + 1). In the same example, the symbol pre is associated with the routing operator A(i, k).(i, k - 1).


1.6. Example - The Gaussian elimination algorithm

This example is one of the most famous in the literature on systolic algorithms, and specially in studies about synthesis methodologies. The Gaussian elimination algorithm solves the linear system AC/ = B where A is an n x n matrix and B an n - dimensional vector by first transforming A into a triangular matrix an then solving a triangular system.

We are only concerned here by the triangularization of A. Since the transformation affects vector B, we consider that A is an n X (n + 1) matrix whose last column is B. Let ai,j and bi the initial values of matrix A, ai,j and b,! the elements of the resulting triangular matrix. They are defined by the following recurrence equation system:

x(i, j, k) =x(i, j, k - 1) -x(i, k, k - 1) xxjk, j, k - l)/x(k, k, k - 1)

k+l<i_<n,k+lsj<n+l,

x(i, j, 0) = 1 ai j b ’

i

llk<n-1 1 liln,l CjSn

lli<,y j=n+l

a i,j=X(i, j, i - 1) 1 l&n, 1 ,<jln

F: =x(i, n + 1, i - 1) 1 liln.

In PEI, this problem is expressed by the following program: with

pre = h(i, j, k).(i, j, k - 1), shift = h(i, j, k ).(i, k, k - l), move = A(i, j, k ).(k, j, k - l), lie = ACi, j, k).(k, k, k - 1)

:‘, B’: [[ A = A(i, j)/(l (pre X, shift X, move X, lie XI, A&j, k)/(l<i<nr\ 1 sj9zAk=O) X=A:A(i, j).(i, j,O), A(i, j, k)/(l liln Aj= n + 1 A k = 0) X = B:Ai.(i, n + 1, 0),

A’:A(i, j).(i, j, i - 1) = A(i, j, k)/(l si <It A 1 5 j 5 n A k = i - 1)X, B’:Ai.(i, YE + 1, i - l)=A(i, j,k)/(l<i<nAj=n+lAk=i-1)X

11

2. Programs transformation rules

Reliable parallel programming techniques are founded on formal transformations of programs, defined from data dependencies analysis and taking architectural-related computing models into account. The formalism PEI has been defined to manage these transformations safely and to express the successive statements, and the transformation rules themselves. So, a problem and its solutions can be expressed with this single language PEI. The design of the solutions consists in transforming statements and maintaining the semantics of the problem. The transformation expressions are specified in PEI too.

The mathematical definition of programs PEI allows classical functional transformations, such as the permutations of equations sets, substitutions, etc. The purpose of this section is to


present other transformation rules of PEI programs. This set of transformation rules which maintain the global semantics of programs, is composed of two subsets. Some rules are such that the transformed programs define output variables denoting equal data fields. We say that the initial program and the transformed one are semantically equal. Some other ones only define a semantical equivalence of programs, i.e. programs that define output variables associated with the same multiset.

2.1. Operator properties

In this section, we present some mathematical properties on data fields and operators, which allow to link two expressions (i.e. two identical data fields) in a PEI equation. They permit then to substitute an expression for an other, and can be considered as transformation rules of programs in PEI, which maintain the semantical equality of these programs.

Notauons - E, E ' denote expressions of type data field - func and func ' denote functionnal operators - rout and rout' denote routing operators - base and base' denote change of basis operators.

Properties

func ~> ( func ' t> E) = func ° func' ~> E

rout ( rou t 'E) ffi rout' o rout E

( E: base): base' - E: base' o base

rout ( func t> E ) =func ~, ( rout E)

( func t> E ) : base - f u n c ~, ( E: base)

( rout E) : base = base o rout ° base ^ ( E: base)

(rune, func') (E, E') = ( func E, func' , E') rout( E, E ' ) f f i (rout E, rout E ' )

( E , E ' ) : base ffi ( E : b a s e , E': base )

Example. The Gaussian elimination algorithm (continued). Let us consider the following expression in program PEI Section 1.6.:

h(i , j , k ) / ( k + 1 <_i < n ^ k + 1 < j <_n + 1 A 1 < k < n - 1)X.

[1] [2] [3] [4] [s] [6] [7] [8] [9]

To illustrate the property [6] we apply the change of basis operator A(x, y, t).(x, t, x + y + t), associated with the symbol timing"

(A(i, j , k ) / ( k + 1 ~ i <_n A k + 1 <_j <_n + 1 A 1 <_k <_n- 1)X)" timing

=( t iming o A(i, j , k ) / ( k + 1 <_i <_n A k + 1 <_j <_n + 1 A 1 <_k <_n- 1)

o timing ^ )( X: timing )

=A(x , y, t ) / ( y + 1 <x < n A 1 _<y < n - 1Ax + 2 × y + 1 < t <x + y + n + 1)

( X: t iming).


Indeed, we can verify:

timing o A(i, j , k ) / ( k + 1 <i <n ^ k + 1 <j <n + 1 ^ 1 < k < n - 1)otiming ^

=timing o A(i, j , k ) / ( k + 1 <i <n A k + 1 <j <n + 1 A 1 < k < n - 1)

o A(X, y, t ) . (X , t - x - y , y)

= A ( x , y, t ) . ( x , t, x + y + t ) o A ( x , y, t)

/ ( y + 1 < x < n ^ y + 1 < t - x - y < n + 1 ^ 1 <y < n - 1).(x, t - x - y , y)

= A ( x , y, t ) / ( y + 1 < x < n Ay + 1 < _ t - - x - y <n + 1 A 1 <y < n - - 1)

= A ( x , y, t ) / ( y + l < x < n A l < y < n - 1 A x + 2 × y + l < t < x + y + n + 1)

2.2. Equivalence rules

Here, we define the semantical equivalence of data fields and programs, denoted by - . We first define formally this concept of equivalence. Then, we propose rules on PEI operators, which lead to equivalent programs. This equivalence is the mathematical foundation for our transformational approach to derive solutions from problem expressions in PEI.

2.2.1. Semantical equivalence Intuitively, we say that two data fields X and X ' are equivalent, iff they describe the same

multiset.

Definition 7. Let X = {(v, t r : z ) / v ~ M } and X ' = {(v, t r : z ) / v ~ M ' } be two data fields,

X - - X ' iff M = M ' (Fig. 6) .

Remark. We can also express this equivalence as:

X - X ' iff "<v/ (v , cr:z) E X > ~ = ' < v / < v , ~r:z) ~ X ' ~ .

Intuitively, we say that two programs ~ and ~ ' a-e equivalent, iff their output variables denote equivalent data fields. For sake of simplicity, in the following definition, the equivalence of data fields is applied on the associated variables.

Definition 8. Let ~ = X: [[PRE; POST]] and ~ ' = X ' : [[ PRE'; POST']],

~ - ~ ' iff X - X ' .

Remark. This definition is given for one output variable. It can be generalized to programs with several output variables.

2 2 . • •

3 4 4 3 • • • • •

2 5 5 • • • •

Fig. 6. Two equivalent data fields.


2.2.2. Rules We present rules which allow to transform a program into an equivalent one.

Notations. base and base,, b a s e 2 , . . . , base~ denote change of basis operators.

Rules

X: [[PRE; POST]] =-X': [[PRE; X ' = X : base, POST]]

x : x - o,,t x , e o s r ] ] -

X': [[ PRE; (rout I o basel~) X ' = routIX): basel, (rout 2 o base 2 ) X ' = rout2X): base2,

(rout n o basen ̂ ) X ' = routnX): basen, POST

]] whererout - Az /Q(z ) , rout i = Az /Qi ( z ) Vi ~ 1..n,

V (i ~ ! . .n)Qi(z) -- Q(z), i ~ j ~ Qi(z) A Qj ( z ) - fa l se Vi, j ~ 1..n

and X ' is a variable which does not appear in PRE or POST.

[lo] [11]

Comments [10] expresses that two data fields are equivalent within the application of a change of basis

operator. [11] is a rewriting of [10] where: - the drawing of X is {z/Q(z)}, - the bijections base i are the restrictions of base o n { z / / Q i ( z ) } , - and the set of {z/Qi(z)} defines a partition of {z/Q(z)} .

In practice the predicates Qi are naturally introduced, due to the regularity of problem expressions. The soundness of our transformational approach lies on transformations such that data fields appearing in a problem are equivalent to data fields of any solution. All along the transformations, this equivalence is obtained from change of basis operators base i which preserve the regularity of the solutions.

Remark. These rule are given for one output variable. It can be generalized to programs with several output variables.

3. Ref inement of PEI programs

3.1. Refinement calculus

Programming techniques founded on formal transformations are called programs refine- ments. A refinement process goes step by step from abstract programs, satifying some specifications, to executable ones, taking architectural-related computing models into account [12]. At a development step, some executable parts of such programs can be derived while other ones are still to be defined.

The refinement calculus of PEI programs is then based on an exteusion of PEI, in which specifications can be written, and a relation of refinement between its program, such that implementations refine their specifications. These extended programs are denoted by:

W:[[ PRE; POST]].


They can be either formal specifications, or PEI programs, or any hybrid statements. W is the list of output variables PRE and POST are either logical predicates or PEI expressions.

This calculus defines a partial order _ between programs as following. We say that a program ~ ' is a refinement of a program ~ if ~ ' satisfies every specification that ~ does. This is its definition:

Definition 9. For programs ~ and ~ ' , we have ~ _ ~ ' iff for all postconditions R,

wp( R) wp( R), where the weakest-precondition wp(~, R) of a program ~ = W: [[ PRE; POST]] is:

pre A ( post ~ R ),

where pre and post are predicates associated with lists of equations PRE and POST.

Properties - If post' ~ post, then W: [[PRE; POST]] c_ W: [[ PRE; POST']], - If pre ~ p r e ' , then W: [[PRE; POST]] c_ W: [[PRE'; POST]].

Remarks - If a program PEI ~ ' is a refinement of a program ~ , any program ~ " such that ~ ' - ~ "

is a refinement of ~ , - Two equivalent programs PEI refine each other.

Example. The Gaussian elimination (continued). Consider again the Gaussian elimination problem, i.e. the problem of the triangularization

of an n x n matrix A, to solve a linear system A . V = B. This is its specification: A', B': [[ A. V = B A pivots o f A ~ 0;

A'. V = B' A A' is a triangular matrix ]]

The program PEI presented in Section 1.6. refines this specification (from previous properties).

3.2. Operational semantics and refinement

Previous Sections 1.2 to 1.4 did define the mathematical semantics of PEI programs. It defines variables as data fields, which express multisets. Two programs were said semantically equal if their output variables denote equal data fields. They were said semantically equivalent if their output variables are associated with the same multiset.

An operational semantics has to define the set of computations associated with any data field definition. This means the definition of an order on the data field elements. This order is a partial order for parallel computations. We define this order, denoted by ~ , for some given partial order < on Z m, by considering the bijection or from Z n to Z m.

Definition 10. Let X a data field defined by X = {(v, or:z) /v ~ M } , where M is a multiset,

(v, t r :z ) ~ ( t " , or :z ' ) iff t r ( z ) < o r ( z ' ) .

Examples. Let us consider a bijection ~r from Z n to 7/m such as or (z)= (p(z) , t(z)), where p is a function from 7/n to 7/m- n and t a function from 7/n to ~. Note that such a definition is a classical way to introduce a scheduling and a mapping of the computations on a processor set.

1180 E. Holard and G.-R, Perrin

Let < an order on 7/m such that or(z)<or(z ' ) iff t ( z ) < t ( z ' ) on N. The induced operational semantics only defines computations scheduling. Let < an order on Z m such that or(z)<or(z ' ) iff p ( z ) = p ( z ' ) and t ( z ) < t ( z ' ) . The induced operational semantics defines computations mapping and the schedulings of the processors.

In the refinement process the explicit definition of the bijection or, by applying step by step some change of basis operators, introduces the operational expression of the solution.

4. Application to the design of a systolic solution for the Gaussian elimination algorithm

The design of a systolic solution for a given problem supposes to define a bijection or such as or(z) = (p(z) , t(z)) by determining a convenient change of basis in order to define the computation scheduling t. It may happen that the existence of such a scheduling requires to transform the initial equation set to re-arrange the data dependencies. In PEI, these preliminary refinement steps consists in introducing sub-data fields and possibly in changing some basis by applying rule [11].

In order to determine a nice mapping function p, next steps may consist in unifon~izing the dependencies to reach local routings: this is achieved in PEI by decomposing functional and routing operators. These points are illustrated hereunder with the Gauss elimination algorithm.

Step 1. Considering the change of basis operator A(x, y, t).(x, t, x + y + t) and the routing ones in the previous program (Section 1.6.), the application of rules [6] and [10] leads to the following equivalent program: with

pre ffi A(x, y, t).(x, y - 1, t - 1), s h i f t - A(x, y, t) .(x, y - 1, x + 2 × y - 1), move --- A( x, y, t).(y, y - 1, t - x + y - 1), lie - A(x, y, t).(y, y - 1, 3 × y - 1)

A', B': [[ A = A(i, j ) / (1 (pre X, shift X , move X, lie X ) , A(x, y, t ) /(1 <x <n ^ y = 0 ^ x + 1 < t < x + n ) X = A : A ( i , j).(i, O, i +j) , A(x, y , t ) / ( l < _ x < n A y f f i O A t = x + n + I ) X f B : A i . ( i , O , i + n + 1) A':A(i, j).(i, i - 1, 2 × i + j - 1)

=A(x , y, t) /(1 < x <_n A y = x - 1Ax + y + 1 < t < x + y + n ) X , B':Ai.(i , i - 1, 2 × i+n) f f iA(x , y, t)/(1 < x < n ^ y = x - 1 ^ t - x + y + n + 1)X

]] This change of basis defines a part of or that shows the scheduling of the solution.

Step 2. From this statement, in order to determine a mapping of this solution, we introduce a new program PEI, where compositions of symbols are substituted for the routing operators move, shift and lie: A', B': [[

A =A(i, j ) / (1 <i<_n ^ 1 < j < n ) A , B - Ai/(1 

( pre X, pre(init X), pre o diag X , pre o diag(init X)), A(x, y, t ) / (1 < x < n A y = 0 A x + 1 < t < x + n ) X = A " A(i, j).(i, 0, i +j) , A(x, y , t ) / ( l < x < n A y = O A t = x + n + 1 ) X = B : A i . ( i , O , i + n + 1), A':A(i, j).(i, i - 1, 2 x i + j - 1)

=A(x, y, t ) / (1 < x < n A y = x - 1 A x + y + 1 < t <x + y + n ) X , B':Ai.(i, i - 1, 2 x i + n) = A(x, y, t ) / (1 < x < n A y - - x - - 1 A t = x + y + n + 1)X

]] From the definition of the function composition, these new symbols associated with these

routing operators:

p r e = A ( x , y, t ) . ( x , y - 1, t - 1),

i n i t = A ( x , y, t ) . ( x , y, x + 2 x y + 1),

d i a g = A ( x , y, t ) . ( y , y, t - x + y )

define a data field X equal to the previous orle. Indeed operators satisfy:

shift = init o pre, move = pre o diag and lie = init o pre o diag.

Step 3. By decomposing the functional operator and using operator property [8], we obtain this new definition: A ' , B ' : [[

A = ~(i, j ) / (1 <i < n A 1 < j < n ) A , B = A i / ( 1 < i < n)B; A(x, y, t ) / ( y + 1 < x < n A 1 < y < n - - 1 A x + 2 X y + 1 < t < x + y + n + 1)X

=A(a, b, c ) . a - b X c ~ , (pre X, pre(init X) , pre o diag(A(a, b ) .a /b ~,(X, init X))),

A(x, y, t ) / (1 < x < n Ay = 0 A x + 1 < t < x + n ) X = A " A(i, j).(i, 0, i +j) , A(x, y , t ) / ( l < x < n A y = O A t = x + n + ! ) X = B : A i . ( i , O , i + n + 1), A': A(i, j).(i, i - 1, 2 × i + j - 1)

=A(x , y, t ) / (1 < x < n A y = x - 1 A x + y + 1 < t <x + y + n ) X , B " Ai.(i, i - 1, 2 × i + n ) = A ( x , y, t ) / (1 <x <n A y = x - - 1 A t = x + y + n + 1)X'

11 Step 4. Let us consider the variable Y defined as pre o diag(A(a, b ) . a / b ~ ( X , init X)) . The data field Y can be recursively re-defined, in the following way:

A(x, y, t ) / ( x - y = 1) Y = p r e o l e f t o A(x, y, t ) / ( x - y = 1)

(A(a, b ) . a / b t> ( X , init X ) ) ,

A (x , y, t ) / ( x - y > 1) Y = l e f t Y

where the symbol left is associated with the routing operator A(x, y, t ) . ( x - 1, y, t - 1). Indeed the routing operator A(x, y, t).(y, y, t - x + y) associated with diag, when applied on any (x, y, t) is equal to x - y compositions of the routing operator associated with left on this (x, y, t). By composition definition, we obtain pre o left o . . . o left - left o . . . o left o pre o left. Notice that this transformation is a very classical one, called uniformization in the fiterature. A', B': [[

A = A(i, j ) / (1 <_ i < n A 1 < j <_ n)A , B = M / ( 1 <_ i <_ n)B; A(x, y, t ) / ( y + l_<x_<nA l _ < y _ < n - 1 A x + 2 X y + l _ < t _ < x + y + n + 1)X

= A(a, b, c).a - b x c t>(pre X, pre(init X) , Y) , A(x, y, t ) / ( l < x <_n A y = O A x + l <_t <_x + n) X = A : A ( i , j).(i, O, i +j) ,


Fig. 7. A processor array for the Gaussian elimination algorithm.

A(x, y, t ) / (1 < x < n ^ y = O ^ t = x + n + 1) X=B:Ai.( i , O, i + n + 1), A(x, y, t ) / ( x - y = 1)Y

=preoleftoA(x, y, t ) / ( x - y = 1XA(a, b).a/bc> (X, init X)), A(x, y, t ) / ( x - y > 1) Y = l e f t Y, A': A(i, j).(i, i - 1, 2 × i + j - 1)

=A(x, y , t ) / ( l < x < n A y = x - - l A x + y + l < t < x + y + n ) X , B': Ai.(i, i - 1, 2×.; + n) = A(x, y, t ) / (1 <x < n A y = x - 1 A t = x + y + n + 1)X

]] At this step we consider that the refinement process is completed. The scheduling and a

regular mapping are determined, which describe the systolic array drawn for n = 4 (Fig. 7", The routing operator pre and left define the processor links. The routing operator int~ defines a memory cell. The systolic array topology is defined by the set {(x, y ) /y + 1 _<x _<n ^ 0 _< y _< n - 1} which is deduced from the data field drawing.

5. Related works

Some proposals exist in parallel programming, using recurrence equations: ALPHA [10] or CRYSTAL [3] for affine recurrence equations on integral convex domains, and syntactical transformations, LUSTRE for reactive systems ([13]), LEQ [4] or AP2L [7] for asynchronous interpretations, UNITY [2] for program specification and transformations, or last non-deterministic languages as GAMMA [1] or LINDA [6].

ALPHA allows to describe affine recurrence equations on integral convex domains. These domains define variable types as polyhedra in 7/n, by a set of inequations. The recurrence definition of any variable may be split in sub-domains, using an instruction case. Dependen- cies are expressed as A-calculus formulaes. A software package, called ALPHA DU CEN- TAUR, allows to edit and transform programs in ALPHA.

CRYSTAL offers a recurrence equation language, and its meta-language to perform transformations. It is founded on data fields, that are distributions of data values over an index domain. An index domain is a directed graph that represents data dependencies of a problem or processor links of an archtitecture. Notice the definition of an index domain includes the definition of the set of nodes of this graph: such an index domain is then what we called a drawing in PEI. A data field in CRYSTAL corresponds to what we called a data field in PEI. So, some transformations presented as domain morphisms in CRYSTAL are change of basis operators on variables in PEI to determine equivalent data fields. Last dependencies


in CRYSTAL, as in ALPHA, are explicit on index domains. On the contrary they can be expressed as symbolic routing operators in PEI, to abstract and structure the programs.

PEI can express synchronous or asynchronous computation schemes, depending on the definition of the bijection or. Only synchronous interpretations are defined in ALPHA or CRYSTAL. Likewise LUSTRE expresses only synchronous programs on infinite sequences of data values. On the other hand languages as LEQ or AP2L offer asynchronous interpretations for similar expressions.

UNITY is an attempt to express specifications and solutions, using the same formalism. Program derivations are presented as statement transformations. A program UNITY defines a set of transitions: in an operational point of view it means a possibly infinite sequence of states, according to some computational model, until defining a fix point. Some particular programs in UNITY, the equational ones, define classical recurrence equations as in ALPHA. Properties of programs are also studied in the GAMMA model.

6. Conclusion

The language PEI presents a program as a set of equations, each of them linking two expressions of a single data field. Such a language appears then as a straightforward generalization of recurrence equation languages (ALPHA or CRYSTAL for example), because of its large use of symbols. Indeed as the previous ones, PEI is a descriptive language. But, for any problem and its solutions., these languages express a concrete view of data domains, operators and dependency relations. This expression uses predefined symbols (convex descriptions in ALPHA, operators pre or when in LUSTRE, etc.) associated with a fixed computation model. On the contrary variables in PEI are defined from operators meaningless, according to some predefined computation model.

The language PEI is then an attempt to express problem statements or parallel solutions, and their transformations, using a single formalism and a refinement calculus. Due to this requirement, parallel programming consists in transforming statements, according to some target constraints. Operator properties and an equivalence relation defined in PEI warrent semantical correctness of these transformations. It means that a bijection exists between a problem representation and any solution. So, our concept of program refinement offers a good framework for parallel program design.

References

[1] J.P. Banatre and D. Le Metayer, The Gamma model and its discipline of programming, Sci. Comput. Programming, 15 (1) (1990) 55-79.

[2] K.M. Chandy and J. Misra, Parallel Program Design: A Foundation, (Prentice Hail, Englewood Cliffs, NJ, 1988). [3] Y. Choo and M.C. Chen, A theory of program optimization, TR-608, University of Yale, 1988. [4] M.C. Eglin and J. Julliand, Translating equational systems into communicating processes, Parallel Computing

'91, Washington (1991). [5] J.A.B. Fortes, K.S. Fu and B.W. Wah, Systematic approaches to the design of algorithmically specified systolic

arrays, lnternat. Conf. on Acoustics, Speech and Signal Processing (1985). [6] D. Gelernter, Generative communication in Linda, ACM Trans. on Programming Languages Syst. 7 (1)(1985)

80-112. [7] J. Julliand and G.R. Perrin, Asynchronous functional parallel programs, ICCI'90, in LNCS 468 (Springer,

Berlin, 1990). [8] R.M. Karp, R.E. Miller and S. Winograd, The organization of computations for uniform recurrence equations,

JACM 14 (3) (1967). [9] S.Y. Kung, S.C. Lo and P.S. Lewis, Optimal systolic design for the transitive closure and the shortest path

problem, IEEE Trans. Comput. C-36 (5)(1987).


[I0] C. Mauras, P. Gachet, P. Quinton and Y. Saouter, Alpha du Centaur: An environment for the design of regular algorithms, Internat. Conf. on Supercomputing, Crete (1989).

[II] C. Mongenet, Ph. Clauss and G.R. Perrin, A geometrical coding to compile affine recurrence equations on regular arrays, IPPS'91, Los Angeles (1991).

[12] C. Morgan, Programming from specifications, C.A.R. Hoare Series (Prentice-Hall, Englewood Cliffs, NJ, 1990). [13] J.A. Plaice, S6mantique et compilation de Lustre: Un langage d~claratif synchrone, Thesis INP Grenoble

(France), 1988. [14] P. Quinton and V. Van Dongen, The mapping of linear recurrence equations on regular arrays, J. VI,SI Signal

Processing (1989).

Documents

PEI: A language and its refinement calculus for parallel programming