Dependence Analysis

Kathy Yelick

Bebop group meeting, 8/3/01

Outline

•Motivation•What the compiler does:

–What is data dependence?–How is it represented?–How is it used?

Motivation: Optimization•Many compiler optimizations are based

on the idea of either:– Reordering statements (or smaller units)– Executing them in parallel

•In the paper on transforming loops to recursive functions, they are reordering the iterations of matrix multiply.

•Goal: do this without changing the semantics of the program.

References

•Chau-Wen Tseng’s lecture notes:–www.cs.umd.edu/class/spring1999/cmsc7

32/

•Gao Guang’s lecture note (U.Del.)•Others

Definition and Notation

•Data dependence:–Given two program statements a and

b, b depends on a if:• b follows a (roughly)•they share a memory location and •one of them writes to it.

–Written: b a–Example:

•a: x = y + 1;•b: z = x * 3;

Because b depends on a, the two statements cannot be reordered, nor can they be run in parallel

Classification•A dependence, a b, is one of the following:– true of flow dependence:

•a writes a location that b later reads•(read-after write or RAW)

–anti-dependence•a reads a location that b later writes•(write-after-read or WAR)

–output dependence•a writes a location that b later writes•(write-after-write or WAW)

More Dependences•The following is also a dependence

– An input dependence a b occurs when• a reads a location that b later reads• (read-after-read or RAR)

•But we usually are not interested in these, because they don’t constraint order or parallelism

•Examples:true anti output input

a = = a a = = a

= a a = a = = a

Example

Example:

S1: A = 0S2: B = AS3: C = A +

DS4: D = 2

S1

S2

S3

S4

S2 S1 :flow depS3 S1 :flow depS4 S3 :flow dep

S1 0 S3 : output-depS2 -1 S3 : anti-dep

S1

S2

S3

S4

How to Compute Dependences?

• Data dependence relations can be found by comparing the IN and OUT sets of each node.

– The IN set of a statement, IN(S), is the set of variables (or, more precisely, the set of memory locations, usually referred to by variable names) that may be used (or read) by this statement.

– The OUT set of a statement, OUT(S), is the set of memory locations that may be modified (written or stored) by the statement.

• Computing and representing the memory locations can be very difficult if the program contains aliases.

• Fortunately, the IN/OUT sets can be computed conservatively• Assuming that S2 is reachable from S1, the following shows

how to intersect these sets to test for data dependence:

OUT(S1) IN(S2) S1 S2 flow dependence IN(S1) OUT(S2) S1 -1 S2 anti-dependence OUT(S1) OUT(S2) S1 0 S2 output dependence

Computing Dependences in Practice

Dependences in Loops•A loop-independent dependence exists

even if there were no loop for (j = 0; j < 100; j++) { a[j] = = a[j] }

•A loop-carried dependence is induced by the iterations of a loop. The source and sink occur on different iterations. for (j = 0; j < 100; j++) { a[j] = = a[j-1] }

Data Dependences in LoopsFind the dependence relations due to the array X in the program below:

(1) for I = 2 to 9 do (2) X[I] = Y[I] + Z[I] (3) A[I] = X[I-1] + 1 (4) end for

Approach: In a simple loop, we can unroll the loop and see which statement instances depend on which others (each array element is a

variable):

(2) X[2]=Y[2]+Z[2] X[3] =Y[3]+Z[3] X[4]=Y[4]+Z[4](3) A[2]=X[1]+1 A[3] =X[2]+1 A[4]=X[3]+1

I = 2 I = 3 I = 4

In the example, there is a loop-carried, lexically forward flow dependence relation. (The (1) annotation in the figure below will be explained shortly.)

Data Dependences in Loops

S2

S3

(1)

- Loop-carried vs loop-independent- Lexical-forward vs lexical backward

Iteration Space•Example

do I = 1, 5 do J = I, 6 . . . enddoenddo

•Written out in lexicographic order the iteration space is:– (1,1), (1,2),…, (1,6),(2,2),(2,3),…

•Equivalent to sequential execution order

J

I

Basic Concepts• Let Rn be the set of all real n-vectors, (n >1)

• A lexicographic order <u on these vectors is a relation:

i <u j on vectors i = { i1 … in }

j = { j1 … jn }

iff i1 = j1, j1 = j2 … and iu< ju

• The leading element of a vector is the first non-zero element

• A negative vector has: leading element < 0• A positive vector has: leading element > 0

• Given 2 n-vectors i,j i = (i1, … in)

j = (j1, … jn)

• Their distance vector = (j1 - i1, j2 - i2, …)– Represents the number of iterations between

accesses to the same location

• Their direction vectorS = (sig (j1 - i1), sig(j2 - i2), …)

– Represents the direction in iteration space

• Given distance vector ==> one can derive direction vector but not vise versa.

Distance/Direction Vectors

Distance/Direction Vectors

• It is often convenient to deal with in-completely specified direction vectors

Example 1:{(0, 0, 0, 1), (0, -1, 0, 1), (0, 0, 1, 1), (0, -1, 1, 1)}

==> {(0, 0, 0, 1)}

Example 2: {(0, -1, 0, -1), (0, 0, 0, -1), (0, 1, 0, -1)}

==> {(0, *, 0, -1)}

Distance/Direction Vectors

• Let i, j denote two vectors in Rn and s their direction vector. Then i < j iff s has one of the following n forms:

(1, *, *, …, *)(0, 1, *, …, *)(0, 0, 1, *, …, *)

(0, 0, …, 0, 1).

More precisely, i <u j for a u in 1 u n, iff s has the form with a leading 1 after (u - 1) zeros, I.e., s is positive.

• Notation

(0, 1, -1) (=, >, <)

do i = 3,100S: A(2i) = B(i) + 2

T: C(i) = D(i) + 2A(2i +1) + A(2i - 4) + A(i)

1. A(2i), D(i)

2. A(2i), A(2i + 1)

3. A(2i), A(2i - 4)

An Example

no dependence

no dependence

?

• Note: the direction is from S to T (i2 > i1)

2i1 = 2i2 - 4, so i2 - i1 = 2

i.e.

a flow dependence is caused for example:

all iteration pairs: (i2 = i1 + 2, 3 i 98)

for each pair: direction vector = (1) distance vector = (2)

i1 = 3i2 = 5

Constant

Example: A(2i) and A (2i - 4)

• For A(2i), A(i)this causes a flow dependence of T on S, the set of associated iteration pairs is

{(i, j) | j = 2i, 3 i 50)}

for i, j in this setdirection vector = (1)distance vector = 2i - i = i

Summary:T is flow-dependent on Swith direction vector = (1)

distance vector between (3) to (50)

Example: A(2i) and A (2i - 4)

Testing for Parallel Loops•A dependence D = (d1,…,dk) is carried at

level i, if di is the first nonzero element of the distance/direction vector

•A loop li is parallel if there does not exist a dependence Dj carried at level i.

•If the loop is parallel, it may also be reordered.