ELEC692 VLSI Signal Processing Architecture

Lecture 6Array Processor Structure

Introduction• Regular and recursive algorithms are common in DSP

applications.– Repetitive apply identical series of pre-determined

operations– E.g. matrix operations– Regular architectures based on identical processing

elements• Utilizing massive parallel processing and intensive

pipelining– Systems with programmable processors – multiprocessor

systems– Systems with application-specific processors – array

processors• Global clock synchronized array – systolic arrays• Self-timed asynchronous data transfer – wavefront arrays

• Issues – How is the array processor design dependent on the

algorithm?– How is the algorithm best implemented in the array

processors?

What is a systolic array• Computational networks with distributed data

storage and distributed arrangement of processing elements, so that a high number of operations are carried out simultaneously.

• Multiple PEs to maximize processing per memory access

PE

memory10ns

Conventional: 100MOPS (Million Operations Per Second)

PE

memory10ns

Array processor: 400MOPS

PEPE PE

Characteristics of Array processors

• Parallelism– Both data operation and data transfers

• Locality– Connection exists only to directly neighbouring PEs

• Regularity and modularity– Both computation and communications structures

• Processing elements can be simple (e.g. a single addition/multiplication) or complex

• Why call systolic array?– Analogy to the circulatory system with its two phases,

systolic and diastole– System that are entirely pipelined and have periodic

computation and data transfer cycles– Synchronous clocking and control signals

Array structure examples

PE PE PE PE

1D array

PE PE PE

PE PE PE

PE PE PE

2D array

PE PE PE

PE PE PE

PE PE PE

PE PE PE

PE PE PE

PE PE PE

PE PE PE

PE PE PE

PE PE PE

3D array

Drawback of Array processing

• Not all algorithms can be mapped and implemented using systolic array architecture– Only fixed operations and operands to be processed

are fixed prior to run-time– Adaptive algorithms in which the particular operations

and operands depend on the data to be processed, cannot be used.

• Cost in hardware and area is high• Cost in latency

Data dependency• Express the algorithm in inherent dependency

– Single-assignment code and local data dependency– E.g. Matrix multiplication

njibacwhereABCn

kijijij

,11

FOR i:=1 to n Do For j:=1 to n Do BEGIN c(i,j) :=0 For k:=1 to n Do c(i,j) := c(i,j)+a(i,k)*b(k,j); END;

FOR i:=1 to n Do For j:=1 to n Do BEGIN c(i,j,0) :=0 For k:=1 to n Do BEGIN if i=1, then b(1,j,k):=b_in(k,j)

else b(i,j,k):=b(i-1,j,k) if j=1, then a(i,1,k):=a_in(i,k)

else a(i,j,k):=a(i,j-1,k) c(i,j,k) := c(i,j,k-1)+a(i,jk)*b(i,j,k); END; c_out(I,j):=c(i,j,n); END;

Data dependency graph (DG)

• A graph specifies the data dependencies in an algorithm

• E.g. DG for the matrix multiplication

DG of matrix-vector multiplication

nibacwhereAbCn

kkiki

11

Systolic Array Fundamentals

• Systolic architecture are designed by using linear mapping techniques on regular dependency graph (DG).

• Regular Dependency Graph: the presence of an edge in a certain direction at any node in the DG represents presence of an edge in the same direction at all nodes in the DG.

• DG corresponds to space representation no time instance is assigned to any computation

• Systolic architectures have a space-time representation where each node is mapped to a certain PE and is scheduled at a particular time instance.

• Systolic design methodology maps an N-dimensional DG to a lower dimensional systolic architecture

Example of DG

• Space representation for a FIR filter– y(n) = w0x(n)+w1x(n-1)+w2x(n-2)

10 32 54

y(0) y(1) y(2) y(3) y(4) y(5)

w2

w1

w0

x(0) x(1) x(2) x3) x(4) x(5)

j’=jProcessor axis

(0,1)T

(1,0)T

(1,-1)T

Time axis

Linear Mapping Methods

• Design an application-specific array processor architecture for a given algorithm– Satisfy the demands

regarding computational performance and throughput

– Minimize hardware cost– Regular structure for

VLSI implementation

algorithm

Dependency graph

Signal flow graph

architecture

Assignment of the operations to processors and points in time (assignment and scheduling)

Estimation of # of PE• # of PE nPE is generally smaller than the # of nodes in the

dependency graph nDG.

• Important to know both since they specify how many nodes of the DG must be mapped to a PE.

• The processing time of a PE = TPE (including transfer time of register).

• Within TPE, each PE carries out a total of nOP/PE operations.

• The computational rate of the processor array is then given by

• Desired throughput = RT,target and the # of operations per sample nOP/SAMPLE, so we have

PE

PEOPPEARRAYC T

nnR /

,

PEettTPEOP

SAMPLEOPPE TR

n

nn arg,

/

/

Estimation of # of PE

• # of PE can be furthered reduced through pipelining within the PEs. Given np as the # of pipeline stages along the datapath of the PE, then the new T’PE is

• Example– Samples of a colour TV signal (27MHz sampling rate) are to be

transformatted into 8X8 blocks, and each of the blocks is to be multiplied by an 8X8 matrix

– Sampling and matrix coefficents are 8-bits wide– Results of the accumulated product is 19-bit– A PE contains 1 MUL and 1ADD and a register at its output

REGP

REGPEPE T

n

TTT

'

Example of Estimation of # of PE

LccFADREG

LccFADARRAYMA

REGARRAYMAPE

SAMPLEOP

PEOP

TT

TnT

TTT

n

ADDMULn

16

1627)33(

82

),(2

',,

',,,

,

/

/

(2*83 operations for 82 samples)

With intensive pipelining

)9.2'(6.0'

)4.22(8.4

42

'

',,,,

,

nsTn

nsTn

TTT

TTT

PEPE

PEPE

LssFADANDDCELLMA

REGCELLMAPE

Assume L=50ps)

Some definitions

• Projection vector (also called iteration vector)– Two nodes that are displaced by d or multiples of d are

executed by the same processor.

• Scheduling vector: sT=(s1,s2)– Any node with index I would be executed at time STI

• Processor space vector pT=(p1,p2)– Any node with index IT=(i,j) would be exeucted by processor

• Hardware Utilization Efficiency, HUE = 1/|sTd|– This is because two tasks executed by the same processor are

spaced |sTd| time units apart.

2

1

d

dd

j

ippIpT ),( 21

Systolic Array Design Methodology• Systolic architectures are designed by

selecting different project, processor space and scheduling vectors,

• Feasibility constraints– Processor space vector and projection vector must

be orthogonal to each other.• Points A and B differ by the projection vector, i.e. IA-IB is

same as d, then they must be executed by the same processor, i.e. PTIA = PTIB and

• If A and B are mapped to the same processor, then they cannot be executed at the same time, i.e.

• Edge mapping: If an edge e exists in the space representation or DG, then an edge pTe is introduced in the systolic array with sTe delays.

dpIIp TBA

T 0)(

0..., dseiIsIs TB

TA

T

Array Architecture Design

• Step 1: mapping algorithm to DG– Based on the space-time indices in the recursive algorithm– Shift-Invariance (Homogeneity) of DG– Localization of DG: broadcast vs. transmitted data

• Step 2: mapping DG to SFG– Processor assignment: a projection method may be

applied (project vector d)– Scheduling: a permissible linear schedule may be applied

(Schedule vector s)• Preserve the inter-dependence• Nodes on an equitemporal hyperplane should not be

projected to the same PE• Step 3: mapping an SFG onto an array processor

Example: FIR Filter

x(0) x(1) x(2) x(3) x(4) x(5) x(6)

h(0)

h(1)

h(2)

h(3)

h(4)

y(0)

y(1)

y(2)

y(3)

y(4) y(5) y(6)

k

n

d

s

Equitemporal hyperplanes

D

D

D

D

D

2D

2D

2D

2D

2D

D

D

D

D

D

x(0)x(1)x(2)

y(0)y(1)y(2)..

Space time transformation

• Space representation or DG can be transformed to a space-time representation by interpreting one of the spatial dimensions as temporal dimension.

• For a two-dimensional (2D) DG, the general transformation is described by i’=t=0, j’=pTI and t’=sTI or equivalently

t

j

i

s

p

t

j

i

T

t

j

i

T

T

0

0

100

'

'

'

• In the space-time representation, the j’ axis represents the processor axis and t’ represents the scheduling time instance.

FIR Systolic Array (Design B1)• B1 design is derived by selecting projection vector,

processor vector and scheduling vector as follows:

• Any node with index IT=(I,j) is mapped to processor

• Therefore all nodes on a horizontal line are mapped to the same processor

• Any node with index IT=(i,j) is executed at time• Since then

• Edge mapping

).10(),10(,1

0

TT spd

jj

iIpT

10

ij

iIsT

01

10

101

dsT 1

||

1

dsHUE

T

eT pTe sTe

Weight (wt(1 0)) 0 1

Input ( i/p(0 1)) 1 0

Result (1 -1) -1 1

B1 design

D

D

input

D

D

D

D

result

Block diagram

Processoraxis

D

D0

D

D1 D2

Input x(n)

Result0

Processor 1Low-level implementation

Time axis

j’=jProcessor axis x(0) x(1) x(2) x(3) x(4)

y(0) y(1) y(2) y(3) y(4)

0

1

2

0 1 2 3 4

Space-timerepresentationof B1 design

weight

FIR Systolic Array (Design B2)• B2 design is derived by selecting projection vector,

processor vector and scheduling vector as follows:

• Any node with index IT=(I,j) is mapped to processor

• Any node with index IT=(i,j) is executed at time• Since then

• Edge mapping

).01(),11(,1

1

TT spd

j

iIpT 11

ij

iIsT

01

11

101

dsT 1||

1

dsHUE

T

eT pTe sTe

Weight (wt(1 0)) 1 1

Input ( i/p(0 1)) 1 0

Result (1 -1) 0 1

Weights move instead of the results as in B1.Inputs are broadcast.

B2 design

D

D

input

D

D

D

Dresult

Block diagram

Processoraxis

D

0 1 2

Input x(n) (…x(3),x(2),x(1),x(0)

Low-level implementation

Time axis t’=I

j’=i+jProcessor axis

x(0) x(1) x(2) x(3) x(4)

y(0)

y(1)

y(2)

y(3)0

12

0 1 2 3 4

Space-time representation of B2 design

weight

D

0

D

0

D

0

D D

Applying space-time transformation

ij

ist

jij

ipj

T

T

'

'

FIR Systolic Array (Design F)• F design is derived by selecting projection vector, processor

vector and scheduling vector as follows:

• Since then

• Edge mapping

).11(),10(,0

1

TT spd

10

111

dsT 1

||

1

dsHUE

T

eT pTe sTe

Weight (wt(1 0)) 0 1

Input ( i/p(0 1)) 1 1

Result (1 -1) -1 0

Weights fixed in space, input vector moves from left to right with 1 delay;Output moves from right to left with no delay elements.

F design

D

input

D

D

D

Dweight

Block diagram

Processoraxis

Low-level implementation

Time axis t’=i+j

j’=jProcessor axis

x(0)x(1)x(2)

y(0) y(1) y(2) y(3)

0

1

2

0 1 2 3 4

Space-time representation of B2 design

result

Applying space-time transformation

jij

ist

jj

ipj

T

T

'

'

D

D

D0

D

D1 D2

Result

Input x(n)

0

Selection of Scheduling Vector

• Finding feasible scheduling vectors using scheduling inequalities.

• Based on the selected scheduling vector ST, the projection vector d and the processor space vector pT can be found by

• Scheduling inequalities:– Consider the dependency relation XY:

– We have

00 dpanddS TT

y

yy

x

xx j

iIY

j

iIX :: Ix and Iy are the indices of node X

and Y.

xxy TSS Where Tx is the time to compute node X and Sx, Sy are the scheduling times for nodes X and Y

Scheduling Equations

• Linear Scheduling

• Affine scheduling ( A transformation followed by a translation

and we have

• Defining edge from XY as ex-y = Iy-Ix, then scheduling inequality for an edge is:

y

yy

Ty

x

xx

Tx

j

issIsS

j

issIsS

)(

)(

21

21

yy

yyy

Ty

xx

xxx

Tx

j

issIsS

j

issIsS

)(

)(

21

21

xxxT

yyT TIsIs

xxyyxT Tes

Scheduling vector sT can be obtained by solving these inequalities

Scheduling Inequalities

• Two steps for select the scheduling vectors:– Capture all the fundamental edges. The

reduced dependence graph (RDG) is used to capture the fundamental edges and the regular iterative algorithm (RIA) description of the corresponding problem is used to construct RDGs

– Construct the scheduling inequalities and solve them for feasible sT.

Regular Iteration Algorithm (RIA)

• Standard input RIA form– If the index of the inputs are the same for all equations.

• Standard output RIA form– If the output indices, i.e. indices on the left side, are the

same

• E.g. FIR filter, RIA description is– W(i+1,j) = W(i,j)– X(i,j+1) = X(i,j)– Y(i+1,j-1)=Y(i,j) + W(i+1,j-1)*X(i+1,j-1)

• Cannot expressed as standard input RIA form, but can be expressed as standard output RIA form– W(i,j) = W(i-1,j)– X(i,j) = X(i,j-1)– Y(i,j)=Y(i-1,j+1) + W(i,j)*X(i,j)

w x

y

(1,0)(0,1)

(1,-1)

(0,0) (0,0) Reduced DG

Example

xxyyxT Tes

2

1

s

sswhere

125,1

1:

0,0

0:

1,0

1:

1,1

0:

0,0

0:

21

1

2

yy

xy

ww

xx

wy

sseYY

eYX

seWW

seXX

eYW

Tmult = 5, Tadd = 2, Tcom = 1w x

y

(1,0)(0,1)

(1,-1)

(0,0) (0,0)

We have 5 edges in the RDG and so

For linear scheduling, x= y=w=0

We have .8,1,1 2121 ssssOne of the solution is s2=1,s1=8+1=9 and sT=(9,1)

Select d=(1,-1) such that sTd=8 0 and select pT = (1,1) so that pTd = 0. HUE = 1/8

Edge mapping = eT pTe sTe

Weight (wt(1 0)) 1 9

Input ( i/p(0 1)) 1 1

Result (1 -1) 0 8

8D 8D 8D

D D

9D 9D

X

W

Matrix-Matrix Multiplication• DG is a three-dimensional (3D)

space representation• Linear projection is used to

design 2D systolic arrays• Given 2 matrices A and B (e.g.

dimension of 2X2) and C= AB, 2222122122

2122112121

2212121112

2112111111

2221

1211

2221

1211

2221

1211

babac

babac

babac

babac

bb

bb

aa

aa

cc

cc

)1,,()1,,(),,()1,,(

),,(),,1(

),,(),1,(

kjibkjiakjickjic

kjibkjib

kjiakjia

),,(),,()1,,(),,(

),,1(),,(

),1,(),,(

kjibkjiakjickjic

kjibkjib

kjiakjia

Standard output RIA form

Matrix-Matrix Multiplication Example

),,(),,()1,,(),,(

),,1(),,(

),1,(),,(

kjibkjiakjickjic

kjibkjib

kjiakjia

0,

0

0

0

:

0,

0

0

0

:1,

1

0

0

:

0,

0

0

1

:0,

0

1

0

:

3

12

bc

ac

ecb

ecasecc

sebbseaa

a b

c

(0,1,0)(1,0,0)

(0,0,1)

(0,0,0) (0,0,0)

RDG of the matrix multi. example

Let Tmult-add=1 and Tcom = 0, we have

For linear scheduling, a= b=c=0

Solution 1:

1

1

0

0

)111(

0

0

1

0

0

010

001

010

001,

1

0

0

),111(

ds

dp

pds

T

T

TT

HUE = 1

Solution 1

Edge mapping =

eT pTe sTe

a(0,1,0) (0,1) 1

b(1,0,0) (1,0) 1

C(0,0,1) (0,0) 1

cD

cD

cD

cD

cD

cD

cD

cD

cD

a a a

b

b

b

D D

D D

D D

i

j

D

D

D

D

D

D

2-dimensional systolic arrayBy S. Y. Kung

Solution 2

11

1

1

1

)111(

0

0

1

1

1

110

101

110

101,

1

1

1

),111(

HUEds

dp

pds

T

T

TT

Edge mapping =

eT pTe sTe

a(0,1,0) (0,1) 1

b(1,0,0) (1,0) 1

C(0,0,1) (1,1) 1

c c c

Dc D D

Dc D D

a a a

b

b

b

D D

D

D D

D

D D

D D

i

j