IMPACT Second Generation EPIC Architecture Wen-mei Hwu IMPACT Second Generation EPIC Architecture Wen-mei Hwu Department of Electrical and Computer Engineering

IMPACT Second Generation EPIC ArchitectureWen-mei Hwu IMPACTIMPACT

IMPACT Second Generation EPIC Architecture

Wen-mei Hwu

Department of Electrical and Computer Engineering

Coordinated Science Laboratory

University of Illinois at Urbana-Champaign

IMPACT Compiler Grouphttp://www.crhc.uiuc.edu/IMPACT/


Vision: Bridging the Gap Between Programs and Hardware

if (x>=0)

if (x==1 ||

x==2 ||

x==3)

m=f(x);

else

m=g(x);

f g

>== = =1 32 0

1

m

+

x

0

enable

x>=0

x!=3

x!=1

x!=2

m=g(x) m=f(x)

TF

T

T

T

F

F

F


Can we get the best of both worlds?

• Hardware– highly speculative– parallel in nature– efficient logic

manipulation– special purpose– area efficient– energy efficient

• Programming– conservative semantics– sequential in nature– awkward logic

manipulation– easily retargeted– area inefficient– energy inefficient


EPIC Design Objectives• To define a programmable architecture

model that allows compiled programs to approach special hardware design in– logic manipulation capability– speculation and parallelism– chip area efficiency– energy efficiency


EPIC - the IMPACT Perspective• IMPACT work done since 1987 to lay

foundation for EPIC architectures– Intel/HP IA-64, Motorola/Lucent StarCore

• Key Technologies– control speculation [ISCA-91] [ASPLOS-92] [MICRO-96]

– data (dependence) speculation [ICS-92] [ASPLOS-94]

– predicated execution [MICRO-92][ISCA-95] [MICRO-97]

– integrated architecture and inline recovery [ISCA-98]

– logic minimization approach to predication [ISCA-99]

– implementation neutral predication architecture [TBD]


Result of Applying EPIC Techniques3.10

1.00

1.20

1.40

1.60

1.80

2.00

2.20

2.4000

8.es

pres

so

023.

eqnt

ott

072.

sc

085.

cc1

124.

m88

ksim

129.

com

pres

s

130.

li

132.

ijpe

g

147.

vort

ex

cccp

cmp

eqn

grep lex

wc

yacc

Spe

edup

ove

r ba

se

Predication onlyControl speculation onlyData speculation onlyPredication and speculation combined


Focus of this Talk

• Inline recovery of speculative exceptions in the IMPACT second generation EPIC architecture– minimal explicit checks

– no recovery blocks

– minimal additional hardware requirement

• Other aspects NOT covered: new predication architecture, new integrated architecture for speculation and predication


Control Speculation

• Executing an instruction before knowing that its execution is required

• Moving an instruction above a branch or a predicate defining instruction– Removes control dependences to increase ILP

• Code seen by hardware is changed!– Must ensure that execution result is unaffected

by such movement


Control Speculation Example

• A: r6 = r4+1• B: If (r9==0) goto L1• C: r1 = MEM(r2+0)• D: r3 = MEM(r2+4)• E: r4 = r3+1• F: r5 = r1+1• G: MEM(r2+r4) = r4

• C: (s) r1 = MEM(r2+0)• D: (s) r3 = MEM(r2+4)• E: (s) r4 = r3+1• F: (s) r5 = r1+1• A: r6 = r6+1• B: if (r9==0) goto L1• G: MEM(r2+4) = r4


Speculative Exceptions and Events

• C and D– speculative instructions

that can potentially cause exceptions and events

• program exceptions– bus errors, segmentation

faults, divide by 0, …

• transparent events– page faults, TLB misses,

cache misses

• C: (s) r1 = MEM(r2+0)• D: (s) r3 = MEM(r2+4)• E: (s) r4 = r3+1• F: (s) r5 = r1+1• A: r6 = r6+1• B: if (r9==0) goto L1• G: MEM(r2+4) = r4


Spurious Data Cache Misses and Exceptions

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%0

08

.esp

ress

o

02

3.e

qnto

tt

07

2.s

c

08

5.c

c1

12

4.m

88

ksi

m

12

9.c

om

pre

ss

13

0.l

i

13

2.i

jpeg

14

7.v

ort

ex

cccp

cmp

eqn

gre

p

lex

wc

yacc

Per

cent

redu

ctio

n (

%)

Cache missesExceptions

• Spurious cache misses, TLB misses, and page faults are frequent in speculated code. Failing to suppress them can have a detrimental effect on performance.


Sentinel Speculation• Design Objective

– Ignore exceptions and events caused by speculative instructions whose execution proves to be unnecessary.

– Support accurate reporting and recovery from exceptions generated by speculative instructions whose execution is confirmed.

– Provide the option to handle speculative transparent events after the need for handling them is confirmed.

– Minimize the extra hardware and instructions incurred


Sentinels and Protected Instructions

• A speculated excepting instruction I is protected if there is a non-speculative instruction in I's home block which directly or indirectly uses the result of I

• The non-speculative use instruction is defined as the sentinel for I

• All speculative excepting instructions must be protected to guarantee exception recovery and delayed event handling


Sentinel Speculation Example

• G is the sentinel for D– G indirectly used D’s result

through E

• H is the sentinel for C– H indirectly uses C’s result

through F

– H is explicitly created to make C protected

• C: (s) r1 = MEM(r2+0)• D: (s) r3 = MEM(r2+4)• E: (s) r4 = r3+1• F: (s) r5 = r1+1• A: r6 = r6+1• B: if (r9==0) goto L1• H: use r5• G: MEM(r2+4) = r4


IMPACT EPIC ArchitectureRegister File

Value/PC E-Tag R-Tag

Memory Conflict Buffer

Register Tag and Attribute

S

Instructions

DS

T/F E-Tag R-Tag

Predicate Register File

LOAD Pred

S

DS CHECK Pred

PredOPERATION

T/F E-Tag R-Tag pR


Speculation Example

• Speculative (affected by exception)

• speculative (not affected)• Non-speculative (not

affected)• branch• check (non-speculative

use) that detects exception

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• E: (s) r4 = r3+1

• F: (s) r5 = r1+1

• A: r6 = r6+1

• B: if (r9==0) goto L1• H: check r5

• G: MEM(r2+4) = r4


Speculative Instruction Execution

• If src(I).E-Tag = 0 • I does not cause an exception, normal execution

• I causes an exception

– dest(I).E-Tag = 1

– dest(I).data = pc of I

• if src(I).E-Tag = 1 (exception propagation) • dest(I).E-Tag = 1,

• dest(I).data = src(I).data


Speculation Example

• D causes a bus error– r3.E-Tag = 1

– r3.data = pc of D

• E propagates exception– r4.E-Tag = 1

– r4.data = r3.data = pc of D

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• E: (s) r4 = r3+1

• F: (s) r5 = r1+1

• A: r6 = r6+1


• G: MEM(r2+4) = r4


Non-Speculative Instruction Execution

• If src(I).E-Tag = 0 • I does not cause an exception - normal execution

• I causes an exception - I reported as source of exception

• If src(I).E-Tag = 1• (report exception for speculative instruction) • processor enters recovery mode

• src(I).data is PC of exception


Speculation Example

• H executes, not affected– r5.E-Tag == 0

– r5.data = result of F

• G initiates recovery– r4.E-Tag == 1

– r4.data ==pc of D

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• E: (s) r4 = r3+1

• F: (s) r5 = r1+1

• A: r6 = r6+1


• G: MEM(r2+4) = r4


Inline Recovery Model• Processor enters recovery mode, set pR

– PC in source register used as recovery PC– The speculative instruction at recovery PC is

executed non-speculatively.– Exception processing is performed.– If exception is non-terminating, the result is

stored into destination register, set R-Tag.– Instructions with R-Tag set in source registers

are executed, set R-Tag in destination register


Inline Recovery Example

• G triggers recovery with pc of D

• D re-executed non-speculatively– r3.R-Tag = 1

– r3.data = result of D

• E re-executed– r4.R-Tag = 1

– r4.data = result of E

• F, A skipped over

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• E: (s) r4 = r3+1

• F: (s) r5 = r1+1

• A: r6 = r6+1


• G: MEM(r2+4) = r4


Inline Recovery Model (cont.)– Non-speculative instructions not

repeated.• Stores, self-incrementing loads and stores, etc.

are safe.• Same effect is achieved by recovery blocks.• Source registers of non-speculative instructions

do not need to be preserved.


Inline Recovery Model (cont.)

– Branches and predicate defines repeated• to reproduce original control flow

• input condition must be preserved• not to be speculated

– Recovery mode is turned off when reaching check with set source R-Tag.


Inline Recovery Example

• B repeated• H skipped over• G re-executed, turns off

recovery mode, R-Tag for all registers = 0

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• E: (s) r4 = r3+1

• F: (s) r5 = r1+1

• A: r6 = r6+1


• G: MEM(r2+4) = r4


Why both E-Tag and R-Tag

• Additional exceptions may occur to speculative instructions during recovery– R-Tag designates new

value generated in recovery mode, execute

– E-tag designates exceptions, propagate pc in source register

• C: (s) r1 = MEM(r2+0)

• D: (s) r3 = MEM(r2+4)

• J: (s) r4 = MEM(r3+0)

• F: (s) r5 = r1+1

• A: r6 = r6+1

• B: if (r9==0) goto L1• H: check r5• G: MEM(r2+4) = r4


Recovery Block - Code Size

1

1.05

1.1

1.15

1.2

1.25

1.3

1.35

1.4

1.45

00

8.e

spre

sso

02

3.e

qnto

tt

07

2.s

c

08

5.c

c1

09

9.g

o

12

4.m

88

ksim

12

9.c

om

pres

s

13

0.l

i

13

2.i

jpeg

14

7.v

ort

ex

cccp

cmp

eqn

grep

lex

wc

yacc

Co

de E

xpan

sio

n


Instruction Cache Miss Comparison (32k, direct mapped)

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%0

08

.esp

ress

o

02

3.e

qnto

tt

07

2.s

c

08

5.c

c1

12

4.m

88

ksim

12

9.c

om

pres

s

13

0.l

i

13

2.i

jpeg

14

7.v

ort

ex

cccp

cmp

eqn

grep

lex

wc

yacc

Per

cent

red

uct

ion

(%)


Implementation Considerations• R-Tag, E-Tag, PC value can all be confined in

the retirement stage of the processor– full register file manipulated at the retirement stage– no need to introduce PC values into main data path– increased detection latency– no impact on size and clock rate of the main data path

• Prediction scheme needed for selective handling of transparent events. Traditional branch prediction schemes could be used but lower cost approaches exists [ISCA-99].


Conclusion

• Inline recovery sets foundation for selective handling of exceptions and transparent events caused by speculative instructions

• Additional architectural cost: one bit per register file entry

Documents

IMPACT Second Generation EPIC Architecture Wen-mei Hwu IMPACT Second Generation EPIC Architecture Wen-mei Hwu Department of Electrical and Computer Engineering