18-447 Computer Architecturecourse.ece.cmu.edu/~ece447/s15/lib/exe/fetch.php?...Microsequencer BEN 2 x Control Store 6 IR[15:11] 6 (J, COND, IRD) 9 26 35 35 Fi g u r e C. 4 : Th e

18-447

Computer Architecture

Lecture 7: Pipelining

Prof. Onur Mutlu

Carnegie Mellon University

Spring 2015, 1/30/2015

Agenda for Today & Next Few Lectures

Single-cycle Microarchitectures

Multi-cycle and Microprogrammed Microarchitectures

Pipelining

Issues in Pipelining: Control & Data Dependence Handling, State Maintenance and Recovery, …

Out-of-Order Execution

Issues in OoO Execution: Load-Store Handling, …

2

Recap of Last Lecture

Multi-cycle and Microprogrammed Microarchitectures

Benefits vs. Design Principles

When to Generate Control Signals

Microprogrammed Control: uInstruction, uSequencer, Control Store

LC-3b State Machine, Datapath, Control Structure

An Exercise in Microprogramming

Variable Latency Memory, Alignment, Memory Mapped I/O, …

Microprogramming

Power of abstraction (for the HW designer)

Advantages of uProgrammed Control

Update of Machine Behavior

3

Review: A Simple LC-3b Control and Datapath

4

C.4. THE CONTROL STRUCTURE 9

Microinstruction

R

Microsequencer

BEN

x2

Control Store

6

IR[15:11]

6

(J, COND, IRD)

269

35

35

Figure C.4: The control structure of a microprogrammed implementation, overall block

diagram

on the LC-3b instruction being executed during the current instruction cycle. This state

carries out the DECODE phase of the instruction cycle. If the IRD control signal in the

microinstruction corresponding to state 32 is 1, the output MUX of the microsequencer

(Figure C.5) will take its source from the six bits formed by 00 concatenated with the

four opcode bits IR[15:12]. Since IR[15:12] specifies the opcode of the current LC-

3b instruction being processed, the next address of the control store will be one of 16

addresses, corresponding to the 14 opcodes plus the two unused opcodes, IR[15:12] =

1010 and 1011. That is, each of the 16 next states is the first state to be carried out

after the instruction has been decoded in state 32. For example, if the instruction being

processed is ADD, the address of the next state is state 1, whose microinstruction is

stored at location 000001. Recall that IR[15:12] for ADD is 0001.

If, somehow, the instruction inadvertently contained IR[15:12] = 1010 or 1011, the

Simple Design

of the Control Structure

A Simple Datapath

Can Become

Very Powerful

Review: The Power of Abstraction

The concept of a control store of microinstructions enables the hardware designer with a new abstraction: microprogramming

The designer can translate any desired operation to a sequence of microinstructions

All the designer needs to provide is

The sequence of microinstructions needed to implement the desired operation

The ability for the control logic to correctly sequence through the microinstructions

Any additional datapath elements and control signals needed (no need if the operation can be “translated” into existing control signals)

7

Review: Advantages of Microprogrammed Control

Allows a very simple design to do powerful computation by controlling the datapath (using a sequencer)

High-level ISA translated into microcode (sequence of u-instructions)

Microcode (u-code) enables a minimal datapath to emulate an ISA

Microinstructions can be thought of as a user-invisible ISA (u-ISA)

Enables easy extensibility of the ISA

Can support a new instruction by changing the microcode

Can support complex instructions as a sequence of simple microinstructions

Enables update of machine behavior

A buggy implementation of an instruction can be fixed by changing the microcode in the field

8

Wrap Up Microprogrammed Control

Horizontal vs. Vertical Microcode

Nanocode vs. Microcode vs. Millicode

Microprogrammed MIPS: An Example

9

Horizontal Microcode

A single control store provides the control signals

10

Microprogram counter

Address select logic

Adder

1

Input

Datapath

control

outputs

Microcode

storage

Inputs from instruction

register opcode field

Outputs

Sequencing

control

[Based on original figure from P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]

ALUSrcA

IorD

IRWrite

PCWrite

PCWriteCond

….

n-bit mPC input

k-b

it c

on

tro

l sig

nal

ou

tpu

t

Control Store: 2n k bit (not including sequencing)

MIPS design From P&H, Appendix D

Vertical Microcode Two-level control store: the first specifies abstract operations

11

Microprogram counter

Address select logic

Adder

1

Input

Datapath

control

outputs

Microcode

storage

Inputs from instruction

register opcode field

Outputs

Sequencing

control

“PC PC+4”

“PC ALUOut”

“PC PC[ 31:28 ],IR[ 25:0 ],2’b00”

“IR MEM[ PC ]”

“A RF[ IR[ 25:21 ] ]”

“B RF[ IR[ 20:16 ] ]”

…………. …….

ROM

ALU

SrcA

IorD

IRW

rite

PC

Write

PC

WriteC

on

d

….

[Based on original figure from P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]

If done right (i.e., m<<n, and m<<k), two ROMs together (2nm+2mk bit) should be smaller than horizontal microcode ROM (2nk bit)

m-bit input

k-bit control signal output

n-bit mPC input

1-bit signal means do this RT (or combination of RTs)

Nanocode and Millicode

Nanocode: a level below traditional microcode

microprogrammed control for sub-systems (e.g., a complicated floating-point module) that acts as a slave in a microcontrolled datapath

Millicode: a level above traditional microcode

ISA-level subroutines that can be called by the microcontroller to handle complicated operations and system functions

E.g., Heller and Farrell, “Millicode in an IBM zSeries processor,” IBM JR&D, May/Jul 2004.

In both cases, we avoid complicating the main u-controller

You can think of these as “microcode” at different levels of abstraction

12

Nanocode Concept Illustrated

13

ROM

mPC

arithmetic datapath

a “mcoded” FPU implementation

ROM

mPC

processor datapath

a “mcoded” processor implementation

We refer to this as “nanocode” when a mcoded subsystem is embedded in a mcoded system

Microcoded Multi-Cycle MIPS Design

Any ISA can be implemented with a microprogrammed microarchitecture

P&H, Appendix D: Microprogrammed MIPS design

We will not cover this in class

However, you can do an extra credit assignment for Lab 2

14

Microcoded Multi-Cycle MIPS Design

15 [Based on original figure from P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]

Control Logic for MIPS FSM


Microprogrammed Control for MIPS FSM


Multi-Cycle vs. Single-Cycle uArch

Advantages

Disadvantages

You should be very familiar with this right now

18

Microprogrammed vs. Hardwired Control

Advantages

Disadvantages

You should be very familiar with this right now

19

Can We Do Better?

What limitations do you see with the multi-cycle design?

Limited concurrency

Some hardware resources are idle during different phases of instruction processing cycle

“Fetch” logic is idle when an instruction is being “decoded” or “executed”

Most of the datapath is idle when a memory access is happening

20

Can We Use the Idle Hardware to Improve Concurrency?

Goal: More concurrency Higher instruction throughput

(i.e., more “work” completed in one cycle)

Idea: When an instruction is using some resources in its processing phase, process other instructions on idle resources not needed by that instruction

E.g., when an instruction is being decoded, fetch the next instruction

E.g., when an instruction is being executed, decode another instruction

E.g., when an instruction is accessing data memory (ld/st), execute the next instruction

E.g., when an instruction is writing its result into the register file, access data memory for the next instruction

21

Pipelining

22

Pipelining: Basic Idea

More systematically:

Pipeline the execution of multiple instructions

Analogy: “Assembly line processing” of instructions

Idea:

Divide the instruction processing cycle into distinct “stages” of processing

Ensure there are enough hardware resources to process one instruction in each stage

Process a different instruction in each stage

Instructions consecutive in program order are processed in consecutive stages

Benefit: Increases instruction processing throughput (1/CPI)

Downside: Start thinking about this… 23

Example: Execution of Four Independent ADDs

Multi-cycle: 4 cycles per instruction

Pipelined: 4 cycles per 4 instructions (steady state)

24

Time

F D E W

F D E W

F D E W

F D E W

F D E W

F D E W

F D E W

F D E W

Time

Is life always this beautiful?

The Laundry Analogy

“place one dirty load of clothes in the washer”

“when the washer is finished, place the wet load in the dryer”

“when the dryer is finished, take out the dry load and fold”

“when folding is finished, ask your roommate (??) to put the clothes away”

25

- steps to do a load are sequentially dependent

- no dependence between different loads - different steps do not share resources

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

Based on original figure from [P&H CO&D, COPYRIGHT 2004 Elsevier. ALL RIGHTS RESERVED.]

Pipelining Multiple Loads of Laundry

26

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

- latency per load is the same

- throughput increased by 4

- 4 loads of laundry in parallel

- no additional resources


Pipelining Multiple Loads of Laundry: In Practice

27

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

the slowest step decides throughput


Pipelining Multiple Loads of Laundry: In Practice

28

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order

A

B

A

B

throughput restored (2 loads per hour) using 2 dryers

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Time76 PM 8 9 10 11 12 1 2 AM

A

B

C

D

Task

order

Task

order


An Ideal Pipeline

Goal: Increase throughput with little increase in cost (hardware cost, in case of instruction processing)

Repetition of identical operations

The same operation is repeated on a large number of different inputs (e.g., all laundry loads go through the same steps)

Repetition of independent operations

No dependencies between repeated operations

Uniformly partitionable suboperations

Processing can be evenly divided into uniform-latency suboperations (that do not share resources)

Fitting examples: automobile assembly line, doing laundry

What about the instruction processing “cycle”?

29

Ideal Pipelining

30

combinational logic (F,D,E,M,W) T psec

BW=~(1/T)

BW=~(2/T) T/2 ps (F,D,E) T/2 ps (M,W)

BW=~(3/T) T/3 ps (F,D)

T/3 ps (E,M)

T/3 ps (M,W)

More Realistic Pipeline: Throughput

Nonpipelined version with delay T

BW = 1/(T+S) where S = latch delay

k-stage pipelined version

BWk-stage = 1 / (T/k +S )

BWmax = 1 / (1 gate delay + S )

31

T ps

T/k ps

T/k ps

Latch delay reduces throughput

(switching overhead b/w stages)

More Realistic Pipeline: Cost

Nonpipelined version with combinational cost G

Cost = G+L where L = latch cost

k-stage pipelined version

Costk-stage = G + Lk

32

G gates

G/k G/k

Latches increase hardware cost

Pipelining Instruction Processing

33

Remember: The Instruction Processing Cycle

Fetch

Decode

Evaluate Address

Fetch Operands

Execute

Store Result

34

1. Instruction fetch (IF) 2. Instruction decode and register operand fetch (ID/RF) 3. Execute/Evaluate memory address (EX/AG) 4. Memory operand fetch (MEM) 5. Store/writeback result (WB)

Remember the Single-Cycle Uarch

35

Shift left 2

PC

Instruction memory

Read address

Instruction [31– 0]

Data memory

Read data

Write data

RegistersWrite register

Write data

Read data 1

Read data 2

Read register 1

Read register 2

Instruction [15–11]



Add

ALU result

Zero


MemtoReg

ALUOp

MemWrite

RegWrite

MemRead

Branch

JumpRegDst

ALUSrc


4

M u x

Instruction [25–0] Jump address [31– 0]

PC+4 [31–28]

Sign extend

16 32Instruction [15–0]

1

M u x

1

0

M u x

0

1

M u x

0

1

ALU control

Control

AddALU

result

M u x

0

1 0

ALU

Shift left 2

26 28

Address

PCSrc2=Br Taken

PCSrc1=Jump

ALU operation

bcond


T BW=~(1/T)

Dividing Into Stages

36

200ps

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instruction

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Address

Data

memory1

ALU result

M u x

ALU

Zero

IF: Instruction fetch ID: Instruction decode/

register file read

EX: Execute/

address calculation

MEM: Memory access WB: Write back

Is this the correct partitioning? Why not 4 or 6 stages? Why not different boundaries?

100ps 200ps 200ps 100ps

RF write

ignore for now


Instruction Pipeline Throughput

37

Instruction

fetchReg ALU

Data

accessReg

8 nsInstruction

fetchReg ALU

Data

accessReg

8 nsInstruction

fetch

8 ns

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)

2 4 6 8 10 12 14 16 18

2 4 6 8 10 12 14

...

Program

execution

order

(in instructions)

Instruction

fetchReg ALU

Data

accessReg

Time

lw $1, 100($0)

lw $2, 200($0)

lw $3, 300($0)

2 nsInstruction

fetchReg ALU

Data

accessReg

2 nsInstruction

fetchReg ALU

Data

accessReg

2 ns 2 ns 2 ns 2 ns 2 ns

Program

execution

order

(in instructions)

200 400 600 800 1000 1200 1400 1600 1800

200 400 600 800 1000 1200 1400

800ps

800ps

800ps

200ps 200ps 200ps 200ps 200ps

200ps

200ps

5-stage speedup is 4, not 5 as predicted by the ideal model. Why?

Enabling Pipelined Processing: Pipeline Registers

38 T

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instruction

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Address

Data

memory1

ALU result

M u x

ALU

Zero

IF: Instruction fetch ID: Instruction decode/

register file read

EX: Execute/

address calculation

MEM: Memory access WB: Write back

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Data

memory

Address

No resource is used by more than 1 stage!

IRD

PC

F

PC

D+4

PC

E+4

nP

CM

AE

BE

Imm

E

Ao

ut M

B

M

MD

RW

A

ou

t W


T/k ps

T/k ps

Pipelined Operation Example

39

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw

Address

Data

memory

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Instruction decode

lw

Address

Data

memory

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw

Address

Data

memory

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Instruction decode

lw

Address

Data

memory

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Execution

lw

Address

Data

memory

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Data

memory

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Memory

lw

Address

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write data

Read dataData

memory

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Write back

lw

Write register

Address

97108/Patterson

Figure 06.15

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Data

memory

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Memory

lw

Address

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write data

Read dataData

memory

1

ALU result

M u x

ALU

Zero

ID/EX MEM/WB

Write back

lw

Write register

Address

97108/Patterson

Figure 06.15

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0

Address

Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

Data

memory

1

ALU result

M u x

ALU

Zero

ID/EX


All instruction classes must follow the same path and timing through the pipeline stages.

Any performance impact?


40

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction decode

lw $10, 20($1)

Instruction fetch

sub $11, $2, $3

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw $10, 20($1)

Address

Data

memory

Address

Data

memory

Clock 1

Clock 2

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction decode

lw $10, 20($1)

Instruction fetch

sub $11, $2, $3

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw $10, 20($1)

Address

Data

memory

Address

Data

memory

Clock 1

Clock 2

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data

Memory

lw $10, 20($1)

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

sub $11, $2, $3

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

lw $10, 20($1)

Instruction decode

sub $11, $2, $3

3216Sign

extend

Address

Data

memory

Data

memory

Address

Clock 3

Clock 4

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data

Memory

lw $10, 20($1)

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

sub $11, $2, $3

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

lw $10, 20($1)

Instruction decode

sub $11, $2, $3

3216Sign

extend

Address

Data

memory

Data

memory

Address

Clock 3

Clock 4

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uction

IF/ID EX/MEMID/EX MEM/WB

Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uctio

n


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Memory

sub $11, $2, $3

Address

Data memory

Address

Data

memory

Clock 6

Clock 5

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uction


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uctio

n


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Memory

sub $11, $2, $3

Address

Data memory

Address

Data

memory

Clock 6

Clock 5


Is life always this beautiful?

Illustrating Pipeline Operation: Operation View

41

MEM

EX

ID

IF Inst4

WB

IF

MEM

IF

MEM

EX

t0 t1 t2 t3 t4 t5

ID

EX IF ID

IF ID

Inst0 ID

IF Inst1

EX

ID

IF Inst2

MEM

EX

ID

IF Inst3

WB

WB MEM

EX

WB

steady state

(full pipeline)

Illustrating Pipeline Operation: Resource View

42

I0

I0

I1

I0

I1

I2

I0

I1

I2

I3

I0

I1

I2

I3

I4

I1

I2

I3

I4

I5

I2

I3

I4

I5

I6

I3

I4

I5

I6

I7

I4

I5

I6

I7

I8

I5

I6

I7

I8

I9

I6

I7

I8

I9

I10

t0 t1 t2 t3 t4 t5 t6 t7 t8 t9 t10

IF

ID

EX

MEM

WB

Control Points in a Pipeline

43

PC

Instruction memory

Address

Instr

uction


MemtoReg

ALUOp

Branch

RegDst

ALUSrc

4

16 32


0

0Registers

Write register

Write data

Read data 1

Read data 2

Read register 1

Read register 2

Sign extend

M u x

1Write

data

Read

data M u x

1

ALU

control

RegWrite

MemRead


6

IF/ID ID/EX EX/MEM MEM/WB

MemWrite

Address

Data memory

PCSrc

Zero

AddAdd

result

Shift

left 2

ALU

result

ALU

Zero

Add

0

1

M u x

0

1

M u x

Identical set of control points as the single-cycle datapath!!


Control Signals in a Pipeline

For a given instruction

same control signals as single-cycle, but

control signals required at different cycles, depending on stage

Option 1: decode once using the same logic as single-cycle and buffer signals until consumed

Option 2: carry relevant “instruction word/field” down the pipeline and decode locally within each or in a previous stage

Which one is better?

44

Control

EX

M

WB

M

WB

WB

IF/ID ID/EX EX/MEM MEM/WB

Instruction

Pipelined Control Signals

45

PC

Instruction memory

Instr

uctio

n

Add


Me

mto

Re

g

ALUOp

Branch

RegDst

ALUSrc

4

16 32Instruction [15– 0]

0

0

M u x

0

1

AddAdd

result

RegistersWrite register

Write data

Read data 1

Read data 2

Read register 1

Read register 2

Sign extend

M u x

1

ALU result

Zero

Write data

Read data

M u x

1

ALU control

Shift left 2

Re

gW

rite

MemRead

Control

ALU


6

EX

M

WB

M

WB

WBIF/ID

PCSrc

ID/EX

EX/MEM

MEM/WB

M u x

0

1

Me

mW

rite

Address

Data memory

Address


Remember: An Ideal Pipeline

Goal: Increase throughput with little increase in cost (hardware cost, in case of instruction processing)

Repetition of identical operations

The same operation is repeated on a large number of different inputs (e.g., all laundry loads go through the same steps)

Repetition of independent operations

No dependencies between repeated operations

Uniformly partitionable suboperations

Processing an be evenly divided into uniform-latency suboperations (that do not share resources)

Fitting examples: automobile assembly line, doing laundry

What about the instruction processing “cycle”?

46

Instruction Pipeline: Not An Ideal Pipeline

Identical operations ... NOT!

different instructions not all need the same stages Forcing different instructions to go through the same pipe stages external fragmentation (some pipe stages idle for some instructions)

Uniform suboperations ... NOT!

different pipeline stages not the same latency Need to force each stage to be controlled by the same clock

internal fragmentation (some pipe stages are too fast but all take the same clock cycle time)

Independent operations ... NOT! instructions are not independent of each other

Need to detect and resolve inter-instruction dependencies to ensure the pipeline provides correct results pipeline stalls (pipeline is not always moving)

47

Issues in Pipeline Design

Balancing work in pipeline stages

How many stages and what is done in each stage

Keeping the pipeline correct, moving, and full in the presence of events that disrupt pipeline flow

Handling dependences

Data

Control

Handling resource contention

Handling long-latency (multi-cycle) operations

Handling exceptions, interrupts

Advanced: Improving pipeline throughput

Minimizing stalls

48

Causes of Pipeline Stalls

Stall: A condition when the pipeline stops moving

Resource contention

Dependences (between instructions)

Data

Control

Long-latency (multi-cycle) operations

49

Dependences and Their Types

Also called “dependency” or less desirably “hazard”

Dependences dictate ordering requirements between instructions

Two types

Data dependence

Control dependence

Resource contention is sometimes called resource dependence

However, this is not fundamental to (dictated by) program semantics, so we will treat it separately

50

Handling Resource Contention

Happens when instructions in two pipeline stages need the same resource

Solution 1: Eliminate the cause of contention

Duplicate the resource or increase its throughput

E.g., use separate instruction and data memories (caches)

E.g., use multiple ports for memory structures

Solution 2: Detect the resource contention and stall one of the contending stages

Which stage do you stall?

Example: What if you had a single read and write port for the register file?

51

Data Dependences

Types of data dependences

Flow dependence (true data dependence – read after write)

Output dependence (write after write)

Anti dependence (write after read)

Which ones cause stalls in a pipelined machine?

For all of them, we need to ensure semantics of the program is correct

Flow dependences always need to be obeyed because they constitute true dependence on a value

Anti and output dependences exist due to limited number of architectural registers

They are dependence on a name, not a value

We will later see what we can do about them

52

Data Dependence Types

53

Flow dependence r3 r1 op r2 Read-after-Write r5 r3 op r4 (RAW)

Anti dependence r3 r1 op r2 Write-after-Read r1 r4 op r5 (WAR) Output-dependence r3 r1 op r2 Write-after-Write r5 r3 op r4 (WAW) r3 r6 op r7


54

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction decode

lw $10, 20($1)

Instruction fetch

sub $11, $2, $3

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw $10, 20($1)

Address

Data

memory

Address

Data

memory

Clock 1

Clock 2

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction decode

lw $10, 20($1)

Instruction fetch

sub $11, $2, $3

Instruction

memory

Address

4

32

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Instruction fetch

lw $10, 20($1)

Address

Data

memory

Address

Data

memory

Clock 1

Clock 2

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data

Memory

lw $10, 20($1)

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

sub $11, $2, $3

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

lw $10, 20($1)

Instruction decode

sub $11, $2, $3

3216Sign

extend

Address

Data

memory

Data

memory

Address

Clock 3

Clock 4

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uctio

n

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

3216Sign

extend

Write register

Write data

Memory

lw $10, 20($1)

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

sub $11, $2, $3

Instruction

memory

Address

4

0

AddAdd

result

Shift

left 2

Instr

uction

IF/ID EX/MEM MEM/WB

M u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

Write register

Write data

Read data

1

ALU result

M u x

ALU

Zero

ID/EX

Execution

lw $10, 20($1)

Instruction decode

sub $11, $2, $3

3216Sign

extend

Address

Data

memory

Data

memory

Address

Clock 3

Clock 4

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uction


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uctio

n


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Memory

sub $11, $2, $3

Address

Data memory

Address

Data

memory

Clock 6

Clock 5

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uction


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

lw $10, 20($1)

Instruction

memory

Address

4

32

0

AddAdd

result

1

ALU result

Zero

Shift

left 2

Instr

uctio

n


Write backM u x

0

1

Add

PC

0Write data

M u x

1

Registers

Read data 1

Read data 2

Read register 1

Read register 2

16Sign

extend

M u x

ALURead data

Write register

Write data

sub $11, $2, $3

Memory

sub $11, $2, $3

Address

Data memory

Address

Data

memory

Clock 6

Clock 5


What if the SUB were dependent on LW?

Data Dependence Handling

55

Readings for Next Few Lectures

P&H Chapter 4.9-4.11

Smith and Sohi, “The Microarchitecture of Superscalar Processors,” Proceedings of the IEEE, 1995

More advanced pipelining

Interrupt and exception handling

Out-of-order and superscalar execution concepts

56

How to Handle Data Dependences

Anti and output dependences are easier to handle

write to the destination in one stage and in program order

Flow dependences are more interesting

Five fundamental ways of handling flow dependences

Detect and wait until value is available in register file

Detect and forward/bypass data to dependent instruction

Detect and eliminate the dependence at the software level

No need for the hardware to detect dependence

Predict the needed value(s), execute “speculatively”, and verify

Do something else (fine-grained multithreading)

No need to detect

57

Interlocking

Detection of dependence between instructions in a pipelined processor to guarantee correct execution

Software based interlocking

vs.

Hardware based interlocking

MIPS acronym?

58

Approaches to Dependence Detection (I)

Scoreboarding

Each register in register file has a Valid bit associated with it

An instruction that is writing to the register resets the Valid bit

An instruction in Decode stage checks if all its source and destination registers are Valid

Yes: No need to stall… No dependence

No: Stall the instruction

Advantage:

Simple. 1 bit per register

Disadvantage:

Need to stall for all types of dependences, not only flow dep.

59

Not Stalling on Anti and Output Dependences

What changes would you make to the scoreboard to enable this?

60

Approaches to Dependence Detection (II)

Combinational dependence check logic

Special logic that checks if any instruction in later stages is supposed to write to any source register of the instruction that is being decoded

Yes: stall the instruction/pipeline

No: no need to stall… no flow dependence

Advantage:

No need to stall on anti and output dependences

Disadvantage:

Logic is more complex than a scoreboard

Logic becomes more complex as we make the pipeline deeper and wider (flash-forward: think superscalar execution)

61

Once You Detect the Dependence in Hardware

What do you do afterwards?

Observation: Dependence between two instructions is detected before the communicated data value becomes available

Option 1: Stall the dependent instruction right away

Option 2: Stall the dependent instruction only when necessary data forwarding/bypassing

Option 3: …

62

We did not cover the following slides in lecture.

These are for your preparation for the next lecture.

Data Forwarding/Bypassing

Problem: A consumer (dependent) instruction has to wait in decode stage until the producer instruction writes its value in the register file

Goal: We do not want to stall the pipeline unnecessarily

Observation: The data value needed by the consumer instruction can be supplied directly from a later stage in the pipeline (instead of only from the register file)

Idea: Add additional dependence check logic and data forwarding paths (buses) to supply the producer’s value to the consumer right after the value is available

Benefit: Consumer can move in the pipeline until the point the value can be supplied less stalling

64

A Special Case of Data Dependence

Control dependence

Data dependence on the Instruction Pointer / Program Counter

65

Control Dependence

Question: What should the fetch PC be in the next cycle?

Answer: The address of the next instruction

All instructions are control dependent on previous ones. Why?

If the fetched instruction is a non-control-flow instruction:

Next Fetch PC is the address of the next-sequential instruction

Easy to determine if we know the size of the fetched instruction

If the instruction that is fetched is a control-flow instruction:

How do we determine the next Fetch PC?

In fact, how do we know whether or not the fetched instruction is a control-flow instruction?

66

Documents

18-447 Computer Architecturecourse.ece.cmu.edu/~ece447/s15/lib/exe/fetch.php?...Microsequencer BEN 2 x Control Store 6 IR[15:11] 6 (J, COND, IRD) 9 26 35 35 Fi g u r e C. 4 : Th e