Design a MIPS Processor (2)

ACCESS IC LABORTORY

93-2 Digital System DesignGraduate Institute of Electronics Engineering, NTU

Design a MIPS Processor (2/2)Design a MIPS Processor (2/2)

Lecturer: Chihhao ChaoAdvisor: Prof. An-Yeu Wu2005/5/13 Friday

Graduate Institute of Electronics Engineering, NTU

P2

Digital System Design

OutlineOutlinev 6.1 An Overview of Pipeliningv 6.2 A Pipelined Datapathv 6.3 Pipelined Controlv 6.4 Data Hazards and Forwardingv 6.5 Data Hazards and Stallsv 6.6 Branch Hazardsv 6.8 Exceptions


P3


Pipelining is Natural!Pipelining is Natural!vPipelining provides a method for executing multiple

instructions at the same time.

vLaundry ExamplevAnn, Brian, Cathy, Dave

each have one load of clothes to wash, dry, and foldvWasher takes 30 minutesvDryer takes 40 minutesv“Folder” takes 20 minutes

A B C D


P4


vSequential laundry takes 6 hours for 4 loadsvIf they learned pipelining, how long would laundry take?

A

B

C

D

30 40 20 30 40 20 30 40 20 30 40 20

6 PM 7 8 9 10 11 Midnight

Task

Order

Time

Sequential LaundrySequential Laundry


P5


vPipelined laundry takes 3.5 hours for 4 loads

Task

Order

A

B

C

D

6 PM 7 8 9 10 11 MidnightTime

30 40 40 40 40 20

Pipelined Laundry: Start work ASAPPipelined Laundry: Start work ASAP


P6


vPipelining doesn’t help latency of single task, it helps throughput of entire workloadvPipeline rate limited by

slowest pipeline stagevMultiple tasks operating

simultaneously using different resourcesvPotential speedup = Number

pipe stagesvUnbalanced lengths of pipe

stages reduces speedupvTime to “fill” pipeline and

time to “drain” it reduces speedupvStall for Dependences

A

B

C

D

6 PM 7 8 9

Task

Order

Time

30 40 40 40 40 20

Pipelining LessonsPipelining Lessons


P7


The 5 Stages of the Load InstructionThe 5 Stages of the Load Instruction

vIFetch: Instruction FetchvFetch the instruction from the Instruction Memory

vReg/Dec: Registers Fetch and Instruction DecodevExec: Calculate the memory addressvMem: Read the data from the Data MemoryvWr: Write the data back to the register file

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5

IFetch Reg/Dec Exec Mem WrLoad


P8


Pipeline ExecutionPipeline Execution

vOn a processor multple instructions are in various stages at the same time.vAssume each instruction takes five cycles

IFetch Dcd Exec Mem WB





IFetch Dcd Exec Mem WBProgram Flow

Time


P9


Single Cycle, MultiSingle Cycle, Multi--cycle, Pipelinedcycle, Pipelined

Clk

Cycle 1

Multiple Cycle Implementation:

IFetch Reg Exec Mem Wr

Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7 Cycle 8 Cycle 9 Cycle 10

Load IFetch Reg Exec Mem Wr

IFetch Reg Exec MemLoad Store

Pipeline Implementation:

IFetch Reg Exec Mem WrStore

Clk

Single Cycle Implementation:Load Store Waste

IFetchR-type

IFetch Reg Exec Mem WrR-type

Cycle 1 Cycle 2


P10


Why Pipeline? Because the Resources Are There!Why Pipeline? Because the Resources Are There!

Instr.

Order

Time (clock cycles)

Inst 0

Inst 1

Inst 2

Inst 4

Inst 3

AL

UIm Reg Dm Reg

AL

UIm Reg Dm Reg

AL

UIm Reg Dm RegA

LUIm Reg Dm Reg

AL

UIm Reg Dm Reg


P11


Pipelining MIPS ExecutionPipelining MIPS Execution


P12


Pipeline HazardsPipeline Hazardsv Structural hazardv An occurrence in which a planned instruction cannot execute in

the proper clock cycle because the hardware cannot support the combination of instructions that are set to execute in the given clock cycle.

v Data hazardv Also called pipeline data hazard. An occurrence in which a

planned instruction cannot execute in the proper clock cycle because data that is needed to execute the instruction is not yet available.

v Control hazardv Also called branch hazard. An occurrence in which the proper

instruction cannot execute in the proper clock cycle because theinstruction that was fetched is NOT the one that is needed; that is, the flow of instruction addresses is not what the pipeline expected.


P13


Data HazardData Hazardv Forwardingv Also called bypassing. A method of resolving a data hazard by

retrieving the missing data element from internal buffers rather than waiting for it to arrive from programmer-visible register or memory.

v Load-use data hazardv A specific form of data hazard in which the data requested by a

load instruction has not yet become available when it is requested.v Pipeline stallv Also called bubble. A stall initiated in order to resolve a hazard


P14


Control HazardControl HazardvUntaken branchvOne that falls through to the successive instruction. A

taken branch is one that causes transfer to the branch target

vBranch predictionvA method of resolving a branch hazard that assumes a

given outcome for the branch, and proceeds from that assumption rather than waiting to ascertain the actual outcome


P15


Pipeline Overview SummaryPipeline Overview SummaryvLatency (pipeline)vThe number of stages in a pipeline or the number of

stages between two instructions during execution.vThroughput (pipeline)vThe number of instructions executed per unit time.


P16




P17


Designing a Pipelined ProcessorDesigning a Pipelined Processorv Examine the datapath and control diagramv Starting with single-or multi-cycle datapath?v Single-or multi-cycle control?

v Partition datapath into stages:v IF (instruction fetch), ID (instruction decode and register file read),

EX (execution or address calculation), MEM (data memory access), WB (write back)

v Associate resources with statesv Ensure that flows do not conflict, or figure out how to

resolvev Assert control in appropriate stage


P18


Use MultiUse Multi--cycle Execution Stepscycle Execution Steps

But, use single-cycle datapath….. (separate memory, why??)


P19


Split SingleSplit Single--cycle Datapathcycle Datapath

What to add to split the datapath into stages


P20


Add Pipeline RegistersAdd Pipeline Registers

v Use registers between stages to carry data and control

(Flip/Flop)


P21


Consider Consider loadload

v IF: Instruction Fetchv Fetch the instruction from the Instruction Memory

v ID: Instruction Decodev Registers fetch and instruction decode

v EX: Calculate the memory addressvMEM: Read the data from the Data MemoryvWB: Write the data back to the register file

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5

Ifetch Reg/Dec Exec Mem Wrlw


P22


Pipelining Pipelining loadload

v 5 functional units in the pipeline datapath are:v Instruction Memory for the IFetch stagev Register File’s Read ports (busA and busB) for the Reg/Dec stagev ALU for the Exec stagev Data Memory for the MEM stagev Register File’s Write port (busW) for the WB stage

Clock

Cycle 1 Cycle 2 Cycle 3 Cycle 4 Cycle 5 Cycle 6 Cycle 7

Ifetch Reg/Dec Exec Mem Wr1st lw

Ifetch Reg/Dec Exec Mem Wr2nd lw

Ifetch Reg/Dec Exec Mem Wr3rd lw


P23


IF Stage of IF Stage of load wordload wordv IF/ID= mem[PC] ; PC = PC + 4


P24


ID Stage of ID Stage of load wordload wordv ID/EX(A)= Reg[IR[25-21]]; ID/EX(B)= Reg[IR[20-16]];v ID/EX = Sign-extension of ID[15:0]


P25


EX Stage of EX Stage of load wordload wordv EX/MEM = A + sign-ext(IR[15-0]) % address computation


P26


MEM Stage of MEM Stage of load wordload wordv MEM/WB = mem[ALUout]


P27


WB Stage of WB Stage of loadloadv Reg[ IR[20-16] ] = MEM/WB


P28


Pipelined DatapathPipelined Datapath


P29


The Four Stages of RThe Four Stages of R--typetype

v IF: fetch the instruction from the Instruction Memoryv ID: registers fetch and instruction decodev EX: v ALU operates on the two register operandsv Update PC

vWB: write ALU output back to the register file

Clock

Cycle 1 Cycle 2 Cycle 3 Cycle 4

Ifetch Reg/Dec Exec WrR-type


P30


Pipelining RPipelining R--type and type and loadload

vWe have a structural hazard:vTwo instructions try to write to the register file at the

same time!vOnly one write port

Clock




Ifetch Reg/Dec Exec Mem WrLoad



We have a problem!


P31


Important ObservationImportant Observationv Each functional unit can only be used once per

instruction v Each functional unit must be used at the same stage for

all instructions:v Load uses Register File’s write port during its 5th stage

v R-type uses Register File’s write port during its 4th stage

Several ways to solve: 1) forwarding, 2) adding pipeline bubble, 3) making

instructions same length




P32


Solution 1: Insert BubbleSolution 1: Insert Bubble

v Insert a bubble into the pipeline to prevent two writes at the same cyclev The control logic can be complexv Lose instruction fetch and issue opportunity

v No instruction is started in Cycle 6

Clock



Ifetch Reg/Dec Exec



Ifetch Reg/Dec Exec WrR-type Pipeline

Bubble



P33


Solution 2: Delay RSolution 2: Delay R--typetype’’s Writes Writev Delay R-type’s register write by one cycle:v R-type also use Reg File’s write port at Stage 5v MEM is a NOP stage: nothing is being done.

Clock


Ifetch Reg/Dec Mem WrR-type





Exec

Exec

Exec

Exec

Ifetch Reg/Dec Mem WrR-type Exec


P34


The Four Stages of The Four Stages of storestore

v IF: fetch the instruction from the Instruction Memoryv ID: registers fetch and instruction decodev EX: calculate the memory addressv MEM: write the data into the Data Memory

Add an extra stage:v WB: NOP


Ifetch Reg/Dec Exec MemStore Wr


P35


The Four Stages of The Four Stages of beqbeq

v IF: fetch the instruction from the Instruction Memoryv ID: registers fetch and instruction decodev EX:

v compares the two register operandv select correct branch target addressv latch into PC

v Add two extra stages:v MEM: NOPv WB: NOP


Ifetch Reg/Dec Exec MemStore Wr


P36


Use Graphical Representation for Pipelined Use Graphical Representation for Pipelined MIPSMIPS

v The graph can help to answer questions like:v How many cycles to execute this code?vWhat is the ALU doing during cycle 4?


P37


Example 1: Cycle 1Example 1: Cycle 1


P38




P39




P40




P41




P42




P43




P44


Pipeline Control: Control SignalsPipeline Control: Control Signals


P45


Group Signals According to StagesGroup Signals According to Stagesv Can use control signals of single-cycle CPU


P46


Data Stationary ControlData Stationary Controlv Pass control signals along just like the datavMain control generates control signals during ID


P47


Data Stationary Control (cont.)Data Stationary Control (cont.)v Signals for EX (ExtOp, ALUSrc, ...) are used 1 cycle laterv Signals for MEM (MemWr, Branch) are used 2 cycles laterv Signals for WB (MemtoReg, MemWr) are used 3 cycles later

IF/ID R

egister

ID/E

x Register

Ex/M

emR

egister

Mem

/Wr

Register

Reg/Dec Exec Mem

ExtOp

ALUOpRegDst

ALUSrc

BranchMemWr

MemtoRegRegWr

MainControl

ExtOp

ALUOpRegDst

ALUSrc

MemtoRegRegWr

MemtoRegRegWr

MemtoRegRegWr

BranchMemWr Branch

MemWr

Wr


P48


Datapath with ControlDatapath with Control


P49


LetLet’’s Try it Outs Try it Out

Sample Assembly Program

lw $10, 20($1)sub $11, $2, $3and $12, $4, $5or $13, $6, $7add $14, $8, $9


P50




P51




P52




P53




P54




P55




P56




P57




P58




P59


Summary of Pipeline BasicsSummary of Pipeline BasicsvPipelining is a fundamental conceptvMultiple steps using distinct resourcesvUtilize capabilities of datapath by pipelined instruction

processingØStart next instruction while working on the current oneØLimited by length of longest stage (plus fill/flush)ØNeed to detect and resolve hazards

vWhat makes it easy in MIPS?vAll instructions are of the same lengthvJust a few instruction formatsvMemory operands only in loads and stores

vWhat makes pipelining hard? hazards


P60




P61


Data HazardsData Hazardsv Order of operand accesses changed by pipelinev Starting next instruction before first is finishedv Dependencies “go backward in time”


P62


Handling Data HazardsHandling Data HazardsvDetect vResolve remaining onesvCompiler inserts NOPvStallvForward


P63


Software SolutionSoftware SolutionvHave compiler guarantee no hazardsvWhere do we insert the NOPs?

sub $2, $1, $3and $12, $2, $5or $13, $6, $2add $14, $2, $2sw $15, 100($2)

vProblem: not efficient enough!


P64


Detecting Data HazardsDetecting Data Hazardsv Hazard conditions:v EX/MEM.RegisterRd = ID/EX.RegisterRs (1a)v EX/MEM.RegisterRd = ID/EX.RegisterRt (1b)vMEM/WB.RegisterRd = ID/EX.RegisterRs (2a)vMEM/WB.RegisterRd = ID/EX.RegisterRt (2b)

v Two optimizations:v Don’t forward if instruction does not write registerà check if RegWrite is assertedv Don’t forward if destination register is $0à check if RegisterRd = 0


P65


Detecting Data Hazards (cont.)Detecting Data Hazards (cont.)vHazard conditions using control signals:

At EX stage (EX hazard):

l If ( EX/MEM.RegWriteand (EX/MEM.RegRd≠0)and (EX/MEM.RegRd=ID/EX.RegRs )

ForwardA = 10


P66


Detecting Data Hazards (cont.)Detecting Data Hazards (cont.)vHazard conditions using control signals:vAt MEM stage:

lMEM/WB.RegWriteand (MEM/WB.RegRd≠0)and (MEM/WB.RegRd=ID/EX.RegRs)

v(replace ID/EX.RegRt for ID/EX.RegRs for the other two conditions)


P67


Resolving Hazards: ForwardingResolving Hazards: Forwardingv Use temporary results, e.g., those in pipeline registers, don’t wait for

them to be written


P68


Forwarding LogicForwarding Logicv Forwarding: input to ALU from any pipe registers

v Add multiplexors to ALU input v Control forwarding in EX à carry Rs in ID/EX

v Control signals for forwarding:v If both WB and MEM forward, e.g., add $1,$1,$2; add $1,$1,$3; add

$1,$1,$4; => let MEM forward

v EX hazard:Ø if (EX/MEM.RegWrite and (EX/MEM.RegRd≠0) and

(EX/MEM.RegRd=ID/EX.RegRs))ForwardA=10

v MEM hazard:Ø if (MEM/WB.RegWriteand (MEM/WB.RegRd≠0)

and (EX/MEM.RegRd≠ID/EX.Reg.Rs)and (MEM/WB.RegRd=ID/EX.RegRs))

ForwardA=01

(ID/EX.RegRt <-> ID/EX.RegRs, ForwardB <-> ForwardA)


P69


No ForwardingNo Forwarding


P70


With ForwardingWith Forwarding


P71


Pipeline with ForwardingPipeline with Forwarding


P72




P73




P74




P75




P76




P77


Can't Always ForwardCan't Always Forwardv lw can still cause a hazard:

v if is followed by an instruction to read the loaded reg.


P78


StallingStallingv Stall pipeline by keeping instructions in same stage and inserting an

NOP instead


P79


Handling StallsHandling Stallsv Hazard detection unit in ID to insert stall between a

load instruction and its use:if (ID/EX.MemRead and

((ID/EX.RegisterRt= IF/ID.RegisterRs) or(ID/EX.RegisterRt= IF/ID.registerRt))

stall the pipeline for one cycle(ID/EX.MemRead=1 indicates a load instruction)

v How to stall?v Stall instruction in IF and ID: not change PC and IF/ID

=> the stages re-execute the instructionsvWhat to move into EX: insert an NOP by changing EX, MEM,

WB control fields of ID/EX pipeline register to 0l as control signals propagate, all control signals to EX, MEM, WB

are deasserted and no registers or memories are written


P80


Pipeline with Stalling UnitPipeline with Stalling Unitv Forwarding controls ALU inputs, hazard detection controls PC, IF/ID,

control signals


P81




P82




P83




P84




P85




P86




P87


OutlineOutlinev 6.1 An Overview of Pipeliningv 6.2 A Pipelined Datapathv 6.3 Pipelined Controlv 6.4 Data Hazards and Forwardingv 6.5 Data Hazards and Stallsv 6.6 Branch Hazardsv 6.8 Exceptions (optional)v (optional)


P88


Branch HazardsBranch HazardsvWhen decide to branch, other instructions are still in the pipe


P89


Handling Branch HazardHandling Branch Hazardv Predict branch always not taken

v Need to add hardware for flushing inst. if wrongv Branch decision made at MEM => need to flush inst. in IF, ID, EX by changing

control values to 0

v Reduce delay of taken branch by moving branch execution earlier in the pipelinev Move up branch address calculation to IDv Check branch equality at ID (using XOR) by comparing the two registers read

during IDv Branch decision made at EX => one inst. to flushv Add a control signal, IF.Flush, to zero instruction field of IF/ID => making the

instruction an NOP

v Dynamic branch predictionv Compiler rescheduling, delay branch


P90


Delayed BranchDelayed Branchv Predict-not-taken + branch decision at ID

=> the following inst. is always executed=> branches take effect 1 cycle later

v 0 clock cycle per branch instruction if can find instruction to put in slot (≅50% of time)


P91


Pipeline with FlushingPipeline with Flushing


P92




P93




P94




P95


What about Exceptions?What about Exceptions?v 5 instructions executing in 5 stage pipelinev How to stop the pipeline? restart?vWho caused the interrupt?vWho to serve first, if multiple interrupts at the same time?

v Need to know in which stage an exception can occurStage Problem interrupts occurringIF Page fault; misaligned memory access;

memory-protection violationID Undefined or illegal opcodeEX Arithmetic exceptionMEM Page fault; misaligned memory access; memory

error; mem-protection violation;


P96


Handling ExceptionsHandling Exceptionsv Suppose overflow occur at add $1,$2,$1v Disable writes of instructions till trap hits WB, e.g., flush

following instructions using IF.Flush, ID.Flush, EX.Flush to cause multiplexorsto zero control signals(overflow exception detected at EX => flush offending instr.)v Force trap instruction into IF, e.g., fetch from 4000 0040hex by

adding 4000 0040hex to PC input MUXv Save address of offending instruction in EPC

vMultiple interrupts: use priority hardware to choose the earliest instruction to interrupt

v External interrupts: flexible in when to interrupt


P97


Pipeline with ExceptionPipeline with Exception

Documents

Design a MIPS Processor (2)