Out-of-Order Execution Structures Optimizations

A. Moshovos © ECE1773 - Fall ‘07 ECE Toronto

Out-of-Order Execution StructuresOptimizations

Tag Elimination

Conventional Schedulers are Overdesigned• For MIPS-like ISA

– Two source tags – One destination tag

• Not all instructions use two source operands– Eg, addi $1, $2, 10

• Not all instructions produce a result that is interesting for scheduling– E.g., beq

• Some operands are ready when the instruction enters the scheduler

• Source: Efficient Dynamic Scheduling Through Tag Elimination, Dan Ernst and Todd Austin, ISCA 2002

Some Operands are Ready when the Instruction Enters the Scheduler

Window Specialization• Have reservation stations with different

source operand wait capabilities

Window Specialization• At rename check how many source operands

are not ready• If there is an appropriate slot proceed to

schedule• If not, stall at rename

• Advantages:– Destination bus only runs over reservation

stations with comparators– Load on the destination bus is reduced

• Disadvantages:– Stalls due to unavailability of reservation stations– Complexity of res. Station assignment

Window Specialization - Performance

Performance as IPC – Actual Clock Frequency not considered

Performance as IPC per ns

Last Tag Prediction• Observe:

– Instruction becomes ready after the last tag it waits for appears

• Last Tag prediction– Predict which of the two tags will that be

• Speculatively execute – Correct speculation: that was the last tag– Incorrect speculation:

• Need to reschedule• Detection? Try to read a value that is not

available

GShare-Style Last Tag Prediction

Two-bit saturating counters

Accuracy

• Over all instructions with two outstanding operands

Performance as IPC – Actual Clock Frequency not considered

Performance as IPC per ns

Prescheduling

Data-flow prescheduling for largeinstruction windows in out-of-order

processorsPierre Michaud, André Seznec,

HPCA 2001

Prescheduling

• Predict latencies• Put scheduled instructions into a FIFO• Slide into a smaller window

Prescheduling Method

Prescheduling Example

Latency Prediction

Latency Prediction Contd.

Broadcast Free Scheduler

Broadcast Free Scheduler• Cyclone design

– D. Ernst, A. Hamel, T. Austin– ISCA 2003

• Preschedule Instructions• Put them into a dual strip cyclical FIFO • Vertical paths allow for motion between

the strips

Cyclone ArchitectureWill be ready in cycle + 6

Cyclone Architecture – Cycle +1

Cyclone Architecture – Cycle + 2

Cyclone Architecture – Mis-scheduling

Estimate new latency

Pre-scheduler

Insert instruction with predicted latency N at the front of the FIFOHave it switch at N/2

Can only do two cascaded MAX calculationsDue to timing considerations

Cyclone IPC Performance

Cyclone True Performance and Area

Matrix Schedulers

Conventional Scheduler

WS requests

IW grants

Conventional Scheduler Timing

Source: A High-Speed Dynamic Instruction Scheduling Schemefor Superscalar ProcessorsMasahiro Goshima Kengo Nishino Yasuhiko Nakashima Shin-ichiro MoriToshiaki Kitamura Shinji TomitaMICRO 2001

Can’t pipeline without introducingBubbles between dependent Instructions:

Towards a Matrix Scheduler• Observe:

– In conventional scheduling dependences are discovered twice:

• Once at renaming• Once during scheduling

– Why? Dependences are implicitly represented

• Producer and Consumer link via a name• This is indirect

• Matrix Scheduler idea:– Represent dependences explicitly

Dependence MatrixW

Who do I depend upon?

Left source Right source

Matrix Scheduler

wakeup

Write port

Inserting an entry

wakeup

Write port

Wakeup

wakeup

Mispeculation Recovery• Do not cleanup• Use external logic to inhibit request

signals

Partial wakeup lines0.18um1.8V85C

Delay measurement points

Scheduling Priorities

Conflict Resolution• More instructions ready than available issue slots

– Which get to go?• Age vs. Pseudo-Random Resolution

• Age is important• Priority Enforcer picks the oldest

– Complex

Source:Matrix Scheduler ReloadedISCA 2007

Compacting Scheduler• Implemented in the Alpha 21264• Physical order within scheduler

corresponds to age• Entry freed:

– Shift up all younger entries

Virtual Physical Registers• Physical register names are used for two

purposes– Scheduling– Communicating

• A physical register is held much in advance than needed– We need the register only after the value is

produced• De-couple scheduling from

communication names

Used vs. Allocated Registers

Virtual Physical Registers

Deadlock• Older instruction completes later than

younger ones– No registers available

• Steal a register and re-execute

Performance vs. Physical Registers

Out-of-Order Execution Structures Optimizations

Documents

Optimizing CUDA. 2 © NVIDIA Corporation 2008 Outline Overview Hardware Memory Optimizations Execution Configuration Optimizations Instruction Optimizations

Control Structures Control structures control the flow of program execution. 3 types of control structures: sequence, selection repetition A sequence control

Improving compiler optimizations using Machine Learningudspace.udel.edu/bitstream/handle/19716/13442/2014_Kulkarni_Sameer_PhD.pdf · Some compiler optimizations are binary optimizations,

Design and Execution of Steel Structures and Composite Steel-concrete Buildings

Lecture 4 Control Structures MIT – AITI 2003. What are Control Structures? Control structures are a way to alter the natural sequence of execution in

Linear Optimizations

Execution of steel structures and aluminium . · PDF fileI.S. EN 1090 Execution of steel structures and aluminium structures. Seán Balfe Director, Sustainability and Built Environment

1718 IEEE TRANSACTIONS ON PARALLEL AND DISTRIBUTED …jrzhou/pub/periscope-journal.pdf · PeriSCOPE further leverages symbolic execution to enlarge the scope of such optimizations

Optimizations - Advanced

ExEcution of StainlESS StEEl StructurES · The European Standard EN 1090: Execution of steel structures and aluminium structures, Part 2: Technical requirements for the execution

Out-of-Order Execution Structures

Optimizations for Faster Execution of Esterel Programs

Optimizing CUDA – Part II - Nvidia · 2009-08-04 · Optimizing CUDA – Part II. Outline Execution Configuration Optimizations Instruction Optimizations Multi-GPU ... Register

Coming CEN and ISO standard for the execution of concrete ... ENV 13670 Execution of concrete structures Eurocode - 2 Design of concrete structures Eurocode - 0 Basis of structural

ExEcution of StainlESS StEEl StructurES - bssa.org.uk Execution of Stainless... · SCI DOCUMENT ED018 Execution of Stainless Steel Structures Technical Requirements and Model Clauses

Interprocedural Optimizations

1 Control Flow Graphs. 2 Optimizations Code transformations to improve program –Mainly: improve execution time –Also: reduce program size Can be done

Daydream: Accurately Estimating the Efﬁcacy of ...serailhydra/publications/daydream-atc20.pdf · optimizations. Daydream models DNN execution with a ﬁne-grained dependency graph

CONFORMITY ASSESSMENT FOR THE EXECUTION OF STEEL & ALUMINIUM STRUCTURES

Local Optimizations