Relaxed Shared Memory Relaxed Shared Memory Consistency ModelsConsistency Models

By Steven Davis & Seth GroderBy Steven Davis & Seth Groder

Multiple Processor Systems 053 Presented: 5/18/06Multiple Processor Systems 053 Presented: 5/18/06

OutlineOutline

What are Shared Memory Consistency ModelsWhat are Shared Memory Consistency ModelsStrict ModelsStrict ModelsSequential ModelsSequential ModelsRelaxed ModelsRelaxed Models

Motivation for Relaxing ModelsMotivation for Relaxing ModelsRequirements of a Relaxed Consistency ModelRequirements of a Relaxed Consistency ModelClasses of Relaxed ModelClasses of Relaxed ModelPopular Relaxed ModelsPopular Relaxed Models

Weak Ordering (WO)Weak Ordering (WO)Release Consistency (RC)Release Consistency (RC)

QuestionsQuestions

Shared Memory Consistency ModelsShared Memory Consistency Models

Specify a contract between programmer and system, wherein Specify a contract between programmer and system, wherein the system guarantees that if the programmer follows the the system guarantees that if the programmer follows the rules, memory will be consistent and the results of memory rules, memory will be consistent and the results of memory operations will be predictable.operations will be predictable.Specify the order by which shared memory access events of Specify the order by which shared memory access events of one process should be observed by other processes in the one process should be observed by other processes in the system.system.A consistency model affects both programming languages and A consistency model affects both programming languages and compilers as well as hardware.compilers as well as hardware.

Hardware and Compilers: The model indicates how much these Hardware and Compilers: The model indicates how much these components can reorder accesses. This limits the amount of components can reorder accesses. This limits the amount of optimization which can be achievedoptimization which can be achievedProgramming Language: The model must supply mechanisms to Programming Language: The model must supply mechanisms to introduce necessary constraints on the reordering of accesses.introduce necessary constraints on the reordering of accesses.

Classes of Consistency ModelsClasses of Consistency Models

Strict ConsistencyStrict ConsistencyA read always returns with most recent write to the A read always returns with most recent write to the same locationsame locationImplicitly assumes the existence of absolute global Implicitly assumes the existence of absolute global timetime

Strict Not Strict

Classes of Consistency ModelsClasses of Consistency Models

Sequential ConsistencySequential ConsistencyThe result of any execution appears the same as the result of The result of any execution appears the same as the result of the interleaving of individual programs strictly in sequential the interleaving of individual programs strictly in sequential program order.program order.This is the easiest consistency model to understand, since the This is the easiest consistency model to understand, since the system is preserving a model behaving in a way expected by an system is preserving a model behaving in a way expected by an average programmer.average programmer.Sequential consistency is weaker than atomic consistency (which Sequential consistency is weaker than atomic consistency (which would demand that operations are seen in order in which they would demand that operations are seen in order in which they were actually issued were actually issued -- which is essentially impossible to secure in which is essentially impossible to secure in distributed system, where deciding global time is virtually distributed system, where deciding global time is virtually impossible)impossible)Does not rely on global timeDoes not rely on global time

Sequential Consistency ExampleSequential Consistency Example

Classes of Consistency ModelsClasses of Consistency Models

Relaxed ConsistencyRelaxed ConsistencyImproves shared memory access performance while ensuring Improves shared memory access performance while ensuring correctness to a specified consistency modelcorrectness to a specified consistency modelAny Memory Consistency model which is weaker than Any Memory Consistency model which is weaker than Sequential ConsistencySequential Consistency

1.1. All accesses to synchronization variables are seen by all procesAll accesses to synchronization variables are seen by all processes (or ses (or nodes, processors) in the same order (sequentially) nodes, processors) in the same order (sequentially) -- these are these are synchronization operations. Accesses to critical sections are sesynchronization operations. Accesses to critical sections are seen en sequentially. sequentially.

2.2. All other accesses may be seen in different order on different pAll other accesses may be seen in different order on different processes rocesses (or nodes, processors). (or nodes, processors).

3.3. The set of both read and write operations in between different The set of both read and write operations in between different synchronization operations is the same in each process.synchronization operations is the same in each process.

Therefore, there can be no access to synchronization variable ifTherefore, there can be no access to synchronization variable ifthere are pending write operations. And there can not be any newthere are pending write operations. And there can not be any newread/write operation started if system is performing anyread/write operation started if system is performing anysynchronization operationsynchronization operation

Motivations for Relaxed ModelsMotivations for Relaxed Models

Limitations of Strict ConsistencyLimitations of Strict ConsistencyUnUn--optimizable, detrimental to performanceoptimizable, detrimental to performanceHighly RestrictiveHighly RestrictiveNot well suited for uniprocessor systems Not well suited for uniprocessor systems much less multiple processor systemsmuch less multiple processor systems

Motivations for Relaxed ModelsMotivations for Relaxed Models

Limitations of Sequential ConsistencyLimitations of Sequential ConsistencyBy preserving the sufficient conditions necessary to By preserving the sufficient conditions necessary to maintain sequential consistency the following critical maintain sequential consistency the following critical performance optimizations cannot be made:performance optimizations cannot be made:

Code MotionCode MotionCommonCommon--subexpression eliminationsubexpression eliminationSoftware pipeliningSoftware pipeliningRegister allocationRegister allocation

Simplicity is achieved at cost of efficiency: distributed Simplicity is achieved at cost of efficiency: distributed systems with sequential consistency model are, systems with sequential consistency model are, without further optimization, such as speculation, one without further optimization, such as speculation, one magnitude slower than those providing weaker magnitude slower than those providing weaker modelsmodels

Assumptions for BenchmarkingAssumptions for Benchmarking

Based on Stanford DASH multiprocessor

Coherent caches, directory based invalidation scheme

Latency = 1:25:100 processor cycles

Detailed simulation, contention modeled

16 processors

Performance BenchmarksPerformance Benchmarks

Performance Evaluation: ControlPerformance Evaluation: ControlSequential ConsistencySequential Consistency

Requirements of a Relaxed Requirements of a Relaxed Consistency ModelConsistency Model

Programmer’s InterfaceProgrammer’s Interface

Translation MechanismTranslation Mechanism

System SpecificationSystem Specification

Programmer’s InterfaceProgrammer’s Interface

A contract such that if the program follows A contract such that if the program follows certain highcertain high--level rules or provides enough level rules or provides enough program annotations, then any system on program annotations, then any system on which the program is run will always which the program is run will always guarantee a sequentially consistent execution.guarantee a sequentially consistent execution.

This execution is guaranteed regardless of the This execution is guaranteed regardless of the default reorderings permitted by the system default reorderings permitted by the system specifications it supports.specifications it supports.

Translation MechanismTranslation Mechanism

This translates the programmer’s This translates the programmer’s annotations to the interface (specifically, annotations to the interface (specifically, the orderthe order--preserving mechanisms) preserving mechanisms) exported by the system specification, so exported by the system specification, so that the system may do its job.that the system may do its job.

System SpecificationSystem Specification

A clear specification of two thingsA clear specification of two things1.1. What program orders among memory operations What program orders among memory operations

are guaranteed to be preserved in an observable are guaranteed to be preserved in an observable sense by the system, including whether or not write sense by the system, including whether or not write atomicity is maintained.atomicity is maintained.

2.2. If all program orders are not guaranteed to be If all program orders are not guaranteed to be preserved by default, then what mechanisms will preserved by default, then what mechanisms will the system provide for a programmer to enforce the system provide for a programmer to enforce order explicitly when desired.order explicitly when desired.

The System SpecificationThe System Specification

Three sets of modelsThree sets of models1.1. WriteWrite--toto--Read RelaxationRead Relaxation

Only allow a read to bypass (complete before) an Only allow a read to bypass (complete before) an earlier incomplete write in program orderearlier incomplete write in program order

2.2. WriteWrite--toto--Write Relaxation Write Relaxation Allow writes to bypass previous writesAllow writes to bypass previous writes

3.3. Complete Relaxation Complete Relaxation Allow reads Allow reads oror writes to bypass previous reads as writes to bypass previous reads as wellwell

Relaxing the WriteRelaxing the Write--toto--Read Read Program OrderProgram Order

Allows the hardware to hide the latency of write Allows the hardware to hide the latency of write operationsoperations

While the write miss is still in the write buffer (and While the write miss is still in the write buffer (and not yet visible to other processors) the processor can not yet visible to other processors) the processor can issue and complete reads that hit in its cache (or a issue and complete reads that hit in its cache (or a single read which misses)single read which misses)

Despite relaxing the writeDespite relaxing the write--toto--read ordering, the read ordering, the programmer’s intuition is decently preserved in programmer’s intuition is decently preserved in most implementations most implementations –– Even without any Even without any special operationsspecial operations

Benefits of Relaxing the WriteBenefits of Relaxing the Write--toto--Read Program OrderRead Program Order

Relaxing this order results in the hiding of Relaxing this order results in the hiding of write latencies and the benefits can be write latencies and the benefits can be substantial.substantial.Most processors are capable of Most processors are capable of implementing and taking advantage of this implementing and taking advantage of this relaxationrelaxation

Performance EvaluationPerformance EvaluationRelaxed WriteRelaxed Write--toto--Read ConsistencyRead Consistency

Relaxing WriteRelaxing Write--toto--Write Program OrderWrite Program Order

Allows writes to bypass earlier writes (to Allows writes to bypass earlier writes (to different locations)different locations)

This allows the write buffer to merge, and This allows the write buffer to merge, and even retire, writes before previous writes in even retire, writes before previous writes in the program order become visiblethe program order become visible

Reordering of writes can violate the Reordering of writes can violate the intuitive nature of sequential consistency intuitive nature of sequential consistency semanticssemantics

Benefits of Relaxing the WriteBenefits of Relaxing the Write--toto--Write Program OrderWrite Program Order

Enables multiple write misses to be fully Enables multiple write misses to be fully overlapped and to become visible out of overlapped and to become visible out of program orderprogram orderFurther reduces the impact of write Further reduces the impact of write latency on processor stall timelatency on processor stall timeMay improve communication efficiencyMay improve communication efficiency

Performance Evaluation:Performance Evaluation:Relaxed Write-to-Write Consistency:

Relaxing All Program OrdersRelaxing All Program Orders

No program orders are guaranteed by default No program orders are guaranteed by default other than data and control dependencies within other than data and control dependencies within a processa processThese models are particularly well matched to These models are particularly well matched to dynamically scheduled processors whose dynamically scheduled processors whose implementation allows them to proceed past implementation allows them to proceed past read misses to other memory referencesread misses to other memory referencesThe only model which allows many of the key The only model which allows many of the key reorderings and eliminations of accesses reorderings and eliminations of accesses necessary for many compiler optimizationsnecessary for many compiler optimizations

Benefits of Relaxing All Program Benefits of Relaxing All Program OrdersOrders

Multiple read requests can be outstanding Multiple read requests can be outstanding simultaneouslysimultaneouslyThese requests can also be bypassed by These requests can also be bypassed by later writes in program orderlater writes in program orderThese writes can complete out of program These writes can complete out of program orderorder

Thus read and write latencies are hidden.Thus read and write latencies are hidden.

Performance EvaluationPerformance EvaluationCompletely Relaxed ConsistencyCompletely Relaxed Consistency

Characteristics of Various System Characteristics of Various System SpecificationsSpecifications

Sparc V9 RMO, Digital Alpha, and IBM PowerPC are all processor specific implementations

Weak Ordering (WO)Weak Ordering (WO)

Developed by Dubois, Scheurich, and Briggs in 1986Developed by Dubois, Scheurich, and Briggs in 1986Uses synchronization to protect critical sectionsUses synchronization to protect critical sectionsSynchronization can be achieved through locks/unlocks Synchronization can be achieved through locks/unlocks or flagsor flags

Release Consistency (RC)Release Consistency (RC)

Developed by Gharachorloo in 1990Developed by Gharachorloo in 1990Developed because WO was not relaxing Developed because WO was not relaxing enough, divides synchronizations into acquires enough, divides synchronizations into acquires and releasesand releasesUses acquire (read) to delay memory accesses Uses acquire (read) to delay memory accesses after it until acquire is completedafter it until acquire is completedUses release (write) to allow access to new Uses release (write) to allow access to new values of data modified before the release is values of data modified before the release is completedcompleted

Weak Ordering vs. Release ConsistencyWeak Ordering vs. Release Consistency

ConclusionConclusion

As with any system, the ideal is the goal. As with any system, the ideal is the goal. By allowing for weaker or relaxed memory By allowing for weaker or relaxed memory models, system utilization is allowed to models, system utilization is allowed to approach 100%approach 100%Programmer has to pay for increased Programmer has to pay for increased performance by dealing with looser performance by dealing with looser constraintsconstraints

Questions?Questions?

References:References:Parallel Computer Architecture by David E. Culler, Jaswinder PalParallel Computer Architecture by David E. Culler, Jaswinder Pal Singh, and Anoop Singh, and Anoop GuptaGuptaDr. Shaaban’s Lecture NotesDr. Shaaban’s Lecture Noteshttp://projects.seas.gwu.edu/~hpcl/upcdev/tut/sld134.htmhttp://projects.seas.gwu.edu/~hpcl/upcdev/tut/sld134.htmhttp://www.cs.utah.edu/classes/cs7460/lectures/lecture10.txthttp://www.cs.utah.edu/classes/cs7460/lectures/lecture10.txthttp://user.it.uu.se/~arnoldp/cse3ms/consist/http://user.it.uu.se/~arnoldp/cse3ms/consist/http://phase.hpcc.jp/emx/isca95/tsld014.htmhttp://phase.hpcc.jp/emx/isca95/tsld014.htm