Tornado: Maximizing Locality and Concurrency in a Shared Memory Multiprocessor Operating System Ben...

Tornado: Maximizing Locality and Concurrencyin a Shared Memory Multiprocessor Operating

System

Ben Gamsa, Orran Krieger, Jonathan Appavoo, Michael Stumm

By : Priya Limaye

Locality

• What is Locality of reference?

Locality

• What is Locality of reference?

sum = 0; for (int i = 0; i < 10; i ++) {

sum = sum + number[i]; }

Locality

• What is Locality of reference?

sum = 0; for (int i = 0; i < 10; i ++) {

sum = sum + number[i]; }

Temporal Locality Recently accessed data and instruction are likely to be

accessed in near future

Locality

• What is Locality of reference?

sum = 0; for (int i = 0; i < 10; i ++) {

sum = sum + number[i]; }

Spatial LocalityData and instructions close to recently accessed data and instructions are likely to be accessed in the near

future.

Locality

• What is Locality of reference?– Recently accessed data and instructions and

nearby data and instructions are likely to be accessed in the near future.

– Grab a larger chunk than you immediately need– Once you’ve grabbed a chunk, keep it

Locality in multiprocessor

• Computation depends on data local to processor– Each processor uses data from its own cache– Once data is brought in cache it stays there

Locality in multiprocessor

Memory

CPU

Cache

CPU

Cache

Counter

Counter: Shared

Memory

CPU CPU

0

Counter: Shared

Memory

CPU

0

CPU

0

Counter: Shared

Memory

CPU

1

CPU

1

Counter: Shared

Memory

CPU

1

CPU

1

1

Read : OK

Counter: Shared

Memory

CPU CPU

2

2

Invalidate

Comparing counter 1. Scales well with old

architecture2. Performs worse with shared

memory multiprocessor

Counter: Array

• Sharing requires moving back and forth between CPU Caches

• Split counter into array • Each CPU get its own counter

Counter: Array

Memory

CPU CPU

0 0

Counter: Array

Memory

CPU

1

CPU

1 0

Counter: Array

Memory

CPU

1

CPU

1

1 1

Counter: Array

Memory

CPU

1

CPU

1

1 1

CPU

2

Read Counter

Add All Counters

(1 + 1)

Counter: Array

• This solves the problem • What about performance?

Comparing counter Does not perform better than ‘shared counter’.

Counter: Array

• This solves the problem • What about performance?• What about false sharing?

Counter: False Sharing

Memory

CPU CPU

0,0

Counter: False Sharing

Memory

CPU

0,0

CPU

0,0

Counter: False Sharing

Memory

CPU

0,0

CPU

0,0

0,0

Sharing

Counter: False Sharing

Memory

CPU

1,0

CPU

1,0

Invalidate

Counter: False Sharing

Memory

CPU

1,0

CPU

1,0

1,0

Sharing

Counter: False Sharing

Memory

CPU CPU

1,1

1,1

Invalidate

Solution?

• Use padded array• Different elements map to different locations

Counter: Padded Array

Memory

CPU CPU

00

Counter: Padded Array

Memory

CPU

1

CPU

1

11

Update independent of each other

Comparing counter Works better

Locality in OS

• Serious performance impact• Difficult to retrofit• Tornado– Ground up design– Object Oriented approach – Natural locality

Tornado

• Object Oriented Approach• Clustered Objects• Protected Procedure Call• Semi-automatic garbage collection– Simplified locking protocol

Object Oriented Approach

Process 1

Process 2

…

Process Table

Object Oriented Approach

Process 1

Process 2

…

Process Table

Process 1

Lock

Object Oriented Approach

Process 1

Process 2

…

Process Table

Process 1

Lock

Process 2

Object Oriented Approach

Process 1

Process 2

…

Process Table

Process 1

Lock

Process 2

Lock

Object Oriented Approach

Class ProcessTableEntry{datalock

code}

Object Oriented Approach

• Each resource is represented by different object

• Requests to virtual resources handled independently– No shared data structure access– No shared locks

Object Oriented Approach

Process

Page Fault Exception

Object Oriented Approach

Process

Page Fault Exception

Region

Region

Object Oriented Approach

Process

Page Fault Exception

Region

Region

FCM

FCM

FCM File Cache Manager

Object Oriented Approach

HAT

Process

Region FCM

Region FCM

HAT Hardware Address TranslationFCM File Cache Manager

Search for responsible region

Page Fault Exception

Object Oriented Approach

Process

Page Fault Exception

Region

Region

FCM

FCM

COR

COR

DRAM

FCM File Cache ManagerCOR Cached Object RepresentativeDRAM Memory manager

Object Oriented Approach

• Multiple implementations for system objects• Dynamically change the objects used for

resource• Provides foundation for other Tornado

features

Clustered Objects

• Improve locality for widely shared objects• Appears as single object– Composed of multiple component objects

• Has representative ‘rep’ for processors– Defines degree of clustering

• Common clustered object reference for client

Clustered Objects

Clustered Objects : Implementation

Clustered Objects : Implementation

• A translation table per processor– Located at same virtual address– Pointer to rep

• Clustered object reference is just a pointer into the table

• ‘reps’ created on demand when first accessed– Special global miss handling object

Counter: Clustered Object

Counter – Clustered Object

CPU CPU

rep 1 rep 1

Object Reference

Counter: Clustered Object

Counter – Clustered Object

CPU

1

CPU

1

rep 1 rep 1

Object Reference

Counter: Clustered Object

Counter – Clustered Object

CPU

2

CPU

1

rep 2 rep 1

Object Reference

Update independent of each other

Clustered Objects

• Degree of clustering• Multiple reps per object – How to maintain consistency ?

• Coordination between reps– Shared memory– Remote PPCs

Counter: Clustered Object

Counter – Clustered Object

CPU

1

CPU

1

rep 1 rep 1

Object Reference

Counter: Clustered Object

rep 1 rep 1

Object Reference

Counter – Clustered Object

CPU

1

CPU

1

CPU

rep 1 rep 1

Read Counter

Counter: Clustered Object

rep 1 rep 1

Object Reference

Counter – Clustered Object

CPU

1

CPU

1

CPU

2

rep 1 rep 1

Add All Counters

(1 + 1)

Clustered Objects : Benefits

• Facilitates optimizations applied on multiprocessor e.g. replication and partitioning of data structure

• Preserves object-oriented design• Enables incremental optimizations• Can have several different implementations

Synchronization

• Two kinds of locking issues– Locking– Existence guarantees

Synchronization: Locking

• Encapsulate locking within individual objects• Uses clustered objects to limit contention• Uses spin-then-block locks– Highly efficient– Reduces cost of lock/unlock pair

Synchronization: Existence guarantees

• All references to an object protected by lock– Eliminates races where one thread is accessing the

object and another is deallcoating it• Complex global hierarchy of locks• Tornado - semi automatic garbage collection– Clustered object reference can be used any time– Eliminates needs for locks

Garbage Collection

• Distinguish between temporary references and persistent references– Temporary: clustered references held privately– Persistent: shared memory, can persist beyond

lifetime of a thread

Garbage Collection

• Remove all persistent references– Normal cleanup

• Remove all temporary references– Event driven kernel– Maintain counter for each processor – Delete object if counter is zero

• Destroy object itself

Garbage Collection

2 5 9

Process 1

Read

Garbage Collection

2 5 9

Process 1

Read

Counter ++

Garbage Collection

2 5 9

Process 1

Read

Counter = 1Process 2

Delete

Garbage Collection

2 5 9

Process 1

Read

Counter = 1Process 2

Delete

GC

If counter = 0

Garbage Collection

2 5 9

Process 1

Counter-- Process 2

Garbage Collection

2 9

Process 1

Counter = 0Process 2

GC

If counter = 0

Interprocess communication

• Uses Protected Procedure Calls• A call from client object to server object– Clustered object call that crosses protection

domain of client to server• Advantages– Client requests serviced on local processor– Client and server share processors similar to

handoff scheduling– Each client request has one thread in server

PPC: Implementation

• On demand creation of server threads• Maintains list of worker threads• Implemented as a trap and some queue

manipulations– Dequeue worker thread from ready workers – Enqueue caller thread on the worker– Return from-trap to the server

• Registers are used to pass parameters

Performance

Performance: summary

• Strong basic design• Highly scalable• Locality and locking overhead are major

source of slowdown

Conclusion

• Object-oriented approach and clustered objects exploits locality and concurrency

• OO design has some overhead, but these are low compared to performance advantages

• Tornado scales extremely well and achieves high performance on shared-memory multiprocessors

References

• http://web.cecs.pdx.edu/~walpole/class/cs510/papers/05.pdf

• Presentation by Holly Grimes, CS 533, Winter 2008

• http://en.wikipedia.org/wiki/Locality_of_reference