Copyright © 2011 Curt Hill MIMD Multiple Instructions Multiple Data

Copyright © 2011 Curt Hill

MIMD

Multiple Instructions Multiple Data

Multiprocessors• Tightly coupled CPUs

– Shared memory system– Share virtual memory as well

• Any threaded program operates in this mode

• Communication is usually through memory

• Example:– Word typing with one thread– Examining spelling in another


Interconnections

• If the number of processors is small the bus is the interconnection device

• Small is typically less than 16-32• When no longer small contention for

the bus will reduce performance• For larger sizes there must be

separate buses and interconnections between them


Cache Complications

• Even if the memory is truly shared such as on a bus we have problems with caches

• The problem is that two CPUs have two different caches and that the cache lines are different for the same memory location– Easy to do with write back policy– Not that hard with write through


Cache Coherence• Coherent caches disallow different

cache lines for same memory locations

• There are several cache coherence protocols but the results should be similar– The caches agree with each other

when they possess the same memory location

• One such protocol is called MESI


Multiple memories

• With multiple bus interconnections each CPU will have memory

• This memory is to be shared with all the others

• The memories are patched together to provide the illusion that it is just one

• This can be done with a variety of switches– Crossbar or multistage


Memory Hierarchies

• There is a limit how many CPUs may be added using the various switches

• Hardware costs increase as well• NonUniform Memory Access • NUMA architectures have quicker

access to local memory and slower to non-local

• Cache coherent NUMAs complicate things even more


Multicomputers

• Often loosely coupled CPUs– Always has some private memory– May have shared memory via a

portion of their virtual memory space– Distributed memory system

• Must pass messages via high speed networking

• Allows larger scaling than multiprocessors


MPP• Massively Parallel Processors• Typically commodity CPUs, such as

Pentiums– Hundreds or thousands of them

• Very high-speed interconnection network– Low latency, high bandwidth message

passing

• Very large I/O capabilities– Massive computing needs massive data


Current King

• As of 11/2011 Japan’s K computer was on top of the super computer list

• Contains 705,024 SPARC64 cores• Reaches 10 Petaflops/sec

– 10 quadrillion floating point operations a second


Cray XT5

• A Cray XT5 at Oak Ridge yielded 1.7– Largest US super computer

• It was the king in 2009• Uses 37,333 AMD Opterons, each

with 6 cores


Tianhe Ia

• Currently second on Top 500 list• 14,336 Xeon X5670 processors

and 7,168 Nvidia Tesla M2050 GPUs

• 2.57 Petaflops


Cluster Computing

• Most cluster computing schemes use ordinary workstations

• They communicate over conventional LAN/WAN

• This is the software alternative to the Multicomputers

• Beowulf cluster is an example


Grid Computing

• Similar to cluster, but the running software does not dominate the machine

• Typically there is a central server that assigns tasks and receives results

• When the workstation is idle the grid program consumes the idle cycles

• Super computing for those with no budget


Examples• BOINC – Berkely Open Infrastructure

for Network Computing– Client for a variety of different projects– 5.6 Petaflops

• Folding@Home – Protein folding– 5 Petaflops

• SETI@Home – Search for Extra Terrestial Intelligence– 730 TeraFlops


Documents

Copyright © 2011 Curt Hill MIMD Multiple Instructions Multiple Data