CS Parallelism

Dr. Vicki H. Allan

2

Concurrency• Concurrency can occur at four levels:

1. Machine instruction level – may have both an adder and a multiplier that are used at the same time.

2. High-level language statement level – might have a loop (a,b,c) where c from one iteration and a from the next are executed at the same time.

3. Unit level: several methods execute together4. Program level: several program execute

together

3

Suppose we have two methods

Populate marsh{ for(i=0;i< 1000; i++){ create frog // high level language statement create carp create mosquitos}}Populate prehistoric world{ for (i=0;i<10,i++) create dinosaur(i)}

4

Concurrency can occur at four levels(termed granularity)

1. Machine instruction level – Create frog is decomposed into basic parts. If one basic instruction is to fold both sides into center, perhaps one “processor” folds the left side and one folds the right.

2. High-level language statement level - different parts of “make frog” happen together3. Unit level: populate marsh occurs with

populate prehistoric world4. Program level: several programs (to do other

things not shown here) execute together

5

What would be the advantages/disadvantages of

each type of parallelism?

6

1. Late 1950s - One general-purpose processor and one or more special-purpose processors for input and output operations

2. Early 1960s - Multiple complete processors, used for program-level concurrency

3. Mid-1960s - Multiple partial processors, used for instruction-level concurrency

4. Single-Instruction Multiple-Data (SIMD) machines. The same instruction goes to all processors, each with different data - e.g., vector processors

5. Multiple-Instruction Multiple-Data (MIMD) machines • Independent processors that can be synchronized (unit-level

concurrency)

The Evolution of Multiprocessor Architectures

7

Making a Frog

Fold in sides

8

Take lower corner andfold up to top. Repeat withother side.

Fold into middle Repeat

9

Examples

• SIMD - all do the same things at the same time. All fold; All Open; All fold again

• Pipelined – one person does fold, and then passes. Problems?

• MIMD – all do different things

10

11

• Def: A thread of control in a program is the sequence of program points reached as control flows through the program

• Categories of Concurrency:1. Physical concurrency - Multiple independent

processors (multiple threads of control)2. Logical concurrency - The appearance of

physical concurrency is presented by time-sharing one processor (software can be designed as if there were multiple threads of control)

What would be the advantage of logical concurrency?

Consider the TV remote as performing “context switch”.

Why does one switch between multiple programs?

What is downside to switch?

13

Example: Smart Remote Ads play when you are not

watching, assume “program” doesn’t continue when you aren’t

watching it

• You might be an E-mail Junkie..

• You might be a computer science major

• Attraction to computer scientists

14

Concerns?• Is switching between tasks confusing?

What would need to be retained?

• Is switching between tasks expensive? Would there be a minimal size at which you spawn more tasks?

• What is the gain?

15

• What is the gain?– Models actual situation better– response time– Use delays in processing

16

Why do we want parallelism?• Price-performance curves• Used to be – paid more for computer - got

more (linear relationship between price and performance).

• Now, for little money, get a lot of power. As you add more money, performance curve levels off. Not an efficient way to get more performance

• Parallelism is the answer – string cheap computers together to do more work.

17

What is a Thread ?

• Just as multitasking OS’s can run more than one process “concurrently”, a process can do the same by running more than a single thread.

• Each Thread is a different stream of control that can execute its instructions independently.

• Compared to a process, a thread is inexpensive to create, terminate, schedule or synchronize.

18

What is a Thread ?

• A process is a HEAVY-WEIGHT kernel-level entity. (process struct)

• A thread is a LIGHT_WEIGHT entity comprising the registers, stack and some other data.

• The rest of the process struct is shared by all threads. (address space, file desc, etc.)

• Most of the thread structure is at the user space allowing very fast access.

19

So for our example

• If we had two processes to populate the marsh and to populate the prehistoric world, each process would be able to stand alone.

• If we had two threads to populate the marsh and to populate the prehistoric world, they would have some shared resources (like the table or paper supply)

20

Concurrency Vs. Parallelism

• Concurrency means that two or more threads can be in the “middle” of executing code.

• Only one can be on the CPU though at any given time.

• Parallelism actually involves multiple CPUs running threads at the same time.

• Concurrency is the illusion of Parallelism

21

What can threads do that can’t be done by processes sharing memory ?

• Answer: Nothing !... If you have – plenty of time to kill programming,

– more time to kill processing,

– willing to burn money by buying RAM

• Debugging cross-process programs are tough.• In Solaris creating a thread is 30 TIMES FASTER

than forking a process.• Synchronization is 10 time faster with threads.• Context Switching - 5 times faster

22

What Applications to Thread?• Multiplexing (communicate two or more signals over

a common channel)– Servers

• Synchronous Waiting (definition?)– clients– I/O

• Event Notification• Simulations• Parallelizable Algorithms

– Shared memory multiprocessing– Distributed Multiprocessing

23

Which Programs NOT to thread?

• Compute bounds threads on a uniprocessor.

• Very small threads (threads are not free)

• Old Code

• Parallel execution of threads can interfere with each other.

• WARNING: Multithreaded applications are more difficult to design and debug than single threaded apps. Threaded programming design requires careful preparation !

24

Synchronization• The problem -

– Data Race - occurs when more than one thread is trying to update the same piece of data.

– Critical Section - Any piece of code to which access needs to be controlled.

• The Solution -– Mutex– Condition Variables– Operations - init, lock, unlock

25

MUTEX

• A MUTual EXclusion allows exactly one thread access to a variable or critical section of code.

• Access attempts by other threads are blocked until the lock is released.

26

• Kinds of synchronization:1. Cooperation – Task A must wait for task B to complete some

specific activity before task A can continue its execution e.g., You cut the paper and then I fold it.

2. Competition – When two or more tasks must use some resource

that cannot be simultaneously used e.g., we both want the scissors.

27

• Liveness means the unit will eventually complete its execution. I’m currently blocked from finishing my frog, but I will eventually get to finish.

• In a concurrent environment, a task can easily lose its liveness. You were supposed to wake me up when the scissors became available, but you forgot.

• If all tasks in a concurrent environment lose their liveness, it is called deadlock. I take the paper and wait for the scissors. You take the scissors and wait for the paper. Circular wait is deadlock.

28

Livelock: theoretically can finish, but never get the resources to finish.

How do you prevent deadlock?

How do you prevent livelock?

29

Questions?