Upload
penelope-carpenter
View
221
Download
1
Tags:
Embed Size (px)
Citation preview
Recognizing Potential Parallelism
Intel Software College
Introduction to Parallel Programming – Part 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
2Recognizing Potential Parallelism
Intel® Software College
Objectives
At the end of this module you should be able to:
Define parallel computing
Explain why parallel computing is becoming mainstream
Explain why explicit parallel programming is necessary
Identify opportunities for parallelism in code segments and applications
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
3Recognizing Potential Parallelism
Intel® Software College
What Can We Do withFaster Computers?
Solve problems faster
Reduce turn-around time of big jobs
Increase responsiveness of interactive apps
Get better solutions in same amount of time
Increase resolution of models
Make model more sophisticated
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
4Recognizing Potential Parallelism
Intel® Software College
What Is Parallel Computing?
Attempt to speed solution of a particular task by
1. Dividing task into sub-tasks
2. Executing sub-tasks simultaneously on multiple processors
Successful attempts require both
1. Understanding of where parallelism can be effective
2. Knowledge of how to design and implement good solutions
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
5Recognizing Potential Parallelism
Intel® Software College
Why Parallel Computing?
“The free lunch is over.” —Herb Sutter
We want applications to execute faster
Clock speeds no longer increasing exponentially
10 GHz
1 GHz
100 MHz
10 MHz
1 MHz’79 ’87 ’95 ’03 ’11
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
6Recognizing Potential Parallelism
Intel® Software College
Clock Speeds Have Flattened Out
Problems caused by higher speedsExcessive power consumptionHeat dissipationCurrent leakage
Power consumption critical for mobile devices
Mobile computing platforms increasingly importantRetail laptop sales now exceed desktop salesLaptops may be 35% of PC market in 2007
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
7Recognizing Potential Parallelism
Intel® Software College
Execution Optimization
Popular optimizations to increase CPU speedInstruction prefetchingInstruction reorderingPipelined functional unitsBranch predictionFunctional unit allocationRegister allocationHyperthreading
Added sophistication more silicon devoted to control hardware
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
8Recognizing Potential Parallelism
Intel® Software College
Multi-core Architectures
Potential performance = CPU speed # of CPUs
Strategy:Limit CPU speed and sophisticationPut multiple CPUs (“cores”) on a single chip
Potential performancethe same
4
4
4
2
21
Speed
CPUs
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
9Recognizing Potential Parallelism
Intel® Software College
History of Parallel Computing, Part I
Multiple-processor systems supporting parallel computing
1960s: Experimental systems
1980s: Microprocessor-based commercial systems
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
10Recognizing Potential Parallelism
Intel® Software College
Old Dynamic of Parallel Computing
Parallel computersare expensive
There are not manyparallel computers
Most people do not learnparallel programming
Parallel computingnot mainstream
Parallel programmingis difficult
Parallel programmingenvironments are inadequate
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
11Recognizing Potential Parallelism
Intel® Software College
Sequential Language Approach
Problem has inherent parallelism
Programming language cannot express parallelism
Programmer hides parallelism in sequential constructs
Compiler and/or hardware must find hidden parallelism
Sadly, doesn’t work
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
12Recognizing Potential Parallelism
Intel® Software College
Alternative Approach: Programmer and Compiler Work Together
Problem has inherent parallelism
Programmer has way to express parallelism
Compiler translates program for multiple processors
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
13Recognizing Potential Parallelism
Intel® Software College
Nothing Radical about a Programmer/Compiler Team
Programmers of modern CPUs must take architecture and compiler into account in order to get peak performance
“…you can actively reorganize data and algorithms to take advantage of architectural capabilities…” Introduction to Microarchitectural Optimization for Itanium® 2 Processors, p. 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
14Recognizing Potential Parallelism
Intel® Software College
History of Parallel Computing, Part II
2004: Intel demos Montecito dual-core CPU
2006: Intel demos Clovertown quad-core CPU
Clovertown scalable to 32+ cores in a single package
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
15Recognizing Potential Parallelism
Intel® Software College
New Dynamic of Parallel Computing
PCs are parallel computers
Everyone has aparallel computer
More people learningparallel programming
Parallel programmingconsidered mainstream
Parallel programminggets easier
Parallel programmingenvironments improve
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
16Recognizing Potential Parallelism
Intel® Software College
Methodology
Study problem, sequential program, or code segment
Look for opportunities for parallelism
Try to keep all processors busy doing useful work
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
17Recognizing Potential Parallelism
Intel® Software College
Ways of Exploiting Parallelism
Domain decomposition
Task decomposition
Pipelining
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
18Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
First, decide how data elements should be divided among processors
Second, decide which tasks each processor should be doing
Example: Vector addition
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
19Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
20Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
21Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
22Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
23Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
24Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
25Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
26Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
27Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
28Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
29Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
30Recognizing Potential Parallelism
Intel® Software College
Domain Decomposition
Find the largest element of an array
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
31Recognizing Potential Parallelism
Intel® Software College
Task (Functional) Decomposition
First, divide tasks among processors
Second, decide which data elements are going to be accessed (read and/or written) by which processors
Example: Event-handler for GUI
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
32Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
33Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
CPU 0
CPU 2
CPU 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
34Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
CPU 0
CPU 2
CPU 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
35Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
CPU 0
CPU 2
CPU 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
36Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
CPU 0
CPU 2
CPU 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
37Recognizing Potential Parallelism
Intel® Software College
Task Decomposition
f()
s()
r()q()h()
g()
CPU 0
CPU 2
CPU 1
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
38Recognizing Potential Parallelism
Intel® Software College
Pipelining
Special kind of task decomposition
“Assembly line” parallelism
Example: 3D rendering in computer graphics
RasterizeClipProjectModel
Input Output
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
39Recognizing Potential Parallelism
Intel® Software College
Processing One Data Set (Step 1)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
40Recognizing Potential Parallelism
Intel® Software College
Processing One Data Set (Step 2)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
41Recognizing Potential Parallelism
Intel® Software College
Processing One Data Set (Step 3)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
42Recognizing Potential Parallelism
Intel® Software College
Processing One Data Set (Step 4)
RasterizeClipProjectModel
The pipeline processes 1 data set in 4 steps
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
43Recognizing Potential Parallelism
Intel® Software College
Processing Two Data Sets (Step 1)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
44Recognizing Potential Parallelism
Intel® Software College
Processing Two Data Sets (Time 2)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
45Recognizing Potential Parallelism
Intel® Software College
Processing Two Data Sets (Step 3)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
46Recognizing Potential Parallelism
Intel® Software College
Processing Two Data Sets (Step 4)
RasterizeClipProjectModel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
47Recognizing Potential Parallelism
Intel® Software College
Processing Two Data Sets (Step 5)
RasterizeClipProjectModel
The pipeline processes 2 data sets in 5 steps
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
48Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 1)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
49Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 2)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
50Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 3)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
51Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 4)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
52Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 5)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
53Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 6)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
54Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 7)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
55Recognizing Potential Parallelism
Intel® Software College
Pipelining Five Data Sets (Step 8)
Data set 0
Data set 1
Data set 2
Data set 3
Data set 4
CPU 0 CPU 1 CPU 2 CPU 3
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
56Recognizing Potential Parallelism
Intel® Software College
Dependence Graph
Graph = (nodes, arrows)
Node for each
Variable assignment (except index variables)
Constant
Operator or function call
Arrows indicate use of variables and constants
Data flow
Control flow
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
57Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #1
for (i = 0; i < 3; i++) a[i] = b[i] / 2.0;
b[0] b[1] b[2]
a[0] a[1] a[2]
///
2 2 2
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
58Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #1
for (i = 0; i < 3; i++) a[i] = b[i] / 2.0;
b[0] b[1] b[2]
a[0] a[1] a[2]
///
2 2 2
Domain decompositionpossible
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
59Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #2
for (i = 1; i < 4; i++) a[i] = a[i-1] * b[i];
b[1] b[2] b[3]
a[1] a[2] a[3]
***
a[0]
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
60Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #2
for (i = 1; i < 4; i++) a[i] = a[i-1] * b[i];
b[1] b[2] b[3]
a[1] a[2] a[3]
***
a[0]
No domain decomposition
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
61Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #3
a = f(x, y, z);b = g(w, x);t = a + b;c = h(z);s = t / c;
x
f
w y z
ab
g
t
c
s/
h
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
62Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #3
a = f(x, y, z);b = g(w, x);t = a + b;c = h(z);s = t / c;
x
f
w y z
ab
g
t
c
s/
h
Taskdecompositionwith 3 CPUs.
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
63Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #4
for (i = 0; i < 3; i++) a[i] = a[i] / 2.0;
a[0] a[1] a[2]
a[0] a[1] a[2]
///
2 2 2
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
64Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #4
for (i = 0; i < 3; i++) a[i] = a[i] / 2.0;
a[0] a[1] a[2]
a[0] a[1] a[2]
///
2 2 2
Domain decomposition
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
65Recognizing Potential Parallelism
Intel® Software College
Dependence Graph Example #5
for (i = 0; i < 3; i++) { a[i] = a[i] / 2.0; if (a[i] < 1.0) break;}
a[0] a[1] a[2]
a[0] a[1] a[2]
///
2 2 2
1
<
1
<
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
66Recognizing Potential Parallelism
Intel® Software College
Can You Find the Parallelism?
Resizing a photo
Searching a document for all instances of a word
Updating a spreadsheet
Compiling a program
Prefetching pages in a Web browser
Using a word processor to type a report
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
67Recognizing Potential Parallelism
Intel® Software College
Good/Bad Opportunities for aParallel Solution
Parallel Solution Easier Parallel Solution More Difficult or Even Impossible
Larger data sets Smaller data sets
Dense matrices Sparse matrices
Dividing space among processors
Dividing time among processors
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
68Recognizing Potential Parallelism
Intel® Software College
Speculative Computation in a Turn-Based Strategy Game
Make moves for distantAI-controlled countries
in parallel
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
69Recognizing Potential Parallelism
Intel® Software College
Risk: Unexpected Interaction
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
70Recognizing Potential Parallelism
Intel® Software College
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
71Recognizing Potential Parallelism
Intel® Software College
Orange Cannot Move a Ship that Has Already Been Sunk by Green
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
72Recognizing Potential Parallelism
Intel® Software College
Solution: Reverse Time
Must be able to “undo” an erroneous, speculative computation
Analogous to what is done in hardware after incorrect branch prediction
Speculative computations typically do not have a big payoff in parallel computing
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
73Recognizing Potential Parallelism
Intel® Software College
References
Richard H. Carver and Kuo-Chung Tai, Modern Multithreading: Implementing, Testing, and Debugging Java and C++/Pthreads/ Win32 Programs, Wiley-Interscience (2006).
Robert L. Mitchell, “Decline of the Desktop,” Computerworld (September 26, 2005).
Michael J. Quinn, Parallel Programming in C with MPI and OpenMP, McGraw-Hill (2004).
Herb Sutter, “The Free Lunch is Over: A Fundamental Turn Toward Concurrency in Software,” Dr. Dobb’s Journal 30, 3 (March 2005).
Copyright © 2006, Intel Corporation. All rights reserved.
Intel and the Intel logo are trademarks or registered trademarks of Intel Corporation or its subsidiaries in the United States or other countries. *Other brands and names are the property of their respective owners.
74Recognizing Potential Parallelism
Intel® Software College