View
213
Download
1
Category
Preview:
Citation preview
CS 470/570:Introduction to Parallel and Distributed
Computing
Lecture 1 Outline
• Course Details• General Issues in Parallel Software Development• Parallel Architectures• Parallel Programming Models
General Information
• Instructor: Thomas Gendreau• Office: 211 Wing• Office Phone: 785-6813• Office Hours: – Monday, Wednesday, Friday 2:15-3:15– Thursday 1:00-3:00 – Feel free to see me at times other than my office hours
• email: gendreau@cs.uwlax.edu• web: cs.uwlax.edu/~gendreau• Textbook: Introduction to Parallel Programming by Peter S.
Pacheco
Grading
• 240 points: 12 quizzes (best 12 out of 14)• 100 points: Assignments • 100 points Cumulative final exam – 4:45-6:45 Wednesday December 17
• 440 total points• A quiz will be given in class every Friday and on Wednesday
November 21st. No makeup quizzes will be given. Unusual situations such as multi-week illness will be handled on an individual basis.
General Issues/Terminology
• Identify possible parallelism• Match amount of parallelism to the architecture• Scalability• Synchronization
– Mutual exclusion
• Choose a parallel programming model• Performance
– Speedup– Amdahl’s law
• Data distribution
General Issues/Terminology
• Why study parallel programming now?– Evolution of computer architecture– Computer networks as potential parallel computers– Manage large data sets– Solve computationally difficult problems
General Issues/Terminology
• Parallelism in computer systems– Multiprogrammed OS– Instruction level parallelism (pipelines)– Vector processing– Multiprocessor machines• Multicores
– Networks of processors
Parallel Architectures
• Classics Taxonomy (Flynn)– Single Instruction Single Data Stream: SISD– Single Instruction Multiple Data Streams: SIMD– Multiple Instructions Multiple Data Streams: MIMD
SIMD
• Single control Unit • Multiple ALUs• All ALUs execute the same instruction on local
data
MIMD
• Multiple control units and ALUs • Separate stream of control on each processor• Possible organization– Shared Physical Address Space• Uniform or Non-uniform Memory Access• Cache Coherance issues
– Distributed Physical Address Space• Network of Computers
Parallel Programming Models
• How is parallelism expressed?– Process– Task– Thread– Implicitly
• How is information accurately shared among parallel actions?
• How are parallel actions synchronized?
Parallel Programming Models
• Shared Address Space Programming– Shared variables
• Message Based Programming– Send/receive values among cooperating processes
• Programming models are independent of the underlying architecture
• SPMD– Single Program Multiple Data
Shared Address Space Programming
• Processes• Threads • Directive Model
Unix Processes
• Heavyweight processes– Requires the creation of a copy of the parent’s
data space– Process creation has high overhead
• Example functions– fork– waitpid
Threads
• Lightweight process– Does not require the creation of a copy of the
parent’s data space
• Example pthread functions– pthread_create– pthread_join
Directive Based Parallel Programming
• Directives in source code tell compiler where parallelism should be used
• Threads are implicitly created• OpenMP– Compiler directives– #pragma omp contruct clause– #pragma omp parallel for
Message Based Parallel Programming
• Send/Receive messages to share values• Synchronous communication• Asynchronous communication• Blocking• Non-blocking
Recommended