Upload
cornelius-gilbert
View
215
Download
0
Tags:
Embed Size (px)
Citation preview
1
Software & the Concurrency Revolution by Sutter & Larus
ACM Queue Magazine, Sept. 2005For CMPS 5433 - Halverson
2
In a Nutshell
“The concurrency revolution is primarily a software revolution. The difficult problem is not building multicore hardware, but programming it in a way that lets mainstream applications benefit from the continued exponential growth in CPU performance.”
3
A look back at computing history
• ENIAC* – 1949• UNIVAC – 1951 @ US Census Bureau
– 1952 – 2 installations– 1953 – 3 installations– 1954 – 13 installations
• IBM – – Punch Card Systems (tabulations)– 1951 – IBM 701
• 1954: Fortran & Cobol established
4
Software Crisis• Rapid growth in
hardware • Little
development in software
• Consider Apple Computer in 1970’s–vs. IBM in the
1980’s
5
Current State of Parallel HW
• Distributed systems• Multicore chips• Graphics processing units (GPU)• Etc.• Software? Severely Lacking
– Similar to 1940’s to 1950’s– Programs may run slower now
6
Turning Point• Performance of programs will not
increase due to HW improvements
• Need for asynchronous processing
• Barriers– It’s hard; new way of thinking– Inadequate tools & languages– Difficult to find concurrency in some
applications
7
Concurrency a disruption?• Concurrency = high performance
– hw, sw, systems, languages w/ will survive
• Concurrent programming is difficult– Context-sensitivity & Synchronization
analysis• These are provably undecidable
• People don’t tend to think concurrently about programming
8
Programming Models• Granularity (fine – coarse*)
– Extent to which a problem is broken in to smaller parts
– Relative measure of the size of a task
• Degree of Coupling (loosely* – tightly)– Relative measure of dependence of tasks
• Communication• Synchronization
• Regularity– Regular vs. Irregular parallelism
9
3 Parallel Models
Independent Parallelism• Inherently, Embarassingly• Operations applied independently to
each data item • Fine grained, uncoupled
– E.G. A PU is assigned to each element of a 100 X 100 element array to double the value
• Coarse grained, uncoupled– E.G. web-based apps, multi-simulations
10
3 Parallel Models (cont’d)
Regular Parallelism• Apply same operation to collection of
data when computations are dependent– Synchronization or dependent results
• Fine grained– E.G. Array value becomes sum of 4
nhbrs.• Coarse grained
– E.G. Web apps with access to common DB
11
3 Parallel ModelsUnstructured Parallelism• Most general, least disciplined
– Threads w/ synchronization• Unpredictable access to shared
data• Requires explicit synchronization
– Messages• Via shared memory• Via message passing
12
Locks• Mechanism for protecting data/code
from conflicting or concurrent access (SW)– E.G. Semaphore, Monitor
• Standard locks don’t work in parallel– Not composable – deadlock– Standard libraries may quit working– Programmers MUST follow all rules!– Global vs. Local procedures– Local synchronization difficult
13
Lock Alternatives• Lock-free programming
– Use knowledge of memory to design d.s. not needing locking
– Difficult, fragile, still publishable• Transactional memory
– Language – ability to write atomic blocks
– Still in research
14
Goal for Programming Languages
“Higher-level language abstractions, including evolutionary extensions to current imperative languages, so that existing applications can incrementally become concurrent.”
• Make concurrency easy to understand– During development– During maintenance
15
3 Methods for Realizing Parallelism
• Explicit Programming• Implicit Programming• Automatic Parallelism
16
Explicit Programmingfor Parallelization
• Programmer states exactly where concurrency can occur
• ADV: Programmer can fully exploit concurrent potential
• DIS: Need higher-level language features & higher level of programming skill
17
Implicit Programmingfor Parallelization
• Concurrency hides in libraries or API’s; programmer maintains sequential view
• ADV: Inexperienced programmers can use concurrency
• DIS: Cannot realize all concurrency & difficult to design
18
Automatic Parallelization• Compiler extracts parallelism in
sequential program.• ADV: Does the work for us• DIS: Has not worked well in practice
– Hard for simple languages– Nearly impossible for complex
languages (pointers)– Sequential algorithms have little
concurrency
19
Imperative vs. Functional Languages
• Pascal, C, C++, C#, Java • Scheme, ML, Haskell• Still many issues to be resolved to
see if Functional Languages can provide needed features.
20
Abstraction• Low-level: Thread & Lock level
– Not a good building block, viewpoint• High-level: express tasks with
inherent concurrency, system schedules on HW– Easier transition to new HW– Easier for programmer
21
High-Level Abstraction• Asynchronous call: non-blocking; call
made but caller keeps working• Future: mechanism for returning result
from an asynchronous call; a placeholder for the value
• Active object: non-standard; each object runs own thread; outside method calls are asynchronous
22
4 Needed Programming Tools
• Defect detection • Debugger• Bottleneck detection • Testing aids
23
Defect (Error) Detection• New types of errors• Race conditions & Livelocks
– Difficult to reproduce, non-deterministic
• Modularity & High-level abstraction help
• Cannot test modules independently– You can, but…
• Too many possible paths• Active research
24
Debuggers
• Loggers–Track Messages–Causality Trails–Reverse execution
25
Bottleneck Detection
• Lock Contention• Cache Coherence Overheads• Lock Convoys
26
Testing Aids
• Extend Coverage metrics–Not just of statement is
executed, but with what other concurrent statements
• Stress testing–Need extending–Needs determinism
27
REPEAT: In a Nutshell
“The concurrency revolution is primarily a software revolution. The difficult problem is not building multicore hardware, but programming it in a way that lets mainstream applications benefit from the continued exponential growth in CPU performance.”
28
Conclusion – Goals
• Parallel Apps will (again) run faster on new HW
• New Interfaces• Systems designers: focus on
concurrency– Operating systems– Languages– Abstractions