Objectives • You will learn/review:
– What a process is – How to fork and wait for processes – What a thread is – How to spawn and join threads – How to handle threads that access shared objects
• Race conditions and prevention of them • Deadlocks and prevention of them
– How processes can communicate – How threads can communicate
2
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
3
Processes
• Program – Executable code
• Process – An instance of a program in execution – A process has its own distinct context
4
Process Context
• Context consists of: – Process id – Address space: TEXT, RODATA, DATA, BSS,
HEAP, STACK – Processor state: RIP, EFLAGS, RAX, RBX,
etc. registers
5
Concurrency Options
• To implement concurrency... • Option 1: Multiple processes
– Process P1 forks process P2 – P1 and P2 execute concurrently – P1 and P2 do not share objects
• P1 and P2 have (initially identical but) distinct memory spaces
– (Relatively) expensive context switching
6
STACK HEAP
DATA
BSS
STACK HEAP
DATA
BSS
RODATA
TEXT
PROCESS 1 PROCESS 2
IP REG IP REG
Concurrent Processes Running Same Program
Concurrent Processes
7
Process-Level Concurrency
• At system boot-up... – Several processes are created automatically – They run concurrently
• When you login... – ssh process forks a child process – Child process execs (overwrites itself with)
your login shell (e.g. bash) – ssh process and your login shell process run
concurrently
8
Process-Level Concurrency • When you execute your program as a
command... – Bash forks a child process – Child process execs (overwrites itself with) your
program – Bash process and your process run concurrently
• Your process can fork/exec additional processes as desired – Your process and child processes run
concurrently
9
Process-Level Concurrency
• COS 333 example… • CGI programming
– When HTTP server receives HTTP request: • HTTP server forks a child process • Child process execs (overwrites itself with) the
specified CGI program • Parent HTTP server process and CGI process run
concurrently
10
Forking Processes
• How to fork child processes? • See forking.c
– Parent process forks “blue” child process and “red” child process
– Three processes execute concurrently – Parent process may exit before child
processes • Malformed!!! • A parent should wait for its children to exit • A parent should reap its children
11
Waiting for Processes
• See waiting.c – Parent waits for (reaps) its children – Parent process is “blocked” until both children
exit – Proper pattern
12
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/
Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
13
Threads
• Thread – A flow of control within a process – A process contains one or more threads – Within a process, all threads execute
concurrently
14
Concurrency Options
• To implement concurrency… • Option 2: Multiple threads
– Within P1, thread T1 spawns thread T2 – T1 and T2 execute concurrently – T1 and T2 may share some of P1’s objects – (Relatively) inexpensive context switching
15
STACK
HEAP
DATA
BSS
RODATA
TEXT
THREAD 1 THREAD 2
IP REG
Concurrent Threads within Same Process
STACK
IP REG
Concurrent Threads
16
Thread-Level Concurrency
• COS 333 examples… • Java virtual machine process
– JVM process must execute given program – JVM process also must collect garbage – Main thread and garbage collector thread run
concurrently
17
Thread-Level Concurrency • COS 333 examples… • Assignment 2 server process
– Server process must communicate with client process for (relatively) long time periods
– Server process also must respond to other client processes
– Server process spawns distinct thread for each client request
– Main thread and “client handler” thread run concurrently
18
Thread-Level Concurrency
• COS 333 examples… • Assignment 4 browser process
– Browser process must fetch data – Browser process also must respond to user
input – Via AJAX, Browser main (UI) thread spawns a
child thread to fetch data – Browser main (UI) thread and “fetch data”
thread run concurrently
19
Spawning Threads
• The “main” thread runs at process startup – Other threads may run at process startup too
• The main thread can spawn other threads • Note terminology:
– One process “forks” another – One thread “spawns” another
20
Spawning Threads in Java
• See Spawning.java – To spawn a thread:
• Define a subclass of Thread • Override run() method • Create an object of that class
– To begin execution of a thread: • Call object’s start() method • start() calls run() • Don’t call run() directly
21
Spawning Threads in Java
• See RunnableInterface.java – Alternative way to spawn a thread:
• Define class that implements Runnable • Must define a run() method • Create a Thread object specifying Runnable
object as argument
22
Spawning Threads in Java
– To begin execution: • Call Runnable object’s start() method • start() calls run() • Don’t call run() directly
– Useful when class extends some class other than Thread
23
Spawning Threads in Python
• See spawning.py – (Much the same as Java) – No need for “Runnable interface” approach
• Python supports multiple inheritance
24
Joining Threads
• Main thread can join a child thread – Main thread can block until child thread
terminates • Terminology
– A parent process can “wait” for a child process
– A parent thread can “join” a child thread
25
Joining Threads in Java
• See Joining.java – thread.join()
• Blocks current thread until thread terminates • May throw InterruptedException • (Explained in Appendix)
26
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
28
Race Conditions
• Problem: – Threads can share objects – Danger if multiple threads update same object – Race condition
• Outcome depends upon thread scheduling
• See RaceCondition.java • See racecondition.py
29
Locking
• Solution: Locking – Current thread gets lock on shared object
• Thereby current thread has exclusive use of shared object
– Other threads cannot obtain lock on shared object until current thread releases lock
– Adds lots of overhead
30
• See LockInUser.java
– Current thread: “Give me the lock on sharedobj”
– Other threads cannot obtain the lock on sharedobj during execution of block
synchronized(sharedobj) { stmt; stmt; …}
User-Level Locking in Java
32
User-Level Locking in Python
• See lockinuser.py – self._lock = Lock()
• In BankAcct constructor • Creates a lock for the newly instantiated object
– self._bankAcct.getLock().acquire() • In each thread • “Give me the lock on the bankAcct object”
– self._bankAcct.getLock().release() • In each thread • “Release the lock on the bankAcct object”
33
• See LockInResource.java
• Is the same as:
• Each BankAcct object thus is thread-safe
public synchronized void method() {…}
public void method() { synchronized(this) {…} }
Java Resource-Level Locking
35
Python Resource-Level Locking
• See lockinresource.py – Same, except... – Locking is performed by BankAcct object
rather than by thread objects – Each BankAcct object thus is thread-safe
36
Locking Strategies
• Which locking approach is better? • Resource-level locking
– Generally safer; shared object is thread-safe • User-level locking
– Generally faster; avoids locking when unnecessary
– Maybe easier to understand?
37
Aside: Thread-Safe Collections
38
Vector myVector = new Vector(); Hashmap myMap = new Hashmap(); List<E> myList = new Collections.synchronizedList(new ArrayList<E>()); Map<K,V> myMap = new Collections.synchronizedMap(new HashMap<K,V>()); ConcurrentLinkedQueue<E> myQueue = new ConcurrentLinkedQueue<E>(); ConcurrentSkipListSet<E> mySet = new ConcurrentSkipListSet<E>(); ConcurrentHashMap<K,V> myMap = new ConcurrentHashMap<K,V>(); ConcurrentSkipListMap<K,V> myMap = new ConcurrentSkipListMap<K,V>();
Thread-safe collections in Java:
Aside: Thread-Safe Collections
39
from Queue import Queue myQueue = Queue()
Thread-safe collections in Python:
Conditions
• Problem: – Thread may need to wait for some condition
on a locked object to become true – Example:
• Withdraw thread must wait for bank account balance to be sufficiently large
• Solution: Conditions
40
Conditions
• Called from within locked code… • object.wait()
– Blocks current thread until it is notified – Releases the lock
• object.notifyAll() – Notifies all waiting threads
• That some significant event has occurred
– Thread then should re-check condition
41
consumer() while (! condition) wait(); // Do what should be done when // condition is true. producer() // Change condition. notifyAll();
Thread conditions pattern:
Conditions Pattern
42
Java Conditions
• See Conditions.java – Consumer method calls object.wait()
repeatedly until condition is true – Producer method calls object.notifyAll()
– Could handle conditions in users rather than in shared resource
43
Python Conditions
• See conditions.py – Condition wraps around lock – Could handle conditions in users rather than
in shared resource
44
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
45
Deadlock
• Problem: deadlock – Two threads: t1 and t2 – Two shared objects: o1 and o2 – t1 has lock on o1; needs lock on o2 – t2 has lock on o2; needs lock on o1 – Both threads block forever
46
(1) run() (2) run()
(3) aliceAcct.transferOneTo(bobAcct) (4) bobAcct.transferOneTo(aliceAcct)
(5) bobAcct.depositOne() (6) aliceAcct.depositOne()
Deadlock
transferThread1 transferThread2
aliceAcct bobAcct
Java Deadlock Example See Deadlock.java
47
Preventing Deadlock
• Conditions for deadlock – Mutual exclusion – Hold and wait – No preemption – Circular chain of events
• Solution – OS level: Negate one of the four conditions – App level: Negate “circular chain of events”…
49
(1) run() (2) run()
(4) alictAcct.transferOneTo(bobAcct) (8) bobAcct.transferOneTo(aliceAcct) (5) bobAcct.withdrawOne() (9) aliceAcct.withdrawOne()
(3) get lock (6) release lock
NoDeadlock
Mutex
transferThread1 transferThread2
Preventing Deadlock
50
(7) get lock (10) release lock
aliceAcct bobAcct
Java Preventing Deadlock
• See NoDeadlock1.java – Mutex object
• See NoDeadlock2.java – Static fields and static methods – Mutex object as a static field in BankAcct class
• See NoDeadlock3.java – Really no need for Mutex object! – Any object will suffice
• E.g., the object which represents the BankAcct class!
51
Python Preventing Deadlock
• See nodeadlock1.py – Mutex object
• See nodeadlock2.py – Static fields and static methods – Mutex object as a static field in BankAcct
class
52
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
53
Inter-Process Comm
• Facts: – Processes on different computers can
communicate via sockets • See Network lecture
– Processes on the same computer can communicate via pipes
• A Unix pipe is a queue for inter-process comm
54
Inter-Process Comm
• See childtochild.c – Parent process creates “producer” and
“consumer” child processes – Producer child process writes data to pipe – Consumer child process reads data from pipe
55
Inter-Process Comm
• See sortmore.c – Parent process forks two child processes – First child executes sort somefile
• With stdout redirected to pipe – Second child executes more
• With stdin redirected to pipe
56
Inter-Process Comm
• Generalization – Unix users routinely command the shell to fork
communicating concurrent processes • E.g. sort somefile | more
– sortmore.c is a hardcoded shell
57
Agenda
1. Process-level concurrency (C) 2. Thread-level concurrency (Java/Python) 3. Race conditions (Java/Python) 4. Deadlock (Java/Python) 5. Inter-process comm (C) 6. Inter-thread comm (Java/Python)
58
Inter-Thread Comm
• Facts: – Threads can communicate via shared objects – Communication via shared objects is
dangerous (race conditions, deadlock) – Often threads are in a producer/consumer
relationship • Is there a safer way for producer/
consumer threads to communicate?
59
Java Inter-Thread Comm
• See ProdConStream.java – PipedOutputStream and PipedInputStream • Allow producer thread to send data to consumer
thread – See also java.io.PipedWriter and PipedReader for sending Unicode characters
60
Python Inter-Thread Comm
• See prodconstream.py – No “stream” mechanism
• Can’t implement Unix pipes model • Can’t implement Java stream model • Instead...
– Queue class • Methods are “synchronized” • Producer thread “puts” to queue object • Consumer thread “gets” from queue object
61
Concurrency Commentary
• Process-level concurrency is: – Essential (Unix relies on it) – Safe (distinct processes share no data)
• Thread-level concurrency is: – Essential (garbage collection, network
pgmming, web pgmming, …) – Dangerous!!! (distinct threads can share
objects)
62
Concurrency Commentary
• Should methods be “synchronized” by default?
• When using thread-level concurrency, should we avoid shared objects?
• Should we use process-level concurrency instead of thread-level concurrency whenever possible?
• In the long run, is thread-level concurrency a passing phase?
63
Summary • We have covered:
– What a process is – How to fork and wait for processes – What a thread is – How to spawn and join threads – How to handle threads that access shared objects
• Race conditions and prevention of them • Deadlocks and prevention of them
– How processes can communicate – How threads can communicate
64
Interrupting Threads
• Bad idea to end a thread abnormally – May leave shared objects in inconsistent
states • How to force normal thread termination?
– That is, how to force normal return from run()?
66
Interrupting Threads
• Forcing normal thread termination – Define interrupted flag in thread object
• Initially set to false
– Define public thread.interrupt() method • Called (by main thread?) to set interrupted flag
to true – run() checks interrupted flag periodically
• true => return from run()
67
Interrupting Threads
• Problem – If thread is joining/waiting/sleeping, then… – Thread might not check flag for a long time
• Solution – Interrupting threads…
68
Java Interrupting Threads • b = thread.interrupted()
– Returns the value (True or False) of thread’s interrupted flag
• thread.interrupt() – Sets thread’s interrupted flag – If thread is joining, waiting, or sleeping...
• join(), wait(), or sleep() throws InterruptedException
• Thread.interrupted() – Checks interrupted flag of current thread
69
Java Interrupting Threads
• Suppose… – Main thread spawns thread t1 – Main thread later calls t1.interrupt() – What happens?
70
if t1 is running, then t1.interrupt() sets t1’s interrupted flag to True t1.run() calls b = t1.interrupted() at leading edge of its loop b is True => t1.run() returns, and thread terminates else // t1 is blocked (for joining, waiting, or sleeping) join(), wait(), or sleep() throws an InterruptedException t1 catches the InterruptedException Catch clause re-calls t1.interrupt()
Java Interrupting Threads
71
try { Thread.sleep(1000); } catch (InterruptedException e) { interrupt(); }
Pattern in thread object:
try { Thread.sleep(1000); } catch (InterruptedException e) { Thread.currentThread().interrupt(); }
Pattern in non-thread object:
public void run() { … while (…) { if (interrupted()) return; … } }
Pattern in thread object:
Python Interrupting Threads
• Not supported!!! • Threads cannot be controlled “from the
outside” • (Java is particularly good for multi-
threaded programming)
73
Daemon Threads
• User thread (as seen so far) – Example: thread that calls main() – Process exits when all user threads have
terminated • Daemon thread
– Example: garbage collector thread – Exists only to service other threads – Process can terminate with running daemon
threads
75
Java Daemon Threads
• thread.setDaemon(b) – Sets daemon status of thread to b (true or false)
• b = thread.isDaemon() – Returns daemon status of thread
• See Daemons.java – Daemon threads end when main thread ends
76
Python Daemon Threads
• thread.setDaemon(b) – Sets daemon status of thread to b (True or False)
• b = thread.isDaemon() – Returns daemon status of thread
• See daemons.py – Daemon threads end when main thread ends
77
Threads in C
• C language does not support threads • C standard library does not support
threads • Use non-standard “pthreads” library
– Not object-oriented; uses function pointers – Specify “-pthread” option to gcc command
79
Threads in C
• An error-handling module – See mypthread.h – See mypthread.c
• Spawning threads – See spawning.c
80
Threads in C
• Joining threads – See joining.c
• Locking – See lockinuser.c – See lockinresource.c
• Conditions – See conditions.c
81