Design with Threads - cs.brown.edu · PDF file4/6/2017 · I Can you safely do unsynchronized lookups in ArrayList? ... Basic operations for run-to-completion threads. ... classes

Announcements

Maps performance tipsI On server: Maintain DB connections, prepared statements (per

thread/request!)F Use Spark.before, Spark.after to open and close.F Use ThreadLocal<T>, or pass the connection around.

I On client: Minimize redraws (not on every mouse event!).F http://benalman.com/projects/jquery-throttle-debounce-plugin/

I Java profiler is jvisualvm, consider YourKit.I There is a JS profiler in the Chrome dev tools.

John Jannotti (cs32) Design with Threads April 5, 2018 1 / 31

Design with Threads

John Jannotti

/course/cs0320/www/docs/lectures/

April 5, 2018


A mental model for synchronized blocks

1 s y n c h r o n i z e d ( bank ) {2 d o u b l e b a l a n c e = bank . g e t ( ” John ” ) ;3 b a l a n c e += 2 3 1 . 2 3 ;4 bank . put ( ” John ” , b a l a n c e ) ;5 }

becomes (CAPITALS == built in “magic”)

1 LOCK l o c k = bank .GETLOCK ( ) ;2 t r y {3 l o c k . OBTAIN ( )4 d o u b l e b a l a n c e = bank . g e t ( ” John ” ) ;5 b a l a n c e += 2 3 1 . 2 3 ;6 bank . put ( ” John ” , b a l a n c e ) ;7 } f i n a l l y {8 l o c k . RELEASE ( ) ;9 }


Java’s synchronized methods

synchronized keyword in front of method.

1 s y n c h r o n i z e d v o i d d e p o s i t ( S t r i n g name , d o u b l e amount ) {2 d o u b l e b a l a n c e = t a b l e . g e t ( name ) ;3 b a l a n c e += amount ;4 t a b l e . put ( name , b a l a n c e ) ;5 }

Equivalent to synchronized (this) for body.

1 v o i d d e p o s i t ( S t r i n g name , d o u b l e amount ) {2 s y n c h r o n i z e d ( t h i s ) {3 d o u b l e b a l a n c e = t a b l e . g e t ( name ) ;4 b a l a n c e += amount ;5 t a b l e . put ( name , b a l a n c e ) ;6 }7 }

What about a static synchronized method?John Jannotti (cs32) Design with Threads April 5, 2018 4 / 31

Examples

ListI add()/remove() in synchronized regionsI Can you safely do unsynchronized lookups in ArrayList?I Iteration?I What is the difference between Vector & ArrayList?

F Prefer Collections.synchronizedList(alist);

Maps, Sets, and other collectionsI Unsynchronized by default.I Map sm = Collections.synchronizedMap(m);I Consider java.util.concurrent.Concurrent(Hash)MapI Avoids corruption, but { get(); ...; put(); } still isn’t atomic!


Dining Philosophers

Five diners (threads) atround table.

I Need 2 forks to eat.I Only 5 on table.

Each picks up left (acquireslock).

I Then the right (acquireslock).

I Then eats (does work).I Then puts both down

(releases locks).

What can go wrong?

How can you ensure itwon’t?


Deadlocks

Thread 1 Thread 2

Lock A . . .. . . Lock B

Lock B . . .wait Lock Await wait

Doesn’t just affect locksI Wait for I/O from another thread/process.I Trying to acquire a resource from a “pool”.I Think about it when waiting on anything.


Avoiding Deadlocks (and Lock Contention)

Limit the number of locks in your programsI By minimizing writable sharing, not by crossing your fingers!

Try to avoid the need to hold multiple locks simultaneously.

If you must hold multiple locks, order them.

Hold locks “briefly” (as measured by time and code)I Keep synchronized regions short.I Minimize calls inside locked regions.I Avoid I/O inside locked regions.I “Briefly” is about helping you understand your code and when locks

are held, not about lower the probability of “bad luck.”


Using Threads

Getting the most out of threads is tricky.

Figuring out where to use threads is hard.I Some things are obvious.I But most potential uses aren’t.

Bottom lineI Use threads where it makes design easier, or you know you have a

performance problem.I Minimize (mutable) data sharing.I Consider “message passing” designs


Read/Write Locks

java.util.concurrent.locks.ReadWriteLock

readLock(), writeLock()

Multiple simultaneous readers allowed.

Only one writer (with no reader) allowed.

Useful for read-heavy workloads (a common situation)


Thread Coordination

Suppose you build your KD-Tree with multiple threads.I At each split, if there’s enough work, “fork” a thread.I When can you do a lookup? (That is, when are the threads done?)

Can be coordinated with synchronized regions.

There are more direct constructs, and you should use them.


Fork/Join

Basic operations for run-to-completion threads.I Fork: create and start a new threadI Join: wait for a thread to exit

“Structured” in some languagesI Fork/join are visible in the lexical structure of the program.I A forked section might look like a special block.

Java is unstructuredI Fork: Thread t = new Thread(work); t.start()I Join: t.join() or t.join(timeout)

How would you join multiple threads?

PS. Java 7 introduced a Fork/Join Framework that you mightconsider.


Monitors

Used when conditions control access, not just mutual exclusion.

We’ll describe what the Java language provides.

But also be aware of what the standard libraries offer.I java.util.concurrentI com.google.common.util.concurrent

Java provides basic control primitivesI Waiting (for condition to hold)I Notifying (of change in condition)I Java does not provide the parenthetical semantics on these operations.


Wait-Notify

A thread wants to yield CPU until an event occurs

Coordination without termination (unlike join())I Wait for another thread to provide needed data.I Wait for user to push a button.

Java provides this in a limited way.I Event must be explicitly noticed by another thread.I That other thread needs to “tell” the waiting thread about it.

Java ImplementationI Wait by calling o.wait() or o.wait(timeout)I Another thread must call o.notifyAll() (on the same object).

F Or o.notify() (in any doubt? use notifyAll)F stackoverflow.com/questions/37026/java-notify-vs-notifyall-all-over-

again


Wait-Notify to implement a Monitor

Details are slightly trickyI Java allows wait() on an object, not a “condition”

F obj.wait() must occur while synchronized on obj.

I Notify must occur on the same object.F And also while synchronized by that object.

Simplified code pattern

1 s y n c h r o n i z e d ( o b j ) {2 w h i l e ( ! c o n d i t i o n ) {3 o b j . w a i t ( ) ;4 }5 }


Producer-Consumer

ConsumerI Gets next item from queue.I Processes that item.

1 w h i l e ( t r u e ) {2 s y n c h r o n i z e d ( queue ) {3 w h i l e ( queue . i sEmpty ( ) ) {4 t r y {5 queue . w a i t ( ) ;6 }7 c a t c h ( I n t e r r u p t e d E x c e p t i o n e ) { }8 }9 n e x t = queue . remove ( ) ;

10 }11 n e x t . p r o c e s s ( ) ;12 }


Producer-Consumer

ProducerI Adds items to queue

1 w h i l e ( t r u e ) {2 i t em = <g e t n e x t item>3 s y n c h r o n i z e d ( queue ) {4 queue . add ( i tem ) ;5 queue . n o t i f y ( ) ;6 }7 }


Building up from Primitives

Now you know all the primitives you need.

But actually, you should rarely use them!

Get to know the java.util.concurrent package.I Executors — more flexible than raw ThreadsI Concurrent Data StructuresI A variety of lock types / synchronizationI Written by smart, focused people.

We’ll motivate some of these utilitiesI and implement some (perhaps poorly)I so you never should!


Raw threads can be inflexible

Consider a Web Server or Ray Tracing Engine.

Both are “embarassingly parallel”I Just start one thread per client connection!I Just break the image into tiles!

And yet. . .I Too many clients. . . nobody served.I How many tiles?


Separate specification of work from thread management

You already wrote your code as Runnables.

Instead of (new Thread(r)).start();

Generalize with an Executor.

1 E x e c u t o r e = <Get E x e c u t o r I mp le m en t at i on >;2 e . e x e c u t e ( r ) ;

Allows flexible execution.I Serial, Pools, Scheduled. . .

How would you implement a ThreadPoolExecutor?I What does ThreadPoolExecutor.execute(Runnable) do?I What will be shared among threads, and how will you protect it?


WorkerThread run()

1 p u b l i c v o i d run ( ) {2 w h i l e ( t r u e ) {3 Runnable t a s k = n u l l ;4 s y n c h r o n i z e d ( work queue ) {5 w h i l e ( work queue . i sEmpty ( ) ) {6 t r y {7 work queue . w a i t ( ) ;8 } c a t c h ( I n t e r r u p t e d E x c e p t i o n e ) { }9 }

10 t a s k = work queue . remove ( ) ;11 }12 t a s k . run ( ) ; /∗ Why not two l i n e s up ? ∗/13 }14 }


Handling Errors

What if the internal run() throws an exception?I Do you want to stop the worker?I Do you want to record the report/problem?

1 t r y {2 t a s k . run ( ) ;3 } c a t c h ( Throwable t ) {4 // l o g an e r r o r ( and s t a c k t r a c e ?)5 }

Be sure to catch Throwable.

Runnable doesn’t support returning values (or errors).


Adding to the queue()

1 p u b l i c v o i d e x e c u t e ( Runnable t a s k ) {2 s y n c h r o n i z e d ( work queue ) {3 work queue . add ( t a s k ) ;4 work queue . n o t i f y A l l ( ) ;5 }6 }

Together, these are the Monitor code pattern.

Specifically Producer/Consumer.


Producer/Consumer

The “monitor” code fragment is not very clear.A BlockingQueue is a nicer abstraction.

1 p u b l i c v o i d run ( ) {2 w h i l e ( t r u e ) {3 t r y {4 Runnable t a s k = work queue . t a k e ( ) ;5 t a s k . run ( ) ; /∗ No w o r r i e s about b e i n g s low . ∗/6 } c a t c h ( I n t e r r u p t e d E x c e p t i o n e ) { }7 }8 }9 p u b l i c v o i d e x e c u t e ( Runnable t a s k ) {

10 t r y {11 work queue . put ( t a s k ) ;12 } c a t c h ( I n t e r r u p t e d E x c e p t i o n e ) { }13 }


Task completion and returning values

There’s no longer a Thread to join().

How can we tell if our tasks are complete?I Do you care about individual tasks?I Or maybe just that all of them are done.

Ideas?


Several designs

Query the queue size (all).I Only if you know all tasks submitted.I Queue would need to “lie” about the currently running task.

Use a CountDownLatch (n).

Provide a “better” Runnable implementation that wraps Runnables.I Make a class that “knows” it has been run.I void execute(DoneRunnable task)I task.isDone()I JDK: void execute(RunnableFuture<T> task)

Use a richer execute() interface.I Future<T> submit(Callable<T> task)I Future and Callable are real. Speculate on their interface?I Available from an ExecutorService


Concurrent Datastructures

Suppose you want to use a Map or Queue in several threads.

You could use Collections.synchronizedMap (et al.)

ButI Those make every method synchronizedI And still require external synchronization for common ops.

1 s y n c h r o n i z e d (map) {2 i f ( ! map . c o n t a i n s K e y ( key ) )3 v a l u e = <some l e n g t h y computat ion >;4 r e t u r n map . put ( key , v a l u e ) ;5 e l s e6 r e t u r n map . g e t ( key ) ;7 }

ConcurrentHashMap offers putIfAbsent and even computeIfAbsent.


Concurrency utilities to the rescue

ConcurrentMap, ConcurrentQueue interfaces.

ConcurrentHashMap, ConcurrentLinkedQueue. . . classes.

Allow multiple readers (and sometimes writers)

oldVal = map.putIfAbsent(key, value);

newVal = map.computeIfAbsent(key, Tile::new)


Simple atomic ooperations

A common shared variable is a statistics counter.

Multiple threads need to increment atomically.

java.util.concurrent.atomic.AtomicInteger


Immutable Objects

Consider a Complex number package.

1 p u b l i c v o i d add ( o t h e r ) {2 r e a l += o t h e r . r e a l ;3 imag += o t h e r . imag ;4 }

vs

1 p u b l i c Complex add ( Complex o t h e r ) {2 r e t u r n new Complex ( r e a l + o t h e r . r e a l ,3 imag + o t h e r . imag ) ;4 }


Synchronization and Files

Files are not inherently thread-safe.I I/O may not be atomic.I POSIX specifies some guarantees, but libraries obfuscate details.I And file position is shared across threads

Try to access files with only one thread

Or use synchronization (on the same object!)I Not like this: synchronized (new File("safe.txt")) { ... }

Don’t worry about pure-read workloads.

Log files are problematic (maybe you don’t mind).I OS supports opening in “append” mode.I new FileWriter(..., true)


Documents

Design with Threads - cs.brown.edu · PDF file4/6/2017 · I Can you safely do unsynchronized lookups in ArrayList? ... Basic operations for run-to-completion threads. ... classes