Concurrency and Multithreading

8/8/2019 Concurrency and Multithreading

1/28

Java Concurrency / Multithreading

This article describes how to do concurrentprogramming with Java. It covers the concepts of parallel programming, immutability, threads, theexecutor framework (thread pools), futures,callables and the fork-join framework.

Table of Contents

1. Concurrency

1.1. Overview 1.2. Process vrs. threads 1.3. Amdahl's Law 1.4. Concurrency issues

2. Concurrency in Java 2.1. Processes and Threads 2.2. Synchronized

2.3. Volatile 3. The Java memory model 3.1. Overview 3.2. Atomic operation 3.3. The "synchronized" keyword 3.4. The "volatile" keyword

4. Immutability and Defensive Copies 4.1. Immutability 4.2. Defensive Copies

5. Threads in Java 6. Threads pools with the Executor Framework 7. Futures and Callables 8. Nonblocking algorithms
http://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_processthreadshttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_amdahlhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_problemshttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjavahttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjava_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjava_synchronizedhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_volatilehttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodelhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_atomichttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_synchronizedhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_volatilehttp://www.vogella.de/articles/JavaConcurrency/article.html#immutabilityhttp://www.vogella.de/articles/JavaConcurrency/article.html#immutability_mainhttp://www.vogella.de/articles/JavaConcurrency/article.html#immutability_defensivecopyhttp://www.vogella.de/articles/JavaConcurrency/article.html#threddshttp://www.vogella.de/articles/JavaConcurrency/article.html#threadpoolshttp://www.vogella.de/articles/JavaConcurrency/article.html#futureshttp://www.vogella.de/articles/JavaConcurrency/article.html#nonblockinghttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_processthreadshttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_amdahlhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_problemshttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjavahttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjava_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrencyjava_synchronizedhttp://www.vogella.de/articles/JavaConcurrency/article.html#concurrency_volatilehttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodelhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_overviewhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_atomichttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_synchronizedhttp://www.vogella.de/articles/JavaConcurrency/article.html#memorymodel_volatilehttp://www.vogella.de/articles/JavaConcurrency/article.html#immutabilityhttp://www.vogella.de/articles/JavaConcurrency/article.html#immutability_mainhttp://www.vogella.de/articles/JavaConcurrency/article.html#immutability_defensivecopyhttp://www.vogella.de/articles/JavaConcurrency/article.html#threddshttp://www.vogella.de/articles/JavaConcurrency/article.html#threadpoolshttp://www.vogella.de/articles/JavaConcurrency/article.html#futureshttp://www.vogella.de/articles/JavaConcurrency/article.html#nonblocking


2/28

9. Fork-Join in Java 7 10. Deadlock 11. Thank you

12. Questions and Discussion 13. Links and Literature 13.1. Source Code 13.2. Concurrency Resources 13.3. vogella Resources

1. Concurrency1.1. Overview

Concurrency is the ability to run several parts of aprogram or several programs in parallel.Concurrency can highly improve the throw-put of aprogram if certain tasks can be performed

asynchronously or in parallel.Almost every computer nowadays has several CPU'sor several cores within one CPU. The ability toleverage theses multi-cores can be the key for asuccessful high-volume application.

1.2. Process vrs. threads

The distinction between processes and threads isimportant.

Process: A process runs independently andisolated of other processes. It cannot directlyaccess shared data in other processes. The
http://www.vogella.de/articles/JavaConcurrency/article.html#forkjoinhttp://www.vogella.de/articles/JavaConcurrency/article.html#dealockhttp://www.vogella.de/articles/JavaConcurrency/article.html#thankyouhttp://www.vogella.de/articles/JavaConcurrency/article.html#questionshttp://www.vogella.de/articles/JavaConcurrency/article.html#resourceshttp://www.vogella.de/articles/JavaConcurrency/article.html#sourcecodehttp://www.vogella.de/articles/JavaConcurrency/article.html#resources_concurrencyhttp://www.vogella.de/articles/JavaConcurrency/article.html#resources_generalhttp://www.vogella.de/articles/JavaConcurrency/article.html#forkjoinhttp://www.vogella.de/articles/JavaConcurrency/article.html#dealockhttp://www.vogella.de/articles/JavaConcurrency/article.html#thankyouhttp://www.vogella.de/articles/JavaConcurrency/article.html#questionshttp://www.vogella.de/articles/JavaConcurrency/article.html#resourceshttp://www.vogella.de/articles/JavaConcurrency/article.html#sourcecodehttp://www.vogella.de/articles/JavaConcurrency/article.html#resources_concurrencyhttp://www.vogella.de/articles/JavaConcurrency/article.html#resources_general


3/28

resources of the process are allocated to it viathe operating system, e.g. memory and CPUtime.

Threads: threads are so called lightweightprocesses which have their own call stack butan access shared data. Every thread has its ownmemory cache. If a thread reads shared data itstores this data in its own memory cache. Athread can re-read the shared data, when thishappens in Java will be explained in Javamemory model part of this article.

Within a Java application you work with severalthreads to archive parallel processing orasynchronously behavior.

1.3. Amdahl's Law

Concurrency promises to perform certain task faster

as these tasks can be divided into subtasks andthese subtasks can be executed in parallel. Of course the runtime is limited by parts of the taskwhich can be performed in parallel.The theoreticalpossible performance gain can be calculated byAmdahl's Law. If F is the percentage of the programwhich can not run in parallel and N is the number of processes then the maximum performance gain is 1/(F+ ((1-F)/n)).


4/28

1.4. Concurrency issues

Threads have there own call stack but can alsoaccess shared data. Therefore you have two basicproblems, visibility and access problems.

A visibility problem occurs if one thread readsshared data which is later changed by other threadand if thread A does not see this change.

A access problem occurs if several thread trying toaccess and change the same shared data at thesame time.

Visibility and access problem can lead to

Liveness failure: The program does not reactanymore due to problems in the concurrentaccess of data, e.g. deadlocks.

Safety failure: The program creates incorrectdata.

2. Concurrency in Java

2.1. Processes and Threads

A Java program runs in its own process. Javasupports within one program the usage of threads.

Java supports threads as part of the Java language.Java 1.5 also provides improved support forconcurrency with the in the package

java.util.concurrent.


5/28

Java also provides locks to protect certain parts of the coding to be executed by several threads at thesame time. The simplest way of locking a certain

method or Java class is to use the keyword"synchronized" in a method or class declaration.

2.2. Synchronized

The synchronized keyword in Java ensures:

that only a single thread can execute a block of code at the same time

ensures that each thread entering asynchronized block of code sees the effects of all previous modifications that were guarded bythe same lock

Synchronization is necessary for mutual exclusiveaccess of code blocks and for reliable

communication between threads.synchronized can be used directly in the definition of a method. This would ensure that only one threadcan access this method. Or you can usedsynchronized blocks which a named block (either astring or a object). Then all code which is protectedby the same lock can only be executed by one

thread.For example the following datastructure will ensurethat only one thread can access the inner block of add() and next()


6/28

package de.vogella.pagerank.crawler;

import java.util.ArrayList;import java.util.List;

/*** Data structure for a web crawler. Keeps track

of the visited sites and keeps* a list of sites which needs still to be

crawled.** @author Lars Vogel*

*/public class CrawledSites {private List crawledSites = new

ArrayList();private List linkedSites = new

ArrayList();

public void add(String site) {synchronized (this) {

if (!crawledSites.contains(site)) {linkedSites.add(site);}

}}

/*** Get next site to crawl. Can return null

(if nothing to crawl)

*/public String next() {if (linkedSites.size() == 0) {

return null;}synchronized (this) {


7/28

// Need to check again if size haschanged

if (linkedSites.size() > 0) {String s = linkedSites.get(0);

linkedSites.remove(0);crawledSites.add(s);return s;

}return null;

}}

}

2.3. Volatile

If a variable is declared as volatile then isguaranteed that any thread which reads the fieldwill see the most recently written value. Thekeyword volatile will not perform any mutualexclusive lock on the variable.

As of Java 5 write access to a volatile variable willalso update non-volatile variables which weremodified by the same thread. This can also be usedto update values within a reference variable, e.g. fora volatile variable person. In this case you must use

a temporary variable person and use the setter toinitialize the variable and then assign the temporaryvariable to the final variable. This will then make theaddress changes of this variable and the valuesvisible to other threads.


8/28

3. The Java memory model

3.1. Overview

The Java memory model describes thecommunication between the memory of the threadsand the main memory, e.g. then are changes of threads visible to others and then does a thread re-fresh its own memory from the main memory.

It also describes which operations are atomic andthe ordering of operations.

3.2. Atomic operation

An atomic operation is an operation which isperformed as a single unit of work without thepossibility of interference from other operations.

In Java the language specification guarantees that

that reading or writing a variable is atomic (unlessthe variable is of type long or double). Long anddouble are only atomic if they declared as volatile.

The operation i++ it not atomic in Java for thestandard variables (int, long, etc). It first reads thevalue which is currently stored in i (atomicoperations) and then it adds one to it (atomicoperation). But between the read and the write thevalue of i might have changed. Java

Since Java 1.5 the java language provides atomicvariables, e.g. AtomicInteger or AtomicLong whichprovide methods like getAndDecrement(),


9/28


10/28

o privateo have no setter methodo they are never directly returned of

otherwise exposed to a callero if they are changed internally in the classthis change is not visible and has no effectoutside of the class

An immutable class may have some mutable datawhich is uses to manages its state but from theoutside this class nor any attribute of this classes

can get changed.For all mutable fields, e.g. Arrays, that are passedfrom the outside to the class during the constructionphase, the class needs to make a defensive-copy of the elements to make sure that no other object fromthe outside still can change the data

4.2. Defensive CopiesYou must protected your classes from calling code.Assume that calling code will do its best to changeyour data in a way you didn't expect it. While this isespecially true in case of immutable data it is alsotrue for non-immutable data which you still notexpect that this data is changed outside your class.

To protect your class against that you should copydata you receive and only return copies of data tocalling code.


11/28

The following example creates a copy of a list(ArrayList) and returns only the copy of the list. Thisway the client of this class cannot remove elements

from the list.

package de.vogella.performance.defensivecopy;

import java.util.ArrayList;import java.util.Collections;import java.util.List;

public class MyDataStructure {List list = new ArrayList();

public void add(String s) {list.add(s);

}

/*** Makes a defensive copy of the List and

return it* This way cannot modify the list itself** @return List*/

public List getList() {return new ArrayList(list);

}}


12/28

5. Threads in Java

TipYou find this examples in the source section in Javaproject "de.vogella.concurrency.threads"

The base means for concurrency are java.lang.Threads. Each thread requires a java.lang.Runnable which contains in the methodrun the unit of work which must be performed.

Think of runnable as the task to perform. The threadis the worker who is doing this task.

For example this is a simple task (Runnable) whichcounts the sum of a given range of numbers.

package de.vogella.concurrency.threads;

/*** MyRunnable will count the sum of the numberfrom 1 to the parameter

* countUntil and then write the result to theconsole.

*

* MyRunnable is the task which will be performed** @author Lars Vogel

**/public class MyRunnable implements Runnable {

private final long countUntil;

MyRunnable(long countUntil) {this.countUntil = countUntil;


13/28

}

@Overridepublic void run() {

long sum = 0;for (long i = 1; i < countUntil; i++) {sum += i;

}System.out.println(sum);

}}

To perform these tasks (Runnables) you needthreads. The following will create threads, assignrunnable to these threads and wait until they arefinished.

package de.vogella.concurrency.threads;

import java.util.ArrayList;import java.util.List;

public class Main {

public static void main(String[] args) {// We will store the threads so that we

can check if they are doneList threads = new

ArrayList();// We will create 500 threadsfor (int i = 0; i < 500; i++) {

Runnable task = newMyRunnable(10000000L + i);

Thread worker = new Thread(task);


14/28

// We can set the name of thethread

worker.setName(String.valueOf(i));// Start the thread, never call

method run() direct worker.start();// Remember the thread for later

usagethreads.add(worker);

}int running = 0;do {

running = 0;

for (Thread thread : threads) {if (thread.isAlive()) {running++;

}}System.out.println("We have " +

running + " running threads. ");} while (running > 0);

}}

Threads were the first approach in Java to supportconcurrency. The usage of the threads has thefollowing disadvantages.

Creating a new thread causes someperformance overhead

Too many threads can lead to reducedperformance, as the CPU needs to switchbetween these threads.


15/28

You cannot easily control the number of threads, therefore you may run into out of memory errors due to too many threads.

The java.util.concurrent package offers improvedsupport for concurrency compared to threads. The

java.util.concurrent package helps solving several of the issues with threads.

6. Threads pools with the Executor Framework

TipYou find this examples in the source section in Javaproject "de.vogella.concurrency.threadpools"

Thread pools manage a pool of worker threads. Thethread pools contains a work queue which holdstasks waiting to get executed.

A thread pool can be described as a collection of runnables (work queue) and a connections of running threads. These threads are constantlyrunning and are checking the work query for newwork. If there is new work to be done they executethis Runnable. The Thread class itself provides amethod, e.g. execute(Runnable r) to add runnablesto the work queue.

The Executor framework provides exampleimplementation of the java.util.concurrent.Executorinterface, e.g. Executors.newFixedThreadPool(int n)which will create n worker threads. TheExecutorService adds lifecycle methods to the


16/28

Executor, which allows to shutdown the Executorand to wait for termination.

TipIf you want to use one thread pool with one threadwhich executes several runnables you can useExecutors.newSingleThreadExecutor();

Create again the Runnable.

package de.vogella.concurrency.threadpools;

/*** MyRunnable will count the sum of the number

from 1 to the parameter* countUntil and then write the result to the

console.*

* MyRunnable is the task which will be performed*

* @author Lars Vogel**/

public class MyRunnable implements Runnable {private final long countUntil;

MyRunnable(long countUntil) {this.countUntil = countUntil;

}

@Overridepublic void run() {

long sum = 0;for (long i = 1; i < countUntil; i++) {

sum += i;}


17/28

System.out.println(sum);}

}

Now you run your runnables with the executorframework.

package de.vogella.concurrency.threadpools;

import java.util.concurrent.ExecutorService;

import java.util.concurrent.Executors;

public class Main {private static final int NTHREDS = 10;

public static void main(String[] args) {ExecutorService executor =

Executors.newFixedThreadPool(NTHREDS);for (int i = 0; i < 500; i++) {

Runnable worker = newMyRunnable(10000000L + i);executor.execute(worker);

}// This will make the executor accept no

new threads// and finish all existing threads in

the queueexecutor.shutdown();

// Wait until all threads are finishwhile (!executor.isTerminated()) {

}System.out.println("Finished all

threads");}


18/28

}

In case the threads should return some value(result-bearing threads) then you could use java.util.concurrent.Callable.

7. Futures and Callables

TipYou find this examples in the source section in Java

project "de.vogella.concurrency.callables"

The executor framework presented in the lastchapter works with Runnables. Runnable do notreturn result.

In case you expect your threads to return acomputed result you can use

java.util.concurrent.Callable. Callables allow toreturn values after competition.

Callable uses generic to define the type of objectwhich is returned.

If you submit a callable to an executor theframework returns a java.util.concurrent.Future.

This futures can be used to check the status of acallable and to retrieve the result from the callable.

On the executor you can use the method submit tosubmit a Callable and to get a future. To retrievethe result of the future use the get() method.


19/28

package de.vogella.concurrency.callables;

import java.util.concurrent.Callable;

public class MyCallable implements Callable{

@Overridepublic Long call() throws Exception {

long sum = 0;for (long i = 0; i


20/28

List list = newArrayList();

for (int i = 0; i < 20000; i++) {Callable worker = new

MyCallable(); Future submit =executor.submit(worker);

list.add(submit);}long sum = 0;System.out.println(list.size());// Now retrieve the resultfor (Future future : list) {

try { sum += future.get();} catch (InterruptedException e) {

e.printStackTrace();} catch (ExecutionException e) {

e.printStackTrace();}

}System.out.println(sum);

}}

8. Nonblocking algorithms

Java 5.0 provides supports for additional atomicoperations. This allows to develop algorithm whichare non-blocking algorithm, e.g. which do notrequire synchronization, but are based on low-levelatomic hardware primitives such as compare-and-swap (CAS). A compare-and-swap operation check if the variable has a certain value and if it has thisvalue it will perform this operation.


21/28

Non-blocking algorithm are usually much faster thenblocking algorithms as the synchronization of threads appears on a much finer level (hardware).

For example this created a non-blocking counterwhich always increases. This example is contained inthe project"de.vogella.concurrency.nonblocking.counter".

packagede.vogella.concurrency.nonblocking.counter;

import java.util.concurrent.atomic.AtomicInteger;

public class Counter {private AtomicInteger value = new

AtomicInteger();public int getValue(){

return value.get();}

public int increment(){return value.incrementAndGet();

}

// Alternative implementation as incrementbut just make the

// implementation explicitpublic int incrementLongVersion(){

int oldValue = value.get();

while(value.compareAndSet(value.get(),oldValue+1)){

oldValue = value.get();}return oldValue+1;

}


22/28

}

And a test.

packagede.vogella.concurrency.nonblocking.counter;

import java.util.ArrayList;import java.util.HashSet;import java.util.List;

import java.util.Set;import java.util.concurrent.Callable;import java.util.concurrent.ExecutionException;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;

public class Test {private static final int NTHREDS = 10;

public static void main(String[] args) {final Counter counter = new

Counter();List list = new

ArrayList();

ExecutorService executor =Executors.newFixedThreadPool(NTHREDS);

for (int i = 0; i < 500; i++) {Callable worker = newCallable() {

@Overridepublic Integer call()

throws Exception {


23/28

int number =counter.increment();

System.out.println(number );

return number ;}};Future submit=

executor.submit(worker);list.add(submit);

}

// This will make the executoraccept no new threads

// and finish all existing threadsin the queue

executor.shutdown();// Wait until all threads are

finishwhile (!executor.isTerminated()) {

}Set set = newHashSet();

for (Future future : list){

try {set.add(future.get());

} catch (InterruptedExceptione) {

e.printStackTrace();} catch (ExecutionException e){

e.printStackTrace();}

}if (list.size()!=set.size()){


24/28

throw newRuntimeException("Double-entries!!!");

}

}

}

The interesting part is how incrementAndGet() isimplemented. It uses a CAS operation.

public final int incrementAndGet() {for (;;) {

int current = get();int next = current + 1;if (compareAndSet(current, next))

return next;}

}

The JDK itself makes more and more use of non-blocking algorithms to increase performance forevery developer. Developing correct non-blockingalgorithm is not a trivial task. For more informationon non-blocking algorithm, e.g. examples for a non-

blocking Stack and non-block LinkedList, please seehttp://www.ibm.com/developerworks/java/library/j-

jtp04186/index.html
http://www.ibm.com/developerworks/java/library/j-jtp04186/index.htmlhttp://www.ibm.com/developerworks/java/library/j-jtp04186/index.htmlhttp://www.ibm.com/developerworks/java/library/j-jtp04186/index.htmlhttp://www.ibm.com/developerworks/java/library/j-jtp04186/index.html


25/28

9. Fork-Join in Java 7

Java 7 introduce a new parallel mechanism forcompute intensive tasks, the fork-join framework.The fork-join framework allows you to distribute acertain task on several workers and when wait forthe result.

For Java 6.0 you can download the package(jsr166y) from

For testing create the Java project"de.vogella.performance.forkjoin". If you are notusing Java 7 you also need to "jsr166y.jar" to theclasspath.

Create first a package "algorithm" and then theproblem class.

package algorithm;

import java.util.Random;

/**** This class defines a long list of integers

which defines the problem we will* later try to solve

**/

public class Problem {private final int[] list = new int[2000000];

public Problem() {Random generator = new Random(19580427);


26/28

for (int i = 0; i < list.length; i++) {list[i] =

generator.nextInt(500000);}

}

public int[] getList() {return list;

}

}

Define now the solver class. This class extendsRecursiveTask.

TipThe API defines other top classes, e.g.RecursiveAction, AsyncAction. Check the Javadoc fordetails.

package algorithm;

import java.util.Arrays;

import jsr166y.forkjoin.RecursiveAction;

public class Solver extends RecursiveAction {private int[] list;

public long result;

public Solver(int[] array) {this.list = array;

}

@Override


27/28

protected void compute() {if (list.length == 1) {

result = list[0];} else {

int midpoint = list.length / 2;int[] l1 = Arrays.copyOfRange(list,0, midpoint);

int[] l2 = Arrays.copyOfRange(list,midpoint, list.length);

Solver s1 = new Solver(l1);Solver s2 = new Solver(l2);forkJoin(s1, s2);result = s1.result + s2.result;

}}}

Now define a small test class for testing it efficiency.

package testing;

import jsr166y.forkjoin.ForkJoinExecutor;import jsr166y.forkjoin.ForkJoinPool;import algorithm.Problem;import algorithm.Solver;

public class Test {

public static void main(String[] args) {Problem test = new Problem();// Check the number of available

processorsint nThreads =

Runtime.getRuntime().availableProcessors();System.out.println(nThreads);


28/28

Solver mfj = new Solver(test.getList());ForkJoinExecutor pool = new

ForkJoinPool(nThreads);pool.invoke(mfj);

long result = mfj.getResult();System.out.println("Done. Result: " +result);

long sum = 0;// Check if the result was okfor (int i = 0; i

Documents

Concurrency and Multithreading