Everything You Wanted to

Know

About Writing Async,

Concurrent HTTP Apps in Java

Agenda

• Mostly this:

Agenda

• And this:

Agenda

• And this:

About your speaker

linkd.in/jbaruch

stackoverflow.com/users/402053/jbaruch

What Frog?

What Frog?

What Frog?

What Frog?

Requirements

– parallel file Downloads

– Parallel file parts

– interrupt/pause/resume

– Progress events

– Checksums caching

First Association for “concurrent

downloader”

Lucky day: Download manager

written in Java!

Let’s look if we can use it!

1. No traceable license2. No website or docs3. No traceable sources4. It’s an app, not a lib

Java.net.urlconnection

1. Memory wasteful (buffering)

2. Minimal API

3. Blocking streams

What we’re looking for

1. Async/non-blocking

2. Event callbacks

What is IT going to take

1. Reactor

2. nio

Welcome to the reactor

– pattern for lightweight concurrency

– Event driven

– Threads reuse

– Uses non-blocking Io

Original pattern

http://www.dre.vanderbilt.edu/~schmidt/PDF/reactor-siemens.pdf

Guess the author by the

diagram

http://gee.cs.oswego.edu/dl/cpjslides/nio.pdf

In Java, Reactor means NIO

Selector as a multiplexer

Java version - Registering

SocketChannel channel= SocketChannel.open();

socketChannel.connect(new

InetSocketAddress("http://remote.com", 80));

...

Selector selector = Selector.open();

channel.configureBlocking(false);

SelectionKey k = channel.register(selector,

SelectionKey.OP_READ);

k.attach(handler);

Java version - Dispatcher

while (!Thread.interrupted()) {

selector.select();

Set selected = selector.selectedKeys();

Iterator it = selected.iterator();

while (it.hasNext())

SelectionKey k = (SelectionKey)(it.next();

((Runnable)(k.attachment())).run();

selected.clear();

}

Handling reactor events is complex

– Need to maintain state

– Buffering – assembling chunks

– Coordinating async events

Nio libraries

– Most of them are servers–Netty, grizzly, etc.

– Apache Mina– Apache HTTP components asyncclient– Ning http client

Nio libraries

– Most of them are servers–Netty, grizzly, etc.

– Apache Mina– Apache HTTP components asyncclient– Ning http client

– Client and server nio library

– Evolved from netty

– Latest release October 2012

Nio libraries

– Most of them are servers–Netty, grizzly, etc

– Apache Mina– Apache HTTP components asyncclient– Ning http client

Ning’s async http client

Here it is!

try (AsyncHttpClient asyncHttpClient = new AsyncHttpClient()) {

ListenableFuture<Response> future = asyncHttpClient.prepareGet(

"http://oss.jfrog.org/api/system/ping").execute(

new AsyncCompletionHandler<Response>() {

@Override

public Response onCompleted(Response response) {

System.out.println(response.getResponseBody());

return response;

}

@Override

public void onThrowable(Throwable t) {

t.printStackTrace();

}

});

Response response = future.get();

}

HAC Concepts

– Request producer

– Response consumer

try(CloseableHttpAsyncClientasyncHttpClient=HttpAsyncClients.createDefault()){asyncHttpClient.start();Future<HttpResponse>future=asyncHttpClient.execute(HttpAsyncMethods.createGet("http://oss.jfrog.org/api/system/ping"),newAsyncByteConsumer<HttpResponse>(){@OverrideprotectedvoidonResponseReceived(finalHttpResponseresponse){System.out.println(response.getStatusLine().getReasonPhrase());}@OverrideprotectedvoidonByteReceived(finalCharBufferbuf,finalIOControlioctrl){}@OverrideprotectedvoidreleaseResources(){}@OverrideprotectedHttpResponsebuildResult(finalHttpContextcontext){return(HttpResponse)context.getAttribute("http.response");}},null);HttpResponseresponse=future.get();}

Choosing between ning and http

asyncclient

"All problems in computer science can

be solved by another level of

indirection"David

Wheeler

public interface HttpProviderDownloadHandler {

void onResponseReceived(int statusCode, Map<String, List<String>> headers);

boolean onBytesReceived(ByteBuffer buf);

void onFailed(Throwable error);

void onCanceled();

void onCompleted();}

Head to head

Feature/Library Ning client Http Async Client

Maturity Good Very new (early 2014)

Download cancelation Easy With bugs

Progress hooks Events not granular enough

Just use onByteReceived()

Documentation A bit sparse Minimal

Server-side counterpart None, client only org.apache.httpcomponentshttpcore-nio

Performance?

0

100

200

300

400

500

600

700

800

900

Small file Medium file Large file

Ning

AHAC

http://blogs.atlassian.com/2013/07/http-client-performance-io/

Rfc2616: a universe of its own

Confused?

Just read some stackoverflow

(and improve your rep as you go)

And that one for discovering that range header is lost on redirect

Question!

What should be

content-length

when using

compression?

https://github.com/http2/http2-spec/issues/46

Question!

Why when redirected to CDN all the chunks start from zero?

HttpAsyncClientBuilder builder = HttpAsyncClients.custom();// add redirect strategy that copies "range" headers, if existbuilder.setRedirectStrategy(new DefaultRedirectStrategy() {

@Overridepublic HttpUriRequest getRedirect(HttpRequest request, HttpResponse response,

HttpContext context)HttpUriRequest redirectRequest = super.getRedirect(request, response, context);// copy "Range" headers, if existHeader[] rangeHeaders = request.getHeaders(HttpHeaders.RANGE);if (rangeHeaders != null) {

for (Header header : rangeHeaders) {redirectRequest.addHeader(header);

}}return redirectRequest;

}});

Question!

How many simultaneous

connections should I open?

Question!

What’s wrong

with the

following

code?

public static String encodeUrl(String urlStr) {URLEncoder.encode(urlStr, "UTF-8");...

}

Decoded URLs cannot be

re-encoded to the same form

http://example.com/?query=a&b==c

Cannot be decoded back after it was encoded:

http://example.com/?query=a%26b==c

Don’t use java.net.URLEncoder

“Utility class for HTML form encoding. This class contains static methods for converting a String to the application/x-www-form-urlencoded MIME format.

For more information about HTML form encoding, consult the HTML specification.”

AHC Alternatives

Question!

How do I

close a

socket

correctly?

How hard can it be to close a

socket?

The art of socket closing

http://www.safaribooksonline.com/library/view/http-the-definitive/1565925092/ch04s07.html

Half-closed: no new customers

Never block in socket close()

• The other side expects you

to clean up nicely

• It will give up on time out

• You will wait (forever)

Remember?

Question!

How can I write

file parts

concurrently?

– Write to separate files, combine on finish

– Write to same file, seeking to the right position

Use FileChannel

• Implements SeekableByteChannel

java.nio.channels.FileChannel#write(java.nio.ByteBuffer src, long position)

Download progress tracking

• PersistentFileProgressInfo– Save the total size, sha1, number of parts

– State of each part (offset, size, completed...)

FileProgressInfo FilePartProgressInfo*

File Locking

File locking Levels

– VM level

– OS level

OS level File locking

• Multiple downloader instances writing to

the same file

• Needed for writing:

– Partial download file

– Persistent download progress

FileLock lock = fileChannel.tryLock();//Non-shared: (0L, Long.MAX_VALUE, false)

if (lock == null) {throw new OverlappingFileLockException();

}return lock;

}

OS Level File Locking -

Exclusive

private FileLock lock(FileChannel fileChannel) throws IOException {FileLock lock = fileChannel.tryLock(Long.MAX_VALUE - 1, 1, false);if (lock == null) {

throw new OverlappingFileLockException();}return lock;

}

OS Level File Locking –

Advisory exclusive

WTF?!

VM Level File Locking

VM Level File Locking– Prevent same VM threads writing to the file when we

started closing it– Closing sequence:

– Release file locks– Close channels– Rename a file to it's final name (remove .part)– Erase progress info

VM Level File LockingReentrantReadWriteLock.ReadLock writeToFileLock = rwl.readLock();ReentrantReadWriteLock.WriteLock closeFileLock = rwl.writeLock();

public void close() throws IOException {this.closeFileLock.lock();

}

public int write(int partIndex, ByteBuffer buf) {if (!this.writeToFileLock.tryLock()) {

throw new IllegalStateException("File is being closed");}...

}

What’s next?

http/2– Mostly standardizing Google's spdy– Header compression– multiplexing– Prioritization– Server push– On the way clear some stuff– E.g. compressed content length

Ease the load

Links!

• RTFM: RFC 2616

• Ultimate book: HTTP: The Definitive

Guide

– Amazon

– Safari

• Reactor pattern

• Doug Lea on NIO

No, Thank you!