Upload
chris-richardson
View
2.766
Download
3
Embed Size (px)
DESCRIPTION
A modular, polyglot architecture has many advantages but it also adds complexity since each incoming request typically fans out to multiple distributed services. For example, in an online store application the information on a product details page - description, price, recommendations, etc - comes from numerous services. To minimize response time and improve scalability, these services must be invoked concurrently. However, traditional concurrency mechanisms are low-level, painful to use and error-prone. In this talk you will learn about some powerful yet easy to use abstractions for consuming web services asynchronously. We will compare the various implementations of futures that are available in Java, Scala and JavaScript. You will learn how to use reactive observables, which are asynchronous data streams, to access web services from both Java and JavaScript. We will describe how these mechanisms let you write asynchronous code in a very straightforward, declarative fashion.
Citation preview
@crichardson
Consuming web services asynchronously with Futures
and Rx ObservablesChris Richardson
Author of POJOs in ActionFounder of the original CloudFoundry.com
@[email protected]://plainoldobjects.com
@crichardson
Presentation goal
Learn how to use (Scala) Futures and Rx
Observables to write simple yet robust and scalable
concurrent code
@crichardson
About Chris
@crichardson
(About Chris)
@crichardson
About Chris()
@crichardson
About Chris
@crichardson
About Chris
http://www.theregister.co.uk/2009/08/19/springsource_cloud_foundry/
@crichardson
About Chris
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Let’s imagine you are building an online store
@crichardson
Shipping
Recomendations
ProductInfo
Reviews
@crichardson
Reviews
ProductInfo Sales
ranking
@crichardson
Related books
Viewing history
Forum
@crichardson
+ mobile apps
@crichardson
Application architecture
Product Info ServiceDesktop browser
Native mobile client
REST
REST
REST
Recomendation Service
Review Service
Front end server
API gatewayREST
Mobile browser
Web Application
HTML/JSON
JSON
@crichardson
Handling getProductDetails()
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
@crichardson
Handling getProductDetails() - sequentially
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Higher response time :-(
@crichardson
Handling getProductDetails() - in parallel
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails()
Lower response time :-)
@crichardson
Implementing a concurrent REST client
Thread-pool based approach
executorService.submit(new Callable(...))
Simpler but less scalable - lots of idle threads consuming memory
Event-driven approach
NIO with completion callbacks
More complex but more scalable
@crichardson
The front-end server must handle partial failure of backend services
Front-end server
Product Info Service
RecommendationsService
ReviewService
getProductInfo()
getRecommendations()
getReviews()
getProductDetails() XHow to provide a good user experience?
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Futures are a great concurrency abstraction
http://en.wikipedia.org/wiki/Futures_and_promises
@crichardson
Worker threadMain threadHow futures work
Outcome
Future
Client
get
Asynchronous operation
set
initiates
@crichardson
Benefits
Client does not know how the asynchronous operation is implemented
Client can invoke multiple asynchronous operations and gets a Future for each one.
@crichardson
Handling ProductDetails request
ProductDetailsController
ProductDetailsService
getProductDetails()
ProductInfoService
getProductInfo()
ReviewService
getReviews()
RecommendationService
getRecommendations()
RestTemplate
@crichardson
REST client using Spring @Async
@Componentclass ProductInfoServiceImpl extends ProducInfoService { val restTemplate : RestTemplate = ...
@Async def getProductInfo(productId: Long) = { new AsyncResult(restTemplate.getForObject(....)...) }
}
Execute asynchronously in
thread pool
A fulfilled Future
trait ProductInfoService { def getProductInfo(productId: Long):
java.util.concurrent.Future[ProductInfo]}
@crichardson
ProductDetailsService@Componentclass ProductDetailsService @Autowired()(productInfoService: ProductInfoService, reviewService: ReviewService, recommendationService: RecommendationService) {
def getProductDetails(productId: Long): ProductDetails = { val productInfoFuture = productInfoService.getProductInfo(productId)
}
}
val recommendationsFuture = recommendationService.getRecommendations(productId)val reviewsFuture = reviewService.getReviews(productId)
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS) val recommendations = recommendationsFuture.get(10, TimeUnit.MILLISECONDS) val reviews = reviewsFuture.get(10, TimeUnit.MILLISECONDS)
ProductDetails(productInfo, recommendations, reviews)
@crichardson
ProductController
@Controllerclass ProductController @Autowired()(productDetailsService : ProductDetailsService) {
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) =
productDetailsService.getProductDetails(productId)
@crichardson
Blocks Tomcat thread until Future completes
Not bad but...
class ProductDetailsService def getProductDetails(productId: Long): ProductDetails = {
val productInfo = productInfoFuture.get(300, TimeUnit.MILLISECONDS)
Not so scalable :-(
@crichardson
... and also...
Java Futures work well for a single-level of asynchronous execution
BUT #fail for more complex, scalable scenarios
Difficult to compose and coordinate multiple concurrent operations
See this blog post for more details:http://techblog.netflix.com/2013/02/rxjava-netflix-api.html
@crichardson
Better: Futures with callbacks ⇒ no blocking! val f : Future[Int] = Future { ... } f onSuccess { case x : Int => println(x) } f onFailure { case e : Exception => println("exception thrown") }
Guava ListenableFutures, Spring 4 ListenableFutureJava 8 CompletableFuture, Scala Futures
@crichardson
Even better: composable futures
def asyncOperation() = Future { ... ; 1 }
val r = asyncOperation().map{ _ * 2 }
assertEquals(4, Await.result(r, 1 second))
Scala, Java 8 CompletableFuture (partially)
Transforms Future
@crichardson
map() is asynchronous - uses callbacks
class Future[T] {
def map[S](f: T => S): Future[S] = { val p = Promise[S]()
onSuccess { case r => p success f(r) } onFailure { case t => p failure t }
p.future }
When ‘this’ completes propagate outcome to ‘p’
Return new Future
@crichardson
Chaining using flatmap()
def asyncOperation() = Future { ... ; 3 }
def asyncSquare(x : Int) : Future[Int] = Future { x * x}
val r = asyncOperation().flatMap { x => asyncSquare(x) }
assertEquals(9, Await.result(r, 1 second))
Scala, Java 8 CompletableFuture (partially)
Chaining asynchronous operations
@crichardson
Combining futuresval f1 = Future { ... ; 1}val f2 = Future { .... ; 2}
val fzip = f1 zip f2assertEquals((1, 2), Await.result(fzip, 1 second))
def asyncOp(x : Int) = Future { x * x}val f = Future.sequence((1 to 5).map { x => asyncOp(x) })assertEquals(List(1, 4, 9, 16, 25), Await.result(f, 1 second))
Scala, Java 8 CompletableFuture (partially)
Combines two futures
Transforms list of futures to a future containing a list
@crichardson
Scala futures are Monadsdef callB() : Future[...] = ...def callC() : Future[...] = ...def callD() : Future[...] = ...
val result = for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
result onSuccess { .... }
Two calls execute in parallel
And then invokes D
Get the result of D
@crichardson
‘for’ is shorthand for map() and flatMap()
(callB() zip callC()) flatMap { r1 => val (b, c) = r1 callD(b, c) map { d => d }}
for { (b, c) <- callB() zip callC(); d <- callD(b, c) } yield d
@crichardson
Scala Future + RestTemplateimport scala.concurrent.Future
@Componentclass ProductInfoService {
def getProductInfo(productId: Long): Future[ProductInfo] = { Future { restTemplate.getForObject(....) } }
}
Executes in thread pool
Scala Future
@crichardson
Scala Future + RestTemplateclass ProductDetailsService @Autowired()(....) {
def getProductDetails(productId: Long) = { val productInfoFuture = productInfoService.getProductInfo(productId) val recommendationsFuture = recommendationService.getRecommendations(productId) val reviewsFuture = reviewService.getReviews(productId)
for (((productInfo, recommendations), reviews) <- productInfoFuture zip recommendationsFuture zip reviewsFuture) yield ProductDetails(productInfo, recommendations, reviews) }
}
Non-blocking!
@crichardson
Async Spring MVC + Scala Futures
@Controllerclass ProductController ... {
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = { val productDetails = productDetailsService.getProductDetails(productId)
val result = new DeferredResult[ProductDetails] productDetails onSuccess { case r => result.setResult(r) } productDetails onFailure { case t => result.setErrorResult(t) } result }
Convert Scala Future to
Spring MVCDeferredResult
@crichardson
Servlet layer is asynchronous but the backend uses thread
pools⇒
Need event-driven REST client
@crichardson
New in Spring 4
Mirrors RestTemplate
Methods return a ListenableFuture
ListenableFuture = JDK 7 Future + callback methods
Can use HttpComponents NIO-based AsyncHttpClient
Spring AsyncRestTemplate
@crichardson
Using the AsyncRestTemplate
http://hc.apache.org/httpcomponents-asyncclient-dev/
val asyncRestTemplate = new AsyncRestTemplate( new HttpComponentsAsyncClientHttpRequestFactory())
override def getProductInfo(productId: Long) = {
val listenableFuture = asyncRestTemplate.getForEntity("{baseUrl}/productinfo/{productId}", classOf[ProductInfo], baseUrl, productId)
toScalaFuture(listenableFuture).map { _.getBody }}
Convert to Scala Future and get entity
@crichardson
Converting ListenableFuture to Scala Future
implicit def toScalaFuture[T](f : ListenableFuture[T]) : Future[T] = { val p = promise[T]()
f.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { p.success(result)} def onFailure(t: Throwable) { p.failure(t) } }) p.future }
The producer side of Scala Futures
Supply outcome
Return future
@crichardson
Now everything is non-blocking :-)
@crichardson
If recommendation service is down...
Never responds ⇒ front-end server waits indefinitely
Consumes valuable front-end server resources
Page never displayed and customer gives up
Returns an error
Error returned to the front-end server ⇒ error page is displayed
Customer gives up
@crichardson
Fault tolerance at NetflixNetwork timeouts and retries
Invoke remote services via a bounded thread pool
Use the Circuit Breaker pattern
On failure:
return default/cached data
return error to caller
Implementation: https://github.com/Netflix/Hystrix
http://techblog.netflix.com/2012/02/fault-tolerance-in-high-volume.html
@crichardson
How to handling partial failures?
productDetailsFuture zip recommendationsFuture zip reviewsFuture
Fails if any Future has failed
@crichardson
Handling partial failures
recommendationService. getRecommendations(userId,productId) .recover { case _ => Recommendations(List()) }
“catch-like” Maps Throwable to value
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Why solve this problem for JavaScript?
Browser invokes web services
Implement front-end server/API gateway using NodeJS
@crichardson
What’s NodeJS?
Designed for DIRTy apps
@crichardson
NodeJS
JavaScript
Reactor pattern
Modules
@crichardson
Asynchronous JavaScript code = callback hell
Scenarios:
Sequential: A ⇒ B ⇒ C
Fork and join: A and B ⇒ C
Code quickly becomes very messy
@crichardson
Callback-based HTTP client
request = require("request") handler = (error, clientResponse, body) -> if clientResponse.statusCode != 200 // ERROR else // SUCCESS
request("http://.../productdetails/" + productId, handler)
@crichardson
Messy callback codegetProductDetails = (req, res) -> productId = req.params.productId result = {productId: productId} makeHandler = (key) -> (error, clientResponse, body) -> if clientResponse.statusCode != 200 res.status(clientResponse.statusCode) res.write(body) res.end() else result[key] = JSON.parse(body) if (result.productInfo and result.recommendations and result.reviews) res.json(result)
request("http://localhost:3000/productdetails/" + productId, makeHandler("productInfo")) request("http://localhost:3000/recommendations/" + productId,
makeHandler('recommendations')) request("http://localhost:3000/reviews/" + productId, makeHandler('reviews'))
app.get('/productinfo/:productId', getProductInfo)
Returns a callback function
Have all three requests completed?
@crichardson
Simplifying code with Promises (a.k.a. Futures)
Functions return a promise - no callback parameter
A promise represents an eventual outcome
Use a library of functions for transforming and composing promises
Promises/A+ specification - http://promises-aplus.github.io/promises-spec
@crichardson
Basic promise API
promise2 = promise1.then(fulfilledHandler, errorHandler, ...)
called when promise1 is fulfilledreturns a promise or value
resolved with outcome of callback
called when promise1 is rejected
About when.jsRich API = Promises spec + use full extensions
Creation:
when.defer()
when.reject()...
Combinators:
all, some, join, map, reduce
Other:
Calling non-promise code
timeout
....
https://github.com/cujojs/when
@crichardson
Simpler promise-based code
rest = require("rest")
@httpClient = rest.chain(mime).chain(errorCode);
getProductInfo : (productId) -> @httpClient path: "http://.../productinfo/" + productId
Returns a promise
No ugly callbacks
@crichardson
Simpler promise-based code class ProductDetailsService getProductDetails: (productId) -> makeProductDetails = (productInfo, recommendations, reviews) -> details = productId: productId productDetails: productInfo.entity recommendations: recommendations.entity reviews: reviews.entity details responses = [@getProductInfo(productId), @getRecommendations(productId),
@getReviews(productId)] all(responses).spread(makeProductDetails)
Array[Promise] ⇒ Promise[Array]
@crichardson
Simpler promise-based code: HTTP handler
getProductDetails = (req, res) -> productId = req.params.productId
succeed = (productDetails) -> res.json(productDetails)
fail = (something) -> res.send(500, JSON.stringify(something.entity || something))
productDetailsService. getDetails(productId). then(succeed, fail)
app.get('/productdetails/:productId', getProductDetails)
@crichardson
Writing robust client code
@crichardson
Recovering from failures
getRecommendations(productId) .otherwise( -> {})
Invoked to return default value if getRecommendations() fails
@crichardson
Agenda
The need for concurrency
Simplifying concurrent code with Futures
Taming callback hell with JavaScript promises
Consuming asynchronous streams with Reactive Extensions
@crichardson
Let’s imagine you have a stream of trades
and you need to calculate the rolling
average price of each stock
@crichardson
Spring Integration + Complex event processing (CEP) engine is a good choice
@crichardson
But where is the high-level abstraction that solves this
problem?
@crichardson
Future[List[T]]
Not applicable to infinite streams
@crichardson
Introducing Reactive Extensions (Rx)
The Reactive Extensions (Rx) is a library for composing asynchronous and event-
based programs using observable sequences and LINQ-style query operators. Using Rx, developers
represent asynchronous data streams with Observables , query
asynchronous data streams using LINQ operators , and .....
https://rx.codeplex.com/
@crichardson
About RxJava
Reactive Extensions (Rx) for the JVM
Implemented in Java
Adaptors for Scala, Groovy and Clojure
https://github.com/Netflix/RxJava
@crichardson
RxJava core concepts
class Observable<T> { Subscription subscribe(Observer<T> observer) ...}
interface Observer<T> {void onNext(T args)void onCompleted()void onError(Throwable e)
}
Notifies
An asynchronous stream of items
Used to unsubscribe
@crichardson
Comparing Observable to...Observer pattern - similar but adds
Observer.onComplete()
Observer.onError()
Iterator pattern - mirror image
Push rather than pull
Future - similar but
Represents a stream of multiple values
@crichardson
So what?
@crichardson
Transforming Observables
class Observable<T> {
Observable<T> take(int n); Observable<T> skip(int n); <R> Observable<R> map(Func1<T,R> func) <R> Observable<R> flatMap(Func1<T,Observable<R>> func) Observable<T> filter(Func1<T,java.lang.Boolean> predicate)
...}
Scala-collection style methods
@crichardson
Transforming observables
class Observable<T> {
<K> Observable<GroupedObservable<K,T>> groupBy(Func1<T,K> keySelector) ...}
Similar to Scala groupBy except results are emitted as items arrive
Stream of streams!
@crichardson
Combining observables
class Observable<T> {
static <R,T1,T2> Observable<R> zip(Observable<T0> o1, Observable<T1> o2, Func2<T1,T2,R> reduceFunction)
...}
Invoked with pairs of items from o1 and o2
Stream of results from reduceFunction
@crichardson
Creating observables from data
class Observable<T> { static Observable<T> from(Iterable<T> iterable]); static Observable<T> from(T items ...]); static Observable<T> from(Future<T> future]); ...}
@crichardson
Arranges to call obs.onNext/onComplete/...
Creating observables from event sources
Observable<T> o = Observable.create( new SubscriberFunc<T> () { Subscription onSubscribe(Observer<T> obs) {
... connect to event source....
return new Subscriber () { void unsubscribe() { ... };
}; });
Called once for
each Observer
Called when unsubscribing
@crichardson
Using Rx Observables instead of Futures
@crichardson
Rx-based ProductInfoService@Componentclass ProductInfoService override def getProductInfo(productId: Long) = {
val baseUrl = ...
val responseEntity = asyncRestTemplate.getForEntity(baseUrl + "/productinfo/{productId}", classOf[ProductInfo], productId)
toRxObservable(responseEntity).map { (r : ResponseEntity[ProductInfo]) => r.getBody }
}
@crichardson
ListenableFuture ⇒ Observable
implicit def toRxObservable[T](future: ListenableFuture[T]) : Observable[T] = { def create(o: Observer[T]) = { future.addCallback(new ListenableFutureCallback[T] { def onSuccess(result: T) { o.onNext(result) o.onCompleted() }
def onFailure(t: Throwable) { o.onError(t) } }) new Subscription { def unsubscribe() {} } }
Observable.create(create _) }
Supply single value
Indicate failure
Do nothing
@crichardson
Rx - ProductDetailsService@Componentclass ProductDetailsService @Autowired() (productInfoService: ProductInfoService, reviewService: ReviewService, ecommendationService: RecommendationService) {
def getProductDetails(productId: Long) = { val productInfo = productInfoService.getProductInfo(productId) val recommendations =
recommendationService.getRecommendations(productId) val reviews = reviewService.getReviews(productId)
Observable.zip(productInfo, recommendations, reviews, (p : ProductInfo, r : Recommendations, rv : Reviews) =>
ProductDetails(p, r, rv)) }
}Just like Scala Futures
@crichardson
Rx - ProductController
@Controllerclass ProductController
@RequestMapping(Array("/productdetails/{productId}")) @ResponseBody def productDetails(@PathVariable productId: Long) = val pd = productDetailsService.getProductDetails(productId) toDeferredResult(pd)
@crichardson
Rx - Observable ⇒ DeferredResult
implicit def toDeferredResult[T](observable : Observable[T]) = { val result = new DeferredResult[T] observable.subscribe(new Observer[T] { def onCompleted() {}
def onError(e: Throwable) { result.setErrorResult(e) }
def onNext(r: T) { result.setResult(r.asInstanceOf[T]) } }) result }
@crichardson
RxObservables vs. Futures
Much better than JDK 7 futures
Comparable to Scala Futures
Rx Scala code is slightly more verbose
@crichardson
Making this code robust
Hystrix works with Observables (and thread pools)
But
For event-driven code we are on our own
@crichardson
Back to the stream of Trades averaging example...
@crichardson
Calculating averages
Observable<AveragePrice> calculateAverages(Observable<Trade> trades) { ...}
class Trade { private String symbol; private double price;
class AveragePrice { private String symbol; private double averagePrice;
@crichardson
Using groupBy()APPL : 401 IBM : 405 CAT : 405 APPL: 403
groupBy( (trade) => trade.symbol)
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
Observable<GroupedObservable<String, Trade>>
Observable<Trade>
@crichardson
Using window()APPL : 401 APPL : 405 APPL : 405 ...
APPL : 401 APPL : 405 APPL : 405
APPL : 405 APPL : 405 APPL : 403
APPL : 405 ...
window(...)
Observable<Trade>
Observable<Observable<Trade>>
N secs
N secs
M secs
@crichardson
Using fold()
APPL : 402 APPL : 405 APPL : 405
APPL : 404
fold(0)(sumCalc) / length
Observable<Trade>
Observable<AveragePrice> Singleton
@crichardson
Using merge()
merge()
APPL : 401
IBM : 405
CAT : 405 ...
APPL: 403
APPL : 401 IBM : 405 CAT : 405 APPL: 403
Observable<Observable<AveragePrice>>
Observable<AveragePrice>
@crichardson
Calculating average pricesdef calculateAverages(trades: Observable[Trade]): Observable[AveragePrice] = {
trades.groupBy(_.symbol).map { symbolAndTrades => val (symbol, tradesForSymbol) = symbolAndTrades ...
tradesForSymbol.window(...).map { windowOfTradesForSymbol => windowOfTradesForSymbol.fold((0.0, 0, List[Double]())) { (soFar, trade) => val (sum, count, prices) = soFar (sum + trade.price, count + 1, trade.price +: prices) } map { x => val (sum, length, prices) = x AveragePrice(symbol, sum / length, prices) } }.flatMap(identity) }.flatMap(identity)}
@crichardson
RxJS / NodeJS examplevar rx = require("rx");
function tailFile (fileName) { var self = this;
var o = rx.Observable.create(function (observer) { var watcher = self.tail(fileName, function (line) { observer.onNext(line); }); return function () { watcher.close(); } }); return o.map(parseLogLine);}
function parseLogLine(line) { ... }
@crichardson
Rx in the browser - implementing completion
TextChanges(input) .DistinctUntilChanged() .Throttle(TimeSpan.FromMilliSeconds(10)) .Select(word=>Completions(word)) .Switch() .Subscribe(ObserveChanges(output));
Your Mouse is a Databasehttp://queue.acm.org/detail.cfm?id=2169076
Ajax call returning Observable
Change events ⇒ Observable
Display completions
@crichardson
Summary
Consuming web services asynchronously is essential
Scala-style are a powerful concurrency abstraction
Rx Observables are even more powerful
Rx Observables are a unifying abstraction for a wide variety of use cases