Building ‘Bootiful’ microservices cloud

Building ‘Bootiful’ microservices cloud

Powered by Spring Cloud Netflix libraries

Idan Fridman, Tikal

Agenda● Microservices - Quick overview● Implementing microservices - Build your endpoints with Spring-Boot● Spring Cloud and Netflix● Service Discovery: Eureka instances● Circuit Breaker: Hystrix● Client Side Load Balancer: Ribbon● Router and Filter: Automatic registration of Zuul ● RxJava: Aggregation of multiple micro-service requests● Declarative REST Client: Feign

● There is a strong trend in distributed systems with lightweight architectures

● People have started to call them "microservices"

http://www.indeed.com/jobtrends?q=JSON+REST%2CSOAP+XML&l=&relative=1#shareCode

What are microservices?(In short, I promise;P)

● Smaller units of a larger systemsmall enough to be owned by a small agile development team re-writable within one or two agile sprints ( typically two to four weeks) The complexity does not require to refactoring or require further divide into another microservice.

● Not monolithic

● Runs in its own processRunning independently of the application If fail app won't fail and deliver workaroundsLife continues as if nothing has happened.

● Lightweight communication protocolsRely on HTTP(Rest, etc..) or a lightweight message busEstablish a boundary between components Ensures loose coupling.

● Single responsibility principle(SRP)Should perform a single business functionMust be functionally completeAlso knowns as bounded context from Domain-Driven Design

● Independent data source

http://martinfowler.com/bliki/BoundedContext.html

http://en.wikipedia.org/wiki/Domain-driven_design

http://en.wikipedia.org/wiki/Domain-driven_design

Soa vs Microservices

- SOA and microservices don’t conflict or replace each other -> they complement

- SOA is about how to reuse existing functionality as services

- Microservices is about how to make functionality to scale better with high resilience and short release cycles...

Challenges with Microservices ● Distributed/versioned configuration

○ Auto configurations and refresh on runtime

● New services can auto-register at startup ○ Service registration and discovery

● Routing and load balancing○ Clients can detect new instances as they are started up

● Centralized log management ○ Collects and visualize log events from distributed processes

● Circuit Breaker○ Prevent problems with chain of failures

● Security

Open source to the rescue

Agenda● Microservices - Quick overview● Implementing microservices - Build your endpoints with Spring-Boot● Spring Cloud and Netflix● Service Discovery: Eureka instances● Circuit Breaker: Hystrix● Client Side Load Balancer: Ribbon● Router and Filter: automatic registration of Zuul ● RxJava: Aggregation of multiple micro-service requests● Declarative REST Client: Feign

Build your endpoints with Spring-Boot

It can get pretty much small:

@RestControllerclass ThisWillActuallyRun { @RequestMapping("/") String home() { "Hello World!" }}

Pretty much micro?

Spring Boot● A tool for getting started very quickly with Spring

● Common non-functional requirements for a "real" application

● Exposes a lot of useful features by default

● Gets out of the way quickly if you want to change defaults

● Available for web, batch, integration, data, amqp, aop, jdbc, ...

Spring Boot Modules● Spring Boot - main library supporting the other parts of Spring Boot

● Spring Boot Autoconfigure - single @EnableAutoConfiguration annotation creates a whole Spring context

● Spring Boot Starters - a set of convenient dependency descriptors that you can include in your application.

Spring Boot Modules● Spring Boot Actuator - common non-functional features that make an app

instantly deployable and supportable in production

● Spring Boot Tools - for building and executing self-contained JAR and WAR archives

● Spring Boot Samples - a wide range of sample apps


Spring cloud - Netflix● Spring Cloud Netflix provides Netflix OSS integration

● With simple annotations enable and configure can build large distributed systems with battle-tested Netflix components.

● The patterns provided include Service Discovery (Eureka), Circuit Breaker (Hystrix), Intelligent Routing (Zuul) and Client Side Load Balancing (Ribbon)..

System SetupSystem Landscape


Service discovery EurekaDiscover

Motivation● When you’re running your software across a large number of replaceable pieces of hardware, it’

s inevitable that one of them will fail at some point.

● In a cloud environment you are guaranteed failure, and it is absolutely

● Critical to have a service discovery system that can survive failures.

Eureka features● Eureka has a built-in concept of service heartbeats to prevent stale data - if a service doesn’t phone home often enough,

then Eureka will remove the entry from the service registry

● HeartBeat - New services can register, but “dead” ones will be kept, just in case a client might still be able to contact them

● Cache - Eureka caches on the client side. So even if every last Eureka server goes down, or there is a partition where a

client can’t talk to any of the Eureka servers, then the service registry still won’t be lost.

● HA - The server can be configured and deployed to be highly available, with each server replicating state about the

registered services to the others.

Demo - 1

Agenda● Microservices - Quick overview● Implementing microservices - Build your endpoints with Spring-Boot● Spring Cloud and Netflix● Service Discovery: Eureka instances● Circuit Breaker: Hystrix● Netflix Turbine● Client Side Load Balancer: Ribbon● Router and Filter: automatic registration of Zuul ● RxJava: Aggregation of multiple micro-service requests● Declarative REST Client: Feign

Circuit Breaker

What is the circuit breaker pattern?● Communication with microservices(or any other services)

○ Remote calls○ Different machines across a network○ API calls and dependencies

● Unresponsive supplier● Hang without a response● Critical resources leading to cascading failures across systems

Failure risks

How it works?

Hystrix uses the bulkhead pattern to isolate dependencies and limit concurrent access.

The idea behind circuit breaker

1. Monitor For failures

2. Failure occur -> Circuit is open -> Protected call wont be made -> Do something else + raise alert

3. Need external intervention to reset it when things are well again

Not all errors should trip the circuit, some should reflect normal failures and be dealt regularly

Remote function call

Circuit breaker object wrapper

Circuit breaker monitoring● Any change in breaker state should be logged

● Breakers should reveal details of their state for deeper monitoring

What’s next after the break?● You need to consider what to do in case of failure.

○ perhaps business decision?○ Workarounds?○ Show partials results?

Examples:

1. A bank action authorization could be put on a queue to deal with later. 2. Show partial content (better than nothing right?)

Circuit Breaker and Hystrix

What’s Hystrix?● Hystrix evolved out of resilience engineering work that the Netflix API team began in 2011

● Isolating points of access between the services

● Handling fallbacks and protect API calls by stopping them(defining thresholds, etc..)

● Provide temporary fallback exchanges

● Monitoring dashboard to watch our our critical access points

Taken from spring.io:

HystrixCommand● Wrap calls to dependencies in a HystrixCommand object

● HystrixCommand follows the command pattern and typically executes within a separate thread

● Hystrix maintains a thread-pool (or semaphore) for each dependency and rejects requests (instead of queuing requests)

● if the thread-pool becomes exhausted -> It provides circuit-breaker functionality that can stop all requests to a dependency.

Spring cloud for the rescue● Using spring-cloud-netflix library we can implement circuit breaker logic

easily:

○ Hystrix Synchronous○ Hystrix Asynchronous○ Hystrix Observable

Hystrix synchronous fallbackMethod

@HystrixCommand(fallbackMethod = "fallbackInvocation")

private void generateService(ObjectInput input) {

//invoke your service

}

public void fallbackInvocation(ObjectInput input) {

//keepDefaultvalues

//raise alert

//getNextServer from Eureka and invoke the method

//return data from a local cache }

Hystrix asynchronous Using future

@HystrixCommand(fallbackMethod = "stubbedBro")

public Future<String> runFutureBro(final String name) {

return new AsyncResult<String>() {

public String invoke() {

return(executeRemoteService(name));

}

};}

//and somewhere else:

this.runFutureBro(name).get();

Hystrix asynchronous Using subscriptions


public Observable<String> executeObservableBro(String name) {

return new ObservableResult<String>() {

@Override


return executeRemoteService(name);

}

};

}

Hystrix asynchronous Using subscriptions

broConsumerService.executeObservableBro(input).subscribe(new Observer<String>() {

@Override

public void onCompleted() {

System.out.println("completed");

}

@Override

public void onError(Throwable e) {

System.out.printf(e.getMessage());

}

@Override

public void onNext(String s) {

System.out.println("on next.." + s);

}

});

Command properties @HystrixCommand(commandProperties = {

//execute on local thread or separate thread(Thread,Semaphore)

@HystrixProperty(name = "execution.isolation.thread.timeoutInMilliseconds", value = "500")

//sets the time in milliseconds after which the caller will observe a timeout and walk away from the command

//execution (default 1000)

},

threadPoolProperties = {

@HystrixProperty(name = "coreSize", value = "30"),

//maximum number of HystrixCommands that can execute concurrently

@HystrixProperty(name="hystrix.command.default.circuitBreaker.requestVolumeThreshold",

value = "30"),

//sets the minimum number of requests in a rolling window that will trip the circuit.

//maximum queue size

@HystrixProperty(name = "metrics.rollingStats.numBuckets", value = "12"),

@HystrixProperty(name = "metrics.rollingStats.timeInMilliseconds", value = "1440")

})

Drawbacks● Each command execution involves:

○ Queueing ○ Scheduling○ Context switching involved in running a command on a separate thread.

● Netflix’s experiences proved it minor enough to not have major cost or performance impact.

Demo - 2

Protected with Hystrix

Hystrix Dashboard● You can Monitor your access points(wrapped points) via ui dashboard and grab metrics

Enabling HystrixDashboardsAdd @EnableHystrixDashboard

Demo-3

Agenda● Microservices - Quick overview● Implementing microservices - Build your endpoints with Spring-Boot● Spring Cloud and Netflix● Service Discovery: Eureka instances● Circuit Breaker: Hystrix● Netflix Turbine● Client Side Load Balancer: Ribbon● Router and Filter: automatic registration of Zuul ● RxJava for aggregation of micro-service requests● Declarative REST Client: Feign

Netflix Turbine● Aggregating metrics streams from all cloud instances into one stream

● Use case: Use Hystrix real-time dashboard that uses Turbine to aggregate from many machines

Demo

Rabbitmq ● Turbine can aggregate all streams by receiving all clients metrics via amqp

● Hystrix commands push metrics to Turbine and Spring Cloud enables that with AMQP messaging

● Hysteric use RabbitMQ to communicate between circuit breakers and dashboards

https://www.rabbitmq.com/

Annotate your turbine server with:

@EnableTurbineAmqp

Have all your Hystrix clients importing:

compile("org.springframework.cloud:spring-cloud-starter-hystrix:1.0.0.RELEASE") compile("org.springframework.cloud:spring-cloud-starter-bus-amqp:1.0.0.RELEASE") compile("org.springframework.cloud:spring-cloud-netflix-hystrix-amqp:1.0.0.RELEASE")

That library has everything you need in order to find running rabbitMQ broker and start pushing messages.

In your Hystrix dashboard add the turbine’s amqp listening port: http://host:8989 (set via application.yaml)

http://host:8989

Demo41. Start rabbitmq sbin/rabbitmq-server2. Stream Turbine using port 8989

[demo]


Netflix RibbonClient Side Load Balancing

● Dynamic Routing

● Load Balancer

● Support rules:○ round robin○ response time weighted○ random load balancing○ Custom rules

● Ribbon does not run as a separate service but instead as an embedded component in each service consumer.

Using Ribbon with Eureka● Ribbon includes load balancers that are capable of service discovery in a dynamic environment

● Ribbon uses the information available in Eureka to locate appropriate service instances.● Ribbon will apply load balancing to spread the requests over the available instances.

● Add new instance dynamically -> Registered automatically on discovery services -> Ribbon choose instance to invoke

Ribbon Example

Ribbon and RestTemplate● Inject RestTemplate and use your instance-id● Spring Cloud Ribbon will handle automatically the load-balancing

@EnableEurekaClient@RestController

public class Application {

@Autowired

RestTemplate restTemplate

@RequestMapping("/")

String consume() {

Response response = restTemplate.getForObject("http://baggage-service", Response.class)

}

}Does make sense: Eureka’s service name

The ribbon-loadbalancer must be on the classpath (e.g. via "spring-cloud-starter-ribbon").

Then you can inject one as a LoadBalancerClient or you can inject a RestTemplate (it will be load-balancer aware if you have a LoadBalancerClient).

Demo

Agenda● Microservices - Quick overview● Implementing microservices - Build your endpoints with Spring-Boot● Spring Cloud and Netflix● Service Discovery: Eureka instances● Circuit Breaker: Hystrix● Netflix Turbine● Client Side Load Balancer: Ribbon● Router and Filter: automatic registration of Zuul ● RxJava for aggregation of micro-service requests● Declarative REST Client: Feign

Netflix Zuul - Edge Server

Zuul the Gatekeeper of Gozer

The Gatekeeper of our microservices world

● Entry point to the microservices

● Zuul is a JVM based router and server side load balancer by Netflix.

● Intelligent Routing

● Using Ribbon to lookup available services and routes the external request to an appropriate service instance

● By default Zuul set up a route to every service it can find in Eureka

https://github.com/Netflix/zuul

https://github.com/Netflix/zuul

Using Routings● Routing in an integral part of a microservice architecture.

For example, / -> mapped to your web application, /api/bookings -> mapped to the flight-booking service /api/coupon -> mapped to the coupon service.

Zuul harmonically integrated with Netflix stack

● Zuul using Ribbon to locate an instance to forward to via Eureka

● All requests are executed in a hystrix command -> failures will show up in Hystrix metrics

● once the circuit is open the proxy will not try to contact the service.

Use cases● Use case where a UI application wants to proxy calls to one or more back end service●

Other use cases:● Authentication

● Stress Testing

● Security

● Service Migration

● Active/Active traffic management

Set your routings

On application.yaml you can set your routings and rolesIn this example all services are ignored except coupon-service.

[Demo]

Demo - 5


How many services being called?

Scenario Example1. Client calls to our app’s web service ->

2. our webservice need to request multiple micro-services->

3. uses a callbacks interface to pass the successful result to the next web service call

4. define another success callback- >

5. and then moves on to the next web service request.

Orchestrator

A B C F

Client Request

Looks like that->

Also known as:“The Callback Hell”

//The "Nested Callbacks" Way

public void fetchUserDetails() {

//first, request the users...

mService.requestUsers(new Callback<GithubUsersResponse>() {

@Override

public void success(final GithubUsersResponse githubUsersResponse,

final Response response) {

Timber.i(TAG, "Request Users request completed");

final List<GithubUserDetail> githubUserDetails = new ArrayList<GithubUserDetail>();

//next, loop over each item in the response

for (GithubUserDetail githubUserDetail : githubUsersResponse) {

//request a detail object for that user

mService.requestUserDetails(githubUserDetail.mLogin,

new Callback<GithubUserDetail>() {

@Override

public void success(GithubUserDetail githubUserDetail,

Response response) {

Log.i("User Detail request completed for user : " + githubUserDetail.mLogin);

githubUserDetails.add(githubUserDetail);

if (githubUserDetails.size() == githubUsersResponse.mGithubUsers.size()) {

//we've downloaded'em all - notify all who are interested!

mBus.post(new UserDetailsLoadedCompleteEvent(githubUserDetails));

}

}

@Override

public void failure(RetrofitError error) {

Log.e(TAG, "Request User Detail Failed!!!!", error);

}

});

}

}

Async our microservices call(Using Reactor)

A library for composing asynchronous and event-based programs by using observable sequences.

● Allow you to compose sequences together declaratively ● Abstracting away :

○ low-level threading○ synchronization○ thread-safety○ concurrent data structures○ non-blocking I/O.

RxJava for the rescue● RxJava is single jar lightweight library. ● Using the Observable abstraction and related higher-order functions.

(Support Java6+)

The following external libraries can work with RxJava:

● Camel RX provides an easy way to reuse any of the Apache Camel components, protocols, transports and data

formats with the RxJava API

● rxjava-http-tail allows you to follow logs over HTTP, like tail -f

● mod-rxvertx - Extension for VertX that provides support for Reactive Extensions (RX) using the RxJava library

And ->> Hystrix latency and fault tolerance bulkheading library. (Which is why we here)

http://camel.apache.org/rx.html

http://camel.apache.org/components.html

http://camel.apache.org/rx.html



https://github.com/myfreeweb/rxjava-http-tail

https://github.com/myfreeweb/rxjava-http-tail

https://github.com/vert-x/mod-rxvertx

https://github.com/vert-x/mod-rxvertx

https://github.com/Netflix/Hystrix/wiki/How-To-Use#wiki-Reactive-Execution

Using Observables“Go do this thing and when that thing is done then invoke

that other code”


public Observable<String> executeObservableBro(String name) {

return new ObservableResult<String>() {

@Override


return executeRemoteService(name);

}

};

}


Demo

Feign Client● It makes writing web service clients easier

● Feign connect your code to http API’S with minimal overhead.

● Maps to the REST API methods that are exposed by a service

● Customizable decoders and error handlings

● Feign works by processing annotations into a templatized request

Feign and Spring-Cloud● Create interface and annotate

● Support Feign annotations and JAX-RS annotations

● Collaborated with Spring-MVC annotations(Using convertors like: HttpMessageConverters )

● Eureka-aware REST clients that uses Ribbon for client-side load-balancing to pick an available service instance

● You can call methods on an interface instead of manually handling URLs, Spring Cloud takes care of auto-instantiating the interface implementation and will use Eureka for discovery.

Demo 6

Thank You:)

Documents

Building ‘Bootiful’ microservices cloud