AtmosphereConf - Let's build a solid base for a scale

@DebskiChris

#AtmosphereConf

Krzysztof Dębski Allegro Group

Let’s build a solid base for a scale

#AtmosphereConf @DebskiChris Allegro The New World Improvement WIIFY

Agenda

The Allegro Situation How to build a new World? The way to improve What’s in it for you?

Allegro The New World Improvement WIIFY #AtmosphereConf @DebskiChris

The Allegro Situation

Allegro

6 million LOC

400 people in IT

Allegro

A New Hope

A New Hope?

Service

Oriented

Ambiguity

Domain Driven Design

(Micro)Services

Business needs Offer

Transaction

(Micro)Services

Transaction

Independent

(Micro)Services

API Offer

Transaction

(Micro)Services

Polyglot

Cassandra

MongoDB

Oracle

Transaction

(Micro)Services

Smart Endpoints

Cassandra

MongoDB

Oracle

Transaction

The first approach

Service

Auto deployable Monitored Auto scalable Auto healable Auto discoverable

How to build a new World?

The first project

Gradle Spring External Jetty server

Let’s REST

JAX-RS / JSR Compliant @Path("/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

// [...] @GET public UserCollectionResponse findAllUsers() { //[...] }

Let’s expose our methods

Swagger @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }

Let’s expose our methods

Swagger

And make them discoverable

Discovery ZooKeeper

Register

Discovery ZooKeeper

Get User

Discovery ZooKeeper

Get User

Discovery ZooKeeper

Get User Get off

er for user

Multiple API versions

Header support public class UserMediaType {

public static final String V1_JSON = "application/vnd.allegro.user.v1+json” public static final String V2_JSON = "application/vnd.allegro.user.v2+json”

curl --dump-header - -H ”Accept: application/vnd.allegro.user.v2+json" -X GET http://localhost:8080/users

Multiple API versions

Content negotiation @Path("/users") @Consumes(AllegroUserMediaType.V1_JSON) @Produces(AllegroUserMediaType.V1_JSON) public class UsersEndpoint {

// [...] @GET @Produces(AllegroUserMediaType.V2_JSON) public UserCollectionResponse findAllUsers() { //[...] }

How to configure it?

Zookeper Environment properties Command-line options

And it became slow

Lower startup time

External Jetty Embedded Jetty Embedded UnderTow

Easier deployment

Cargo deployment Built to zip package On immutable images

Do you test your tests?

Pitest

Integration tests

Test without mocks Run from IDE

public class UsersIntegrationTest {

private static Map<String, String> overrideConfiguration = Maps.newHashMap();

static { overrideConfiguration.put("property.name", "This was overwritten value"); }

@ClassRule public static final RestServiceStarted DEPLOYED_SERVICE = new RestServiceStarted(overrideConfiguration);

private WebTarget getUsersResourceWebTarget() { return DEPLOYED_SERVICE.getWebTarget().path("users"); }

Monitoring

Let’s see what happens

LogStash Kibana NxLog

Let’s see what happens

Graphite # Metrics metrics.reporters.graphite.enabled=true metrics.reporters.graphite.host=graphite.service metrics.reporters.graphite.port=2003 metrics.reporters.graphite.prefix=stats.Prod.service metrics.reporters.interval=30

Monolith Alert

Multi module project support

The way to improve

Authentication

OAuth2

Patch Support

Standardized by IETF https://tools.ietf.org/html/rfc6902

PATCH /user/1 HTTP/1.1 Host: user.service.local Content-Length: 312

Content-Type: application/json-patch+json [

{”op”: ”test”, ”path”: ”/firstname”, ”value”: ”Jane”}, {”op”: ”add”, ”path”: ”/maidenname”, ”value”: ”Smith”}, {”op”: ”replace”, ”path”: ”/lastname”, ”value”: ”Doe”},

{”op”: ”remove", ”path”: ”/meetings”}, {”op”: ”move”, ”from”: ”/balance”, ”path”: ”sharedbalance”}, {”op”: ”copy”, ”from”: ”/a”, ”path”: ”/b”}

Simplify annotations

Swagger? @Path("/users") @Api(value = "/users") @Consumes(CONTENT_TYPE_JSON) @Produces(CONTENT_TYPE_JSON) public class UsersEndpoint {

// [...] @ApiOperation(value=“Get all users”, response=UserCollectionResponse.class) @GET public UserCollectionResponse findAllUsers() { //[...] }

Deployment

Docker support

What’s in it for you?

Use the latest tools

GradleW issue in Continuous Integration Environment and parent POM issues. Don’t do DDOS yourself and your partners’ sites.

Change is the only constant

HTTP Server Service API provider Dependency injection engine Deployment tools

Focus on right metrics

import org.junit.Test;

import static net.trajano...

public class MediaTypesTest {

@Test public void mediaTypesShouldBeValidUtilityClasses()

throws Throwable { assertUtilityClassWellDefined(UserMediaType.class); } }

Tools also lie

Metrics tend to lie – Default PHP metrics in SonarQube – Tested file

•  4535 CLOC •  Whole code written using imperative programming

How many violations are there?

Tools also lie

How many violations are there?

4 At least according to Sonar

Learn to REST

REST is not so obvious

/api/bi/XYZ123/1

/recommendations/1/items

/offer/100?output=ESI/JSON

Community

Involve all developers in building the Bootstrap. Or they will build their own tools.

Łukasz Drumiński Mateusz Gajewski @wendigo

AtmosphereConf - Let's build a solid base for a scale

Technology

Let's build a smarter planet energy and utilities ejc

Let's Build an Unpacker (Unpackers 101)

Build Solid Financial Foundation

Let's build a quantum computer!

Let's build an Airport – How to estimate large scale projects

Let's build a better future for Translators and Interpreters - Together

A BUILD BESIDE THE SEASIDE, PART 4 – Fitting out . Let's

Infocentric at IntraNET.Reloaded: Let's Build Cities!

Federal Election Let's Build Canada 2015

Build Teams of - Let's Grow Leaders

Let's build stronger teams!

Let's build something "unuseless" :)

Let's Build Science Port Holland – issue 1

Let's Build an Angular App!

Let's build modular robots!

Let's Build a Better Password

Let's Build the Bridge

Let's Build a Service Oriented Data Pipeline!

Let's build a smarter planet energy and utilities americana

Atmosphere 2014: Let's build a solid base for a scale. - Krzysztof Debski