Maximize Messaging

Performance and

Lowering Infrastructure

Footprint Senior Software Architect, WSO2 Inc.

Visiting Faculty, University of Moratuwa

Member, Apache Software Foundation

Research Scientist, Lanka Software Foundation

Outline Goals: Optimizing the deployments

o Message performance

o Reduce footprint

Design and Capacity Planning

Design for Performance and Minimal footprint o Select right transport

o Minimize footprint and maximize sharing

- MT

- Using Carbon o Tuning

Wrapping Up

About Mission critical webinar

series

Example Scenario

Outline on Capacity planning We discussed this in the Asanka’s webinar last

week

Rough task list o Design an architecture

o Estimate expected capacity of the who system

o Find the capacity for each type of node

o Calculate the number of nodes required

o Build a POC and verify

Outline on Capacity planning (Contd.)

Our Goal Building smaller systems saves lot of money

and effort o Maximizing the performance

o minimizing the footprint

This webinar explains how to capitalize WSO2

products to that end

How? WSO2 Platform provides several techniques to

optimize deployments

Some of the techniques o Selecting the right transport

o New performance improvements to ESB

o Use carbon to create minimal servers

o Use Multi-tenancy to maximize sharing

o Tuning your setup

Select the right Message Transport WSO2 platform provide several choices for your

system to communicate o SOAP

o HTTP/JSON

o Thrift

o AMQP

o Others - XMPP

- TCP

- Mail/ SMTP

- FTP

- Domain specific formats like FIX

Message Transports Performance

(2 core, 4G

with 4-10K

messages )

TPS

Interopera

bility

Reliability Security Asynchron

ous

messaging

SOAP 2K Very good Has

transactions

WS-

Security/SSL

?

JSON+HTT

P

5k Good X SSL X

Thrift 20K-100K OK, but not

widely

adopted

X SSL X

AMQP 1K OK within

JMS

systems

Has

transactions

and

persistent

messaging

SSL Yes

Select the right Message Transport Several concerns

o Performance

o Interoperability

o Support for Add-On features like transactions,

reliability, security

o Ease of use

o Asynchronous nature

Use what is enough o E.g. internal communication can use thrift while

external communication can use HTTP/JSON for

SOAP

Select the right Message Transport

Data Bridge Highly optimized event collection framework

o Many agents can emit events using the data bridge

o They are collected and processed as receivers

o Use by WSO2 CEP and WSO2 BAM

Very Fast o Can do few 100k events per second

o Uses Apache thrift

o Asynchronous

o Support buffering and batch delivery

o Data formats are predefined and transferred as

tuples

Minimize the Product Footprint What do we mean by foot print?

o Product size (disk space, download)

o Memory foot print

o Unused functionality

Two main ideas o Remove what is not needed

o Maximize sharing

WSO2 platform provides 2 choices to minimize

the footprint o Use carbon to build a minimal Product

o Use Multi-tenancy to maximize sharing

WSO2 Carbon

o Read “Carbon: towards a server building

framework for SOA platform” for more details.

OSGI based

components

runtimes

Products are

created by

composing

components

Build Minimal Product So you can customize and create a minimal

product by adding and removing components E.g. We were able to make WSO2 IS with 64MB heap this

way for a given scenario.

In smaller load cases, you can combine

multiple products to a one product o E.g. Add service hosting to BPS

What is Multi-tenancy ?

Many Parties share the same set of resources,

while giving each one his own space

Why Multi-tenancy? 1. Increased sharing

• Cloud shares

resources across a

large pool of users.

• Now sharing

happens in the

application level as

oppose to sharing at

OS level for multiple

processes and

sharing at HW level

with VMs.

• That can bring

greater savings

photo by Ben Gray on Flickr, http://www.flickr.com/photos/ben_grey/4582294721/, Licensed under CC

“There is no delight in owning anything unshared.”

Seneca (Roman philosopher, mid-1st century AD)

Why Multi-tenancy? 2. Provide “pay for what you

use”

• Often there will be many accounts

in a PaaS or a SaaS, but only a

fraction of them will be in use.

• We cannot allocate runtime

resource per account (disk may be

ok, as it is cheap). For example,

we cannot run a VM per account.

• By sharing the same server with

many users, Multi-tenancy provides

much reduced runtime cost per

server.

How does it Help? • All WSO2 products support

• multi-tenancy

• Running in the cloud in pay as you go fashion

• So you can share same physical server across

many (e.g. departments) logically

• Or you can outsource some parts of the design

to a PaaS (e.g. Stratos Live)

Recent Performance Improvements Native support for JSON

Pass-through support for ESB

Streaming Xpath and XSLT

CEP 2.0

Native Support for JSON Now WSO2 platform has native support for

JSON through Gson library

ESB Passthrough and Streaming XSLT

ESB Passthrough and Streaming XSLT ESB request = message + headers

If mediation logic only depends on headers, we

skip message building o Binary relay – we copy the message as bytes from

input side to output side

o Pass-through – we copy messages at NIO buffer

level

If mediation logic does not edit the message,

but read the message, we build the message,

but write the incoming bytes out as is to avoid

serialization.

ESB Improved Performance

Complex Event Processor 2.0 Now CEP server 2.0 comes with a new runtime

called Siddhi

Very fast o About 2.5M events/sec with java events

o About 250k events/sec over data bridge

Supports Distributed Cache and Persistence for

HA

More details from o http://wso2.com/products/complex-event-

processor/

Tuning OS Level Tuning

o For example,

http://www.lognormal.com/blog/2012/09/27/linu

x-tcpip-tuning/

o Generally available with the product

o It depends on OS etc.

Product Tuning o Thread pools sizes mainly, there may be others

Application Tuning o This is your application, it is good idea to do a

profile and see.

Performance of a Server

What we expect from Tuning Goals

o Max Throughput

o Within Given Latency

How do we know we are at Max? o Does resource consumption is high?

o What are the resources - CPU

- Memory

- IO (Network and Disk)

o At last one should max out

o E.g. if you have too many locks, all above 3 may be

low and you get sub-optimal throughput

We fix problems via profiling o Detective time!!

Profiling First step is connect through JConsole, and get

basic idea.

Also monitor the load average, CPU usage, and

IO operations (via top or Sar)

Then you can switch to tools like JProfiler,

Yourkit

Important views o CPU views (bottlenecks)

o Thread views and monitor views

o Memory views (GC, allocations, and memory leaks)

o IO and JDBC tracing

Some Profiling views

Some Profiling views

Conclusion Goals: Optimizing the deployments

o Message performance

o Reduce footprint

Design and Capacity Planning

Design for Performance and Minimal footprint o Select right transport

o Minimize footprint and maximize sharing

- MT

- Using Carbon

o Tuning

We also discussed some of the new

performance improvements.

Questions?