Coherence For Extreme Performance, Availablity and Scalability

© C2B2 Consulting Limited 2011www.c2b2.co.uk

All Rights Reserved

Oracle Coherence for Extreme Performance, Scalability and

AvailabilitySteve Millidge

Director

C2B2

http://www.c2b2.co.uk/


All Rights Reserved

“Reliability, Availability, Scalability and Performance are prerequisites

for functionality!”

They are Priority 1 Requirements



All Rights Reserved

Availability

• System is available for customers to use

• No availability results in no transactions

• Transactions = $$$• Receive your Pink

Slip if you can’t sort it!



All Rights Reserved

Multipliers in Availability

System1

System2

System3

99% Availability 99% Availability 99% Availability

Overall Availability = 0.99*0.99*0.99 = 97%



All Rights Reserved

HA Techniques

Redundancy Decoupling

System

System

99% Availability

99% Availability

Pair = 1 – (0.01*0.01) = 99.99%

Overall = 0.9999 x 0.9999 x 0.9999 = 99%

System 1

99% Availability

System 2

System 3

99% Availability

99% Availability

Overall = 99%



All Rights Reserved

Performance

How fast does a single transaction take to execute!

• Faster Performance = Happier Customers• Faster Performance = More Transactions



All Rights Reserved

Barriers to Performance

• Raw Algorithmic Performance• Resource Limitations

– Not enough cpu, disk, memory• Resource Contention

– Locks • IO Latency

– Network, Disk



All Rights Reserved

Latency

Time delay in requesting an operation and it being initiated

• Key factor in large scale distributed applications

• Typically not taken into account during development



All Rights Reserved

Latency Factors

• Network Distance• Network Reliability• Data Size• Operation Granularity• Resource Contention• JVM GC



All Rights Reserved

Move the Data and ProcessingClose Together



All Rights Reserved

Scalability

Ability to add more hardware in response to more demand.

Without a reduction in performance!



All Rights Reserved

Business Imperatives

• Success of the Business or Service• Growth of Mobile• Huge Variation of Load through a period• Sudden Large Spikes due to events

Cloud Enables Elastic Scalability



All Rights Reserved

Scaling OutHorizontal Scaling

• Add Additional Servers

• Add Load Balancer• Distribute traffic

across the servers• Much Cheaper than

Scale Up• Has HA benefits



All Rights Reserved

Linear Scalability(Nirvana)

1 2 3 40

100

200

300

400

500

600

700

800

900

Linear ScalabilityTypical Scalability

Users

Cluster Nodes



All Rights Reserved

Typical Scale Out Architecture

Node1

Load Balancer

Node3

Node4

Node2

Database

Nodes Host Stateless Services

Database containsPersistent State



All Rights Reserved

Stateless Services

True Stateless Services• Static HTML Serving• Basic Calculations• State Received from

Client

Pseudo Stateless• Read, Update and Store

state in the DB• Use sticky session to

route to non critical state• Typical of Most Online

applications• Push scalability issue to

the database



All Rights Reserved

Scaling a Stateless Middletier is easy

however

Scaling Databases is hard and very expensive



All Rights Reserved

Radical Idea

Put state back into the Middleware



All Rights Reserved

Caching



All Rights Reserved

Read Through Cache

Application

Data Store

Cache

GE

T A

A

AA

Cache Loader



All Rights Reserved

Write Through Cache

Application

Cache

PU

T B

GE

T BB

BB

Data Store

Cache Writer



All Rights Reserved

Write Behind Cache

Application

Cache

PU

T BB

B

Data Store

Write Behind Processor

GE

T B

B



All Rights Reserved

Caches

• Caches aren’t New– Hibernate Session Cache– Entity Bean Cache– JPA Cache– Custom Caches– Open Source Caches

• Typically Cache Database Data or Page Fragments



All Rights Reserved

Local Caching (Roll your Own)

Benefits• Pretty Simple to Write

– Concurrent Hashmap

• Used in many applications

• Use JCache API

Challenges• Cache Eviction• Cache Loading/Storing• Cache Prefetching• Cache Refresh• Write Behind Processing• Clustering !!

THINK LONG AND HARD!!



All Rights Reserved

Clustering Challenges

Application

Cache

B

Application

CacheB

GE

T B

GE

T B

B

B

Data Store



All Rights Reserved

Update Replication

Application

Cache

Application

CacheB1 B1

UP

DA

TE

BB2

B2B2

Data Store



All Rights Reserved

Update Invalidation

Application

Cache

Application

Cache B1B1

UP

DA

TE

BB2

B2 Invalidate

Data Store



All Rights Reserved

Replication Write Performance

Application

Cache

Application

Cache

Application

Cache

Application

Cache

PU

T

BB

BBBB



All Rights Reserved

Cache Partitioning

Application

Cache

Application

Cache

Application

Cache

Application

Cache

PU

T

BB

B

GE

T

B

B BCC

PU

T C

C



All Rights Reserved

Elasticity in Partitioned Caches

Application

Cache

Application

Cache



All Rights Reserved

HA Cache Partitioning

Application

Cache

Application

Cache

Application

Cache

Application

Cache

PU

T

BB

BB

NODECRASH!!!

B



All Rights Reserved

Partitioned Cache

• Linear Scalability– 2 hops for Read (Worst Case)– 2 hops for Write (Worst Case)

• High Availability– Configurable Duplicates

• Location Independent Access– Grid knows where data is

• More Nodes = More Data in Memory



All Rights Reserved

Consider a Large Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache



All Rights Reserved

How Much Can We Store

• 21 Amazon xLargeMemory Instances– 17Gb RAM

• 3 Nodes Per Instance– 4Gb 64bit JVM Heap + 5 Gb OS

• 63 Cluster Nodes• 252 Gb JVM Heap Available• Approx 125Gb Data in the Coherence• Cost per Month $9000



All Rights Reserved

Coherence can Even Overflow

Application

Cache

• Passivates Data to a Local Backing Store (NIO memory mapped file)

• Use Java NIO for Off Heap Storage

• Berkely DB local Storage

• Elastic Data (Journaled SSD Store)

Local Drive



All Rights Reserved

HA In Memory Data

Data Centre

Server Rack 1 Server Rack 2

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

CacheData Centre

Server Rack 1 Server Rack 2

ApplicationCache

ApplicationCache

ApplicationCache

ApplicationCache

ApplicationCache

ApplicationCache

Do We Need the Database?



All Rights Reserved

Database as Business Audit

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

ApplicationCach

e

Data Store

Business Audit Data



All Rights Reserved

Now you Have a DATA GRID



All Rights Reserved

So Much More than an L2 Cache



All Rights Reserved

Computation on Coherence

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid Node

Process



All Rights Reserved

In Place Processing

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid Node

ProcessProcessProcessProcess



All Rights Reserved

Application

Grid Node

Querying Coherence

Application

Grid Node

Application

Grid Node

Application

Grid Node

QueryQueryQueryQuery



All Rights Reserved

Coherence Events Subsystem

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid NodeMapListener



All Rights Reserved

Putting it All Together

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

ApplicationCac

he

JEEClusterNode

JEEClusterNode

JEEClusterNode

JEEClusterNode

JEEClusterNode

Load BalancerWeb Sockets

ProcessProcessProcess

Data Grid



All Rights Reserved

BE RADICAL

Build New Architectures

With Coherence!



All Rights Reserved


Technology

Coherence For Extreme Performance, Availablity and Scalability