Data Grids for Extreme Performance, Scalability and Availability JavaOne 2011 Steve Millidge

Data Grids for Extreme Performance, Scalability and

AvailabilitySteve Millidge

Director

“Reliability, Availability, Scalability and Performance are prerequisites

for functionality!”

They are Priority 1 Requirements

Availability

• System is available for customers to use

• No availability results in no transactions

• Transactions = $$$• Receive your Pink

Slip if you can’t sort it!

Multipliers in Availability

System1

System2

System3

99% Availability 99% Availability 99% Availability

Overall Availability = 0.99*0.99*0.99 = 97%

HA Techniques

Redundancy Decoupling

System

99% Availability

Pair = 1 – (0.01*0.01) = 99.99%

Overall = 0.9999 x 0.9999 x 0.9999 = 99%

System 1

99% Availability

System 2

System 3

99% Availability

Overall = 99%

Performance

How fast does a single transaction take to execute!

• Faster Performance = Happier Customers• Faster Performance = More Transactions

Barriers to Performance

• Raw Algorithmic Performance• Resource Limitations

– Not enough cpu, disk, memory• Resource Contention

– Locks • IO Latency

– Network, Disk

Latency

Time delay in requesting an operation and it being initiated

• Key factor in large scale distributed applications

• Typically not taken into account during development

Latency Factors

• Network Distance• Network Reliability• Data Size• Operation Granularity• Resource Contention• JVM GC

Move the Data and ProcessingClose Together

Scalability

Ability to add more hardware in response to more demand.

Without a reduction in performance!

Business Imperatives

• Success of the Business or Service• Growth of Mobile• Huge Variation of Load through a period• Sudden Large Spikes due to events

Cloud Enables Elastic Scalability

Scaling OutHorizontal Scaling

• Add Additional Servers

• Add Load Balancer• Distribute traffic

across the servers• Much Cheaper than

Scale Up• Has HA benefits

Linear Scalability(Nirvana)

1 2 3 40

Linear ScalabilityTypical Scalability

Cluster Nodes

Typical Scale Out Architecture

Load Balancer

Database

Nodes Host Stateless Services

Database containsPersistent State

Stateless Services

True Stateless Services• Static HTML Serving• Basic Calculations• State Received from

Client

Pseudo Stateless• Read, Update and Store

state in the DB• Use sticky session to

route to non critical state• Typical of Most Online

applications• Push scalability issue to

the database

Scaling a Stateless Middletier is easy

however

Scaling Databases is hard and very expensive

Radical Idea

Put state back into the Middleware

Caching

Read Through Cache

Application

Data Store

Cache Loader

Write Through Cache

Application

Data Store

Cache Writer

Write Behind Cache

Application

Data Store

Write Behind Processor

Caches

• Caches aren’t New– Hibernate Session Cache– Entity Bean Cache– JPA Cache– Custom Caches– Open Source Caches

• Typically Cache Database Data or Page Fragments

JSR 107JCACHE - Java Temporary Caching API

• Been around a Long Time– 10 years

• Focussed on Java SE– With some JEE Integration for JEE7

• Caching API– V get(Object key) throws CacheException;– void put(K key, V value) throws

CacheException;

JSR 107Get Involved

• Google Group for Discussion– http://groups.google.com/group/jsr107

• Google Docs for Spec– https://docs.google.com/document/d/1YZ-lrH6

nW871Vd9Z34Og_EqbX_kxxJi55UrSn4yL2Ak• GitHub for Code

– https://github.com/jsr107/jsr107spec

Local Caching (Roll your Own)

Benefits• Pretty Simple to Write

– Concurrent Hashmap

• Used in many applications

• Use JCache API

Challenges• Cache Eviction• Cache Loading/Storing• Cache Prefetching• Cache Refresh• Write Behind Processing• Clustering !!

THINK LONG AND HARD!!

Clustering Challenges

Application

CacheB

Data Store

Update Replication

Application

CacheB1 B1

Data Store

Update Invalidation

Application

Cache B1B1

B2 Invalidate

Data Store

Replication Write Performance

Application

Cache Partitioning

Application

Elasticity in Partitioned Caches

Application

HA Cache Partitioning

Application

NODECRASH!!!

Partitioned Cache

• Linear Scalability– 2 hops for Read (Worst Case)– 2 hops for Write (Worst Case)

• High Availability– Configurable Duplicates

• Location Independent Access– Grid knows where data is

• More Nodes = More Data in Memory

Consider a Large Cache

Application

How Much Can We Store

• 21 Amazon xLargeMemory Instances– 17Gb RAM

• 3 Nodes Per Instance– 4Gb 64bit JVM Heap + 5 Gb OS

• 63 Cluster Nodes• 252 Gb JVM Heap Available• Approx 125Gb Data in the Grid!• Cost per Month $9000

Grids can Even Overflow

Application

• Passivates Data to a Local Backing Store (NIO memory mapped file)

• Use Java NIO for Off Heap Storage

• Berkely DB local Storage

• Reduces GC overheadLocal Drive

HA In Memory Data

Data Centre

Server Rack 1 Server Rack 2

Application

CacheData Centre

Server Rack 1 Server Rack 2

ApplicationCache

Do We Need the Database?

Database as Business Audit

ApplicationCach

Data Store

Business Audit Data

Now you Have a DATA GRID

So Much More than an L2 Cache

Computation on the Grid

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid Node

Process

In Place Processing

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid Node

ProcessProcessProcessProcess

Application

Grid Node

Querying the Grid

Application

Grid Node

Application

Grid Node

Application

Grid Node

QueryQueryQueryQuery

Data Grid Events Subsystem

Application

Grid Node

Application

Grid Node

Application

Grid Node

Application

Grid NodeMapListener

Putting it All Together

ApplicationCac

JEEClusterNode

Load BalancerWeb Sockets

ProcessProcessProcess

Data Grid

BE RADICAL

Build New Architectures

With Data Grids!

Extreme Performance

• Reduced Latency– Data close to processing– Reduce roundtrips and expensive calculations

• Parallel Processing– Distributed Processing (Map-Reduce-like)– Distributed Query Processing

Extreme Scalability

• O(1) Writes and Reads– Worst Case two hops– No increase with number of nodes

• Data Volume Increases with Nodes– Large data volumes stored in the Data Grid

• Elastic Topology– Clusters Rebalance with node changes

Extreme Availability

• No Single Point of Failure– Duplicates prevent data loss– Duplicate Numbers Configurable

• Write Behind – decouples Database Availability

• Self Healing– Removing Nodes causes rebalancing

Data Grids for Extreme Performance, Scalability and Availability JavaOne 2011 Steve Millidge

Technology

JavaOne 2010 Keynote

Icycle: On Thin Ice, A Postmortem by Reece Millidge

JavaOne 2015 Event Demographics - Oracle...Title JavaOne 2015 Event Demographics Author Oracle Corporation Subject JavaOne 2015 Event Demographics Keywords "Java, Java FY15, PowerPoint

Autumn collection JavaOne 2014

（JavaOne） COMMUNITY JAVAONEのための - Oracle...ORACLE.COM/JAVAMAGAZINE ///// JULY/AUGUST 2013 COMMUNITY 07 JAVA TECH ABOUT US JAVA IN ACTION blog （JavaOne） JavaOne 2012の「Community

Participate @JavaOne Brazil 2012

HJUG JavaOne 2k14 Wrapup

JavaOne Tokyo LT : Internationalization

DataFX - JavaOne 2013

Retour JavaOne 2009

2013 michael coates-javaone

JavaOne 2015 Keynote Presentation

DataFX 8 (JavaOne 2014)

Project Zero JavaOne 2008

Live Demo from JavaOne

JCP.Next Panel at JavaOne

Linkedin Javaone

JavaOne 2011 Report

JavaOne 2013 Report

Javaone ts4693 v2