58
Steve Millidge | C2B2 Fast Data: Parallel Processing on the Grid

Jax London 2013

Embed Size (px)

DESCRIPTION

Fast Data: Parallel Processing on the Grid. You may think of in-memory datagrids as a big cache, a key value store accessed using put and get. However to really reap the power and scale of modern data grids you need to move beyond cache semantics and turn your architecture on its head. Moving the processing to the data rather than pulling the data across the grid massively increases parallelism and reduces network latency, leading to huge increases in throughput. In this session, we will explore why traditional cache semantics sometimes struggles to scale and how through using parallel processing and event processing on the grid we can rearchitect our systems for massive scalability and utilise all our grid cores for parallel processing.

Citation preview

Page 1: Jax London 2013

Steve Millidge | C2B2

Fast Data: Parallel Processing on the Grid

Page 2: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

SAFE HARBOR

This talk may contain nonsense.

Page 3: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

WHO AM I

Founder C2B2Java since 1.0 EG Member JSR107 and JSR347 (bit dead)

Our MissionWe ensure the middleware that forms the foundation of our customer’s mission critical systems is Fast, Reliable, Manageable and Secure.

Page 4: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Data PerformanceProblem?

What do you mean?

Page 5: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 6: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Diagnose Before

You Prescribe

Page 7: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Measure Everything!

Page 8: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 9: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Algorithm?

Page 10: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

CAPACITY?Can you add more?

CPUDiskMemoryBandwidth

Page 11: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

SCALABILITY?

1 2 3 40

100

200

300

400

500

600

700

800

900

Linear Scalability

Typical Scalability

Ability to add more capacity in response to more demand.

Without a reduction in performance!

Page 12: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 13: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

LATENCYTime delay in requesting an operation and it being initiated

Page 14: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

1Gb Ethernet : 25 – 30MB/s10Gb Ethernet : 250 – 350MB/s

Infiniband : 6GB/s (maybe)

Page 15: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

PING TIMESLocal : 57µsLAN segment : 300µsLAN: switches : 4msUK : 30msUSA : 100ms3G : 100s ms

Page 16: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

GRANULARITY?

Page 17: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Typical SSD Speed540MB/s

Source: tomshardware.com

Spinning Rust

Disk BUS SpeedsSATA 1.0 : 150MB/sSATA 2.0 : 300MB/sSATA 3.0 : 600MB/sSAS : 600MB/sFibre Channel : 1GB/sInfiniband : 1GB/s

Page 18: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

DDR3 1600 : 12.8GB/s

Page 19: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Source www.cs.virginia.edu Approx: 3-5GB/s per cpu still about 1 order magnitude faster than SSD and scales with coresThe general rule for STREAM is that each array must be at least 4x the size of the sum of all the last-level caches used in the run, or 1 Million elements -- whichever is larger.

Page 20: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

MEMORYIS

FAST!Cache Data

InMemory

Page 21: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 22: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

“As an extreme example, several current high-end machines run simple arithmetic kernels for out-of-cache operands at 4-5% of their rated peak speeds --- that means that they are spending 95-96% of their time idle and waiting for cache misses to be satisfied”http://www.cs.virginia.edu/stream/ref.html

Page 23: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

PARALLELIS

THE FUTURE!

Page 24: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Other manufacturers are available ;-)

Settle into soft grain leather seats with 10/6 way electric adjustment and heating for driver and passenger. The soft grain stitched instrument panel and Gloss Rich Oak veneer complete the luxurious look, while assertive 18 inch Vela alloy wheels further enhance the dynamic exterior. 

The rear parking aid and Touch-screen visual indicator make slipping into tight spaces easy, while distractions at night are reduced by auto-dimming rear view mirrors. The Meridian™  380W Sound System, with Digital Audio Broadcasting (DAB) radio receiver and Front Media Interface with USB and iPod® connectivity, lets you listen to your favourite music in rich detail.

XF PREMIUM LUXURY

Page 25: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 26: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 27: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

http://www.datasciencecentral.com/forum/topics/the-3vs-that-define-big-data

Page 28: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Application

Key Partition

Application

Key Partition

Application

Key Partition

Application

Key Partition

PU

T

BB

GE

T

B

BBC

PU

T C

C

Page 29: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

public static void main( String[] args )    {        CacheManager cacheManager = Caching.getCachingProvider().getCacheManager();       

MutableConfiguration<String, String> config = new MutableConfiguration<String, String>();

        cacheManager.configureCache("C2B2",config);

        Cache cache = cacheManager.getCache("C2B2");

        cache.put("Key", "Value");

        System.out.println(cache.get("Key"));    }

http://blog.c2b2.co.uk/2013/08/jsr107-getting-started.html

Page 30: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Consistent Hashing

Key SpaceHash Ring

Node 1Hash

Node 2Hash

Node 3Hash

Node 4Hash

Key Hash

Page 31: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Key Partitioning

• Keys Assigned Partition via Hash

• Nodes Own a Partition

• Number of Partitions Fixed

• Number of Nodes isn’t

Partition

Partition

Partition

Partition

Partition

Partition

Partition

Page 32: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 33: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 34: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Application

Key Partition

Application

Key Partition

Application

Key Partition

Application

Key Partition

PU

T

BB

BB

NODECRASH!!!

B

Page 35: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

Application

Cache

21 Amazon m2.xLargeMemory Instance with 17Gb RAM3 Nodes Per Instance with 4Gb 64bit JVM Heap + 5 Gb OS63 Cluster Nodes252 Gb JVM Heap AvailableApprox 125Gb Data in the Grid when taking into account duplicates42 vCPUsApprox $3633 per month

Page 36: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 37: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Application

Key Partition

Application

Key Partition

Application

Key Partition

Application

Key partition

Code

Page 38: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

1Gb Ethernet : 25 – 30MB/s10Gb Ethernet : 250 – 350MB/s

Infiniband : 6GB/s (maybe)

Page 39: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Application

Cache

Application

Cache

Application

Cache

Application

Cache

CodeCodeCodeCode

Page 40: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Class Size of code to Serialize is a few Kb

Page 41: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

// Invoking a MapObject oResult = map.invoke(key, agent);

// Invoke on a subset of the cacheNamedCache map = CacheFactory.getCache("trades");Filter filter = new AndFilter(new EqualsFilter("getTrader", traderid), new EqualsFilter("getStatus", Status.OPEN));Map mapResults = map.invokeAll(filter, agent);

// The Agent Interfacepublic interface EntryProcessor extends Serializable { public Object process(Entry entry); Map processAll(Set setEntries);}

Page 42: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

<T> T invoke(K key,               EntryProcessor<K, V, T> entryProcessor,               Object... arguments) throws EntryProcessorException;

<T> Map<K, T> invokeAll(Set<? extends K> keys,                          EntryProcessor<K, V, T> entryProcessor,                          Object... arguments);

T process(MutableEntry<K, V> entry, Object... arguments)      throws EntryProcessorException;}

Page 43: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

• Financial Risk Calculation– Can I book this Order

• Bet Placement– Can I place this Bet

• Hotel Availability Search– What hotels are available

• Web Translation– Gather Translated Text

Application

Keys

Application

Keys

Application

Keys

Page 44: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

• Consumer Trend Analysis– Who orders what when

• Route Optimisation– Delivery Trucks

• Social Graph Search– Patterns in social media

graphs

• Fraud Detection– Transaction analysis

Application

Keys

Application

Keys

Application

Keys

Page 45: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 46: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

public interface CacheEntryListener<K, V> extends EventListener {

}

public interface CacheEntryUpdatedListener<K, V> extends CacheEntryListener<K, V> {

/*** Called after one or more entries have been updated.** @param events The entries just updated.* @throws CacheEntryListenerException if there is problem executing the listener*/  void onUpdated(Iterable<CacheEntryEvent<? extends K, ? extends V>> events)      throws CacheEntryListenerException;}

Page 47: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

• Product Recommendations– Related products

• Alerting– Truck has Stopped!

• Financial Position Calc– Check regs during booking

• Nudge Purchasing– 5 people are looking at this

hotelApplication

Keys

Application

Keys

Application

Keys

Page 48: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

http://blog.c2b2.co.uk/2012/06/infinispan-event-push-to-tomcat.htmlhttp://demo.c2b2.co.uk:8080/infinispan/

Page 49: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Page 50: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

CoherenceGlassFishWebSocket

http://demo.c2b2.co.uk:7080 or Full URLhttp://demo.c2b2.co.uk:7080/CleanStockTicker/stockticker.jsphttp://blog.c2b2.co.uk/2012/10/java-one-2012-slides-and-code.html

Page 51: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

MEMORYIS

FAST!

Page 52: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Cache DataIn

Memory

Page 53: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

PARALLELIS

THE FUTURE!

Page 54: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Send ProcessingAcross theData Grid

Page 55: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

PUSH IS HERE

Page 56: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

Data Grids + Event Listeners +

WebSockets =

Real Time Push

Page 57: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts

RememberYou Have a

Latency Problem!

Page 58: Jax London 2013

www.c2b2.co.uk blog.c2b2.co.uk @c2b2consulting

The Leading Independent Middleware Experts