Upload
others
View
3
Download
0
Embed Size (px)
Citation preview
JSR107: The new Caching Standard
Greg Luck CTO Terracotta/Founder Ehcache
JFokus 2012
Wednesday, 15 February 2012
What is Caching?Temporary Storage of data or results that are likely to be used more than once
3
Wednesday, 15 February 2012
Caching Characteristics• Fastest To Implement• Offload• Performance• Scale up• Scale out (Distributed Caches Only)• Buffer against load variability
4
Wednesday, 15 February 2012
Maximising Cache Efficiency
cache efficiency = cache hits / total hits
➡ High efficiency = high offload➡ High efficiency = high performance
5
Wednesday, 15 February 2012
Sca
le U
P
Scale OUT
Commodity Server
Application
Types of Scaling
StandaloneCaching(in-process)
DistributedCaching
Types of Caching
Wednesday, 15 February 2012
824
Local Disk Store(Re-Startable)
Local Storage
HeapStore
BigMemoryOff-Heap Store
5,000,000+
1,000,000
2
500
1,000+
Speed (TPS) Size (GB)
Scaling Example: Ehcache
100,000
Wednesday, 15 February 2012
824
Local Disk Store(Re-Startable)
Local Storage
HeapStore
BigMemoryOff-Heap Store
5,000,000+
1,000,000
2
500
1,000+
Speed (TPS) Size (GB)
Scaling Example: Ehcache
100,000
10,000s
Network Storage
Terracotta Server Array 1,000s
Wednesday, 15 February 2012
9
Network Topology Example: Ehcache
36
L1 L2
ELC
Eventual consistencyStrong consistencyLocal Transac=onsExplicit LockingXA Transac=ons
PACAc=ons on par==on: -‐ excep=on, -‐ noop-‐ local reads
Par==on healing: -‐ reconnect-‐ rejoin
Wednesday, 15 February 2012
Compared to NoSQL• NoSQL focused on persistence - Caching on temporary Storage
• NoSQL focused on BigData - Caching on valuable data• Caching focused on RAM storage• Caches are key-value stores, like key-value NoSQL• Caching is a use case for NoSQL• Much Lower latencies
10
Wednesday, 15 February 2012
10
•! 95/5 Read/update
Comment: Sherpa does very well here, with better read latency – only one lookup into a B-tree is needed for reads, unlike log-structured systems where records must be reconstructed. Cassandra also performs well, matching Sherpa until high throughputs. HBase does well also, although read time is higher.
Workload B – Read heavy
0
2
4
6
8
10
12
14
16
18
20
0 2000 4000 6000 8000 10000
Ave
rag
e r
ea
d la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Read latency
Cassandra HBase Sherpa MySQL
0
5
10
15
20
25
30
35
40
0 2000 4000 6000 8000 10000
Ave
rag
e u
pd
ate
la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Update latency
Cassandra Hbase Sherpa MySQL
The code is available publicly on GitHub: https://github.com/brianfrankcooper/YCSB
Comparative Speeds
Wednesday, 15 February 2012
11
10
•! 95/5 Read/update
Comment: Sherpa does very well here, with better read latency – only one lookup into a B-tree is needed for reads, unlike log-structured systems where records must be reconstructed. Cassandra also performs well, matching Sherpa until high throughputs. HBase does well also, although read time is higher.
Workload B – Read heavy
0
2
4
6
8
10
12
14
16
18
20
0 2000 4000 6000 8000 10000
Ave
rag
e r
ea
d la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Read latency
Cassandra HBase Sherpa MySQL
0
5
10
15
20
25
30
35
40
0 2000 4000 6000 8000 10000
Ave
rag
e u
pd
ate
la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Update latency
Cassandra Hbase Sherpa MySQL
The code is available publicly on GitHub: https://github.com/brianfrankcooper/YCSB
Comparative Speeds
Wednesday, 15 February 2012
11
10
•! 95/5 Read/update
Comment: Sherpa does very well here, with better read latency – only one lookup into a B-tree is needed for reads, unlike log-structured systems where records must be reconstructed. Cassandra also performs well, matching Sherpa until high throughputs. HBase does well also, although read time is higher.
Workload B – Read heavy
0
2
4
6
8
10
12
14
16
18
20
0 2000 4000 6000 8000 10000
Ave
rag
e r
ea
d la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Read latency
Cassandra HBase Sherpa MySQL
0
5
10
15
20
25
30
35
40
0 2000 4000 6000 8000 10000
Ave
rag
e u
pd
ate
la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Update latency
Cassandra Hbase Sherpa MySQL
The code is available publicly on GitHub: https://github.com/brianfrankcooper/YCSB
Compared with hybrid in-‐process and distributed cache:
Latency = L1 speed * propor=on + L2 speed * propor=on
Comparative Speeds
Wednesday, 15 February 2012
11
10
•! 95/5 Read/update
Comment: Sherpa does very well here, with better read latency – only one lookup into a B-tree is needed for reads, unlike log-structured systems where records must be reconstructed. Cassandra also performs well, matching Sherpa until high throughputs. HBase does well also, although read time is higher.
Workload B – Read heavy
0
2
4
6
8
10
12
14
16
18
20
0 2000 4000 6000 8000 10000
Ave
rag
e r
ea
d la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Read latency
Cassandra HBase Sherpa MySQL
0
5
10
15
20
25
30
35
40
0 2000 4000 6000 8000 10000
Ave
rag
e u
pd
ate
la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Update latency
Cassandra Hbase Sherpa MySQL
The code is available publicly on GitHub: https://github.com/brianfrankcooper/YCSB
Compared with hybrid in-‐process and distributed cache:
Latency = L1 speed * propor=on + L2 speed * propor=on
L1 = 0ms (< 5us) for on-‐heap and 50-‐100 us off-‐heapL2 = 2-‐3ms
80% L1 Pareto Model:
= 0 * .8 + 3 * .2= .6 ms
Comparative Speeds
Wednesday, 15 February 2012
11
10
•! 95/5 Read/update
Comment: Sherpa does very well here, with better read latency – only one lookup into a B-tree is needed for reads, unlike log-structured systems where records must be reconstructed. Cassandra also performs well, matching Sherpa until high throughputs. HBase does well also, although read time is higher.
Workload B – Read heavy
0
2
4
6
8
10
12
14
16
18
20
0 2000 4000 6000 8000 10000
Ave
rag
e r
ea
d la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Read latency
Cassandra HBase Sherpa MySQL
0
5
10
15
20
25
30
35
40
0 2000 4000 6000 8000 10000
Ave
rag
e u
pd
ate
la
ten
cy (
ms)
Throughput (operations/sec)
Workload B - Update latency
Cassandra Hbase Sherpa MySQL
The code is available publicly on GitHub: https://github.com/brianfrankcooper/YCSB
Compared with hybrid in-‐process and distributed cache:
Latency = L1 speed * propor=on + L2 speed * propor=on
L1 = 0ms (< 5us) for on-‐heap and 50-‐100 us off-‐heapL2 = 2-‐3ms
80% L1 Pareto Model:
= 0 * .8 + 3 * .2= .6 ms
90% L1 Pareto Model:
latency = 0 * .9 + 3 * .1= .3 ms
Comparative Speeds
Wednesday, 15 February 2012
Compared to (Concurrent)Map
12
• a Map is an in-process key-value store
Even local in caches add:
- expiry- eviction once full of least valuable entries
• Map is always store by reference• Caches typically are distributed
Wednesday, 15 February 2012
JSR107: Java Caching Standard• javax.cache.Cache• Being developed by JSR107• Java 6 and above is required• Included in JSR 342: Java EE 7 due end of 2012• Immediately usable by Java EE 6 and Spring• Immediately usable by any Java based app
13
Wednesday, 15 February 2012
Open, Transparent Standards Approach• Terracotta and Oracle have tasked an FTE (Greg and Yannis) with developing the spec
• Developed in the open• 15 expert group members• Lots of healthy debate. See the mailing list: [email protected]
• Specification is standard spec license - free to use and implement
• Reference Implementation is Apache 2• Tests which is the major part of the TCK is Apache 2
14
Wednesday, 15 February 2012
Expected Implementations• Terracotta - Ehcache• Oracle - Coherence• JBoss - Infinispan• IBM - ExtemeScale• SpringSource - Gemfire• GridGain • TMax• Google App Engine Java memcache client• Spymemcache memcache client
15
Wednesday, 15 February 2012
Getting StartedAPI in Maven Central<dependency> <groupId>javax.cache</groupId> <artifactId>cache-api</artifactId> <version>0.3-SNAPSHOT</version></dependency>
Everything to get startedhttps://github.com/jsr107/jsr107spec
16
Wednesday, 15 February 2012
Key Concepts• CacheManager => Caches• Cache => Entries • Entry => Key, Value• The basic API can be thought of map--like with the following additional features:
• atomic operations, similar to java.util.ConcurrentMap• read-through caching• write-through caching• cache event listeners• statistics
17
Wednesday, 15 February 2012
API FeaturesMap-like with the following additional features:• atomic operations, similar to java.util.ConcurrentMap• read-through caching• write-through caching• cache event listeners• statistics• transactions including all isolation levels • caching annotations• generics
18
Wednesday, 15 February 2012
How to Please Everyone - No Dependencies• Java SE - no dependencies. • EE/Spring - provided dependencies - they are already there.
19
Wednesday, 15 February 2012
How to Please Everyone - Optional FeaturesOptional Features are:
• storeByReference• XA and Local Transactions • Caching Interceptor Annotations e.g.
Options interrogation at runtime via Capabilities API:• ServiceProvider.isSupported(OptionalFeature feature)• CacheManager.isSupported(OptionalFeature feature)
Works for implementers and Users20
Wednesday, 15 February 2012
Aimed at Standalone and Distributed CachingStandalone Features
• storeByReference - allows speeds similar to CHM• CacheEventListener callbacks - useful for triggering eventsDistributed Features
• storeByValue• NotificationScope in CacheEventListener• modifications/differences to Map and ConcurrentHashMap to reduce network cost. e.g. - No values() and many others. - Calls may not return a value e.g. remove(Object key) returns boolean rather than the old value 21
Wednesday, 15 February 2012
Not a Data Grid Specification• Infinispan, Coherence and Extreme Scale are Data Grids• Ehcache and Memcache are distributed client-server caches• NoSQL key value stores are distributed client-server key stores which could be used for cachingSo:
• JSR107 does not mandate a topology• JSR347 does - it is for data grids and builds on JSR107
22
Wednesday, 15 February 2012
Classloading• Caches contain data shared by multiple threads/JVMs which may be using Java SE, EE, OSGi or custom class loading.
• This makes class loading tricky• A classloader can be specified when the CacheManager is created or a default is used. Either way all classes will be loaded by the CacheManager’s classloader, not the environment’s classloader.
• public static CacheManager getCacheManager(ClassLoader classLoader)
• public static CacheManager getCacheManager(ClassLoader classLoader, String name) 23
Wednesday, 15 February 2012
Creating a CacheManagerServiceLoader CreationWe support the Java 6 java.util.ServiceLoader creational approach. It will automaticaly detect a cache implementation in your classpath. You then create a CacheManager with:CacheManager cacheManager = CacheManagerFactory.getCacheManager();or more fully:CacheManager cacheManager = CacheManagerFactory.getCacheManager(“app1”, Thread.currentThread().getContextClassLoader());
“new” CreationCacheManager cacheManager = new RICacheManager(“app1”, Thread.currentThread().getContextClassLoader());
24
Wednesday, 15 February 2012
Creating a CacheTo programmatically configure a cache named “testCache” which is set for read-through
CacheManager cacheManager = getCacheManager();Cache testCache = cacheManager.createCacheBuilder(“testCache).setReadThrough(true).setSize(Size.UNLIMITED)..setExpiry(Duration.ETERNAL).build();
25
Text
Wednesday, 15 February 2012
Using a CacheYou get caches from the CacheManager. To get a cache called “testCache”:
Cache<Integer, Date> cache = cacheManager.getCache(“testCache”);
26
Wednesday, 15 February 2012
Putting a value in a CacheCache<Integer, Date> cache = cacheManager.getCache(cacheName);Date value1 = new Date();Integer key = 1;cache.put(key, value1);
27
Wednesday, 15 February 2012
Getting a ValueCache<Integer, Date> cache = cacheManager.getCache(cacheName);Date value2 = cache.get(key);
28
Wednesday, 15 February 2012
Removing a mappingCache<Integer, Date> cache = cacheManager.getCache(cacheName);Integer key = 1;cache.remove(1);
29
Wednesday, 15 February 2012
Exposing the underlying Cache’s APIUnwrap Method on Cache <T> T unwrap(java.lang.Class<T> cls);
Ehcache Examplenet.sf.ehcache.Cache cache = javax.cache.cache.unwrap(net.sf.ehcache.Cache.class);
30
Wednesday, 15 February 2012
AnnotationsJSR107 introduces a standardised set of caching annotations, which do method level caching interception on annotated classes running in dependency injection containers. Caching annotations are becoming increasingly popular:
•Ehcache Annotations for Spring•Spring 3’s caching annotations.
32
Wednesday, 15 February 2012
Annotation OperationsThe JSR107 annotations cover the most common cache operations including:
•@CacheResult•@CachePut•@CacheRemoveEntry•@CacheRemoveAll
33
Wednesday, 15 February 2012
Specific Overrides
public class DomainDao { @CachePut(cacheName="domainCache") public void updateDomain(String domainId, @CacheKeyParam int index, @CacheValue Domain domain) { ... } }
34
Wednesday, 15 February 2012
Fully Annotated Class Examplepublic class BlogManager {
@CacheResult(cacheName="blogManager") public Blog getBlogEntry(String title) {...}
@CacheRemoveEntry(cacheName="blogManager") public void removeBlogEntry(String title) {...}
@CacheRemoveAll(cacheName="blogManager") public void removeAllBlogs() {...}
@CachePut(cacheName=”blogManager”) public void createEntry(@CacheKeyParam String title, @CacheValue Blog blog) {...}
@CacheResult(cacheName="blogManager") public Blog getEntryCached(String randomArg, @CacheKeyParam String title){...}}
35
Wednesday, 15 February 2012
Wiring Up Spring<beans ...> <context:annotation-config/> <jcache-spring:annotation-driven proxy-target-class="true"/> <bean id="cacheManager" class="javax.cache.Caching" factory-method="getCacheManager" /> <bean class="manager.CacheNameOnEachMethodBlogManagerImpl"/> <bean class="manager.ClassLevelCacheConfigBlogManagerImpl"/> <bean class="manager.UsingDefaultCacheNameBlogManagerImpl"/></beans>
36
Wednesday, 15 February 2012
Wiring Up CDI1. Create an implementation of javax.cache.annotation.BeanProvider
2. Declare a resource named javax.cache.annotation.BeanProvider in the classpath at /META-INF/services/.
For an example using the Weld implementation of CDI, see the CdiBeanProvider in our CDI test harness.
37
Wednesday, 15 February 2012
More InformationJumping Off Point to Everything Elsehttps://github.com/jsr107/jsr107spec
Maven Snippet<dependency> <groupId>javax.cache</groupId> <artifactId>cache-api</artifactId> <version>0.x</version></dependency>
38
Wednesday, 15 February 2012