www.hazelcast.com

WHO AM IChristoph Engelbert (@noctarius2k)8+ years of professional Java developmentSpecialized to performance, GC, traffictopicsApache DirectMemory PMCPrevious companies incl. Ubisoft and HRSOfficial Inhabitant of HazelcastiaCastMapR MapReduce for Hazelcast 3Co-Author Java Off-heap JEP Proposal

www.hazelcast.com

TOPICSJava Memory LayoutDefinition of Off-Heap?Direct Memory Access?Why do we need Off-Heap Memory?Options for Off-Heap StorageAdvantages and Disadvantages of (Off-)HeapAvailable Frameworks(Not so) Secret FutureSmall demonstration

www.hazelcast.com

JAVA MEMORY LAYOUTA FAST WALK-THROUGH

www.hazelcast.com

JAVA MEMORY LAYOUT (1/5)

www.hazelcast.com

JAVA MEMORY LAYOUT (2/5)

YOUNG GENERATION - EDEN SPACEOnly used for Object-AllocationTLAB¹ allocation availableAffected by Minor and Major GCObjects moved to Survivor on Minor GC

¹ TLAB: Thread Local Allocation Buffer

www.hazelcast.com

JAVA MEMORY LAYOUT (3/5)

YOUNG GENERATION - SURVIVOR SPACESAlways one From-Space and one To-SpaceFrom-Space contains living objects on last GCMinor GC switches the Survivor SpacesAlive Objects are moved to To-SpaceAffected by Minor and Major GCLong living objects eventually moved to Tenured

www.hazelcast.com

JAVA MEMORY LAYOUT (4/5)

TENURED SPACEContains only long living objectsAffected only by Major GCMore objects means more GC spend timeConcurrent object inspectation availableLong Stop-The-World pauses possibleNot optimal for big datasets

www.hazelcast.com

JAVA MEMORY LAYOUT (5/5)

PERMGEN / META SPACEClass bytecodesClass metadataRuntime informationCode Compilation Cacheetc.

www.hazelcast.com

G1 (GARBAGE FIRST COLLECTOR)Generational GCHeap splitted into same sized regionsEvery region is either Eden, Survivor or Tenured space

www.hazelcast.com

DEFINITION OFOFF-HEAP?

www.hazelcast.com

Off-Heap is a (continuously) self allocated, managed and freed direct memory region.

It is not under control of the Java GarbageCollector and needscustom written allocation and cleanup of data-regions.

Off-Heap can not be used for allocation of Java objects.

DEFINITION OF OFF-HEAP?

www.hazelcast.com

I'M DUKE SKYWALKERI'M HERE TO RESCUE YOU!

www.hazelcast.com

DIRECT MEMORYACCESS?

GIVE YOURSELF TO THE DARK SIDE

www.hazelcast.com

DIRECT MEMORY ACCESS?ARE YOU SERIOUS?

Fast memory areaNot affecting the GCOfficially available since Java 1.4Serialization overhead when storing objectsLimited by -XX:MaxDirectMemorySize

www.hazelcast.com

WHY DO WE NEEDOFF-HEAP MEMORY?

www.hazelcast.com

WHY DO WE NEED OFF-HEAP MEMORY?Storing of huge datasetsZero-Copy writes to channelsLower pressure on GC / less GC pausesStorage Space only limited by RAMCompact data representation possibleIPC SharedMemory with Memory-Mapped-Files

www.hazelcast.com

OPTIONS FOROFF-HEAP STORAGE

www.hazelcast.com

OPTIONS FOR OFF-HEAP STORAGE (1/3)

JNI (Java Native Interface)int* buffer = ( int* ) malloc( 1024 * sizeof(int) );if( buffer == NULL ) throwOutOfMemoryException();for( int i = 0; i < 1024; i++ ) buffer[sizeof(int) * i] = i;free( buffer );

www.hazelcast.com

OPTIONS FOR OFF-HEAP STORAGE (2/3)

(Direct)ByteBufferByteBuffer buffer = ByteBuffer.allocateDirect( 1024 * 4 );for( int i = 0; i < 1024; i++) buffer.putInt( i );// buffer is automatically freed by GC

www.hazelcast.com

OPTIONS FOR OFF-HEAP STORAGE (3/3)

sun.misc.UnsafeUnsafe unsafe = trickToRetrieveUnsafe();long address = unsafe.allocateMemory( 1024 * 4 );for( int i = 0; i < 1024; i++ ) unsafe.putInt( address + 4 * i, i );unsafe.freeMemory( address );

www.hazelcast.com

ADVANTAGESAND

DISADVANTAGESOF (OFF-)HEAP

www.hazelcast.com

ADVANTAGES ON-HEAPNo messing with mallocAutomatic Garbage CollectionNo need for Serialization

www.hazelcast.com

DISADVANTAGES ON-HEAPSlowdown on high allocation rateBig overhead on small objects (header)Heavily depending on GC Combination

www.hazelcast.com

ADVANTAGES OFF-HEAPNo limitation* in sizeCompact data layoutZero-Copy socket read/write possibleFrameworks available to manage memory

www.hazelcast.com

DISADVANTAGES OFF-HEAPAllocation is up to youDeallocation is up to youData layout is up to youSerialization may be requiredJNI requires native library

www.hazelcast.com

AVAILABLEFRAMEWORKS

www.hazelcast.com

AVAILABLE FRAMEWORKS (1/5)Apache DirectMemoryTerracotta BigMemoryHazelcast ElasticMemoryMapDB.orgetc.

www.hazelcast.com

AVAILABLE FRAMEWORKS (2/5)

Apache DirectMemoryCacheService<String, String> cacheService = new DirectMemory<...>() .setNumberOfBuffers( 10 ).newCacheService();

cacheService.put( "MyKey", "SomeValue" );String value = cacheService.get( "MyKey" );

www.hazelcast.com

AVAILABLE FRAMEWORKS (3/5)

Terracotta BigMemory<ehcache xml:noNamespaceSchemaLocation="..." name="MyCacheDefinition"> <cache name="MyOffheapCache" maxBytesLocalOffHeap="2G" /></ehcache>

CacheManager cacheManager = new CacheManager();Cache dataStore = cacheManager.get( "MyOffheapCache" );Element element = new Element( "MyKey", "SomeValue" );dataStore.put( element );String value = (String) dataStore.get( "MyKey" ).getObjectValue();

www.hazelcast.com

AVAILABLE FRAMEWORKS (4/5)

Hazelcast Offheap<hazelcast xml:noNamespaceSchemaLocation="..."> <map name="MyMap"> <storage-type>OFFHEAP</storage-type> </map></hazelcast>

HazelcastInstance hz = Hazelcast.newInstance();// Returns a IMap extends ConcurrentMapMap<String, String> map = hz.getMap( "MyMap" );map.put( "MyKey", "SomeValue" );String value = map.get( "MyKey" );

www.hazelcast.com

AVAILABLE FRAMEWORKS (5/5)

MapDB.orgDB db = DBMaker.newDirectMemoryDB().sizeLimit( 2 ).make();// Returns a HTreeMap extends ConcurrentMapMap<String, String> map = db.createHashMap( "cache" ).make();map.put( "MyKey", "SomeValue" );String value = map.get( "MyKey" );

www.hazelcast.com

FURTHER INFORMATION AVAILABLEThe Must-Read for Off-Heap

http://bit.ly/must-read-off-heap

PacketObject Description from IBMhttp://bit.ly/packet-objects

Compact Datastructures (Martin Thompson)http://bit.ly/compact-datastructures

C++ Like Performance for Serializationhttp://bit.ly/serialization-performance

Tricks with Direct Memory Access in Javahttp://bit.ly/direct-memory-tricks

www.hazelcast.com

THE

OFF-HEAP JEP PROPOSALMAY THE FORCE BE WITH YOU.

THE FORCE IS STRONG WITH THIS ONE.

www.hazelcast.com

OFF-HEAP JEP PROPOSALByteBuffer like interfaceFully 64-bit sizes and offsetsCompare-And-Swap operationsVolatile and ordered operationsOptional bounding checksSupports Memory mappingSupport Foreign Function Interface (FFI) JEP

www.hazelcast.com

OFF-HEAP JEP PROPOSAL - CODE EXAMPLEDISCLAIMER: PROVISIONAL API

import javax.direct.*;

BytesFactory factory = createBytesFactory();Bytes bytes = factory.boundsChecking( false ).deallocationChecks( false ) .freeOnGC( false ).create( ByteOrder.LITTLE_ENDIAN, 1024 * 4 );

for( int i = 0; i < 1024; i++ ) bytes.putVolatileInt( i );

bytes.release();

www.hazelcast.com

OFF-HEAP JEP PROPOSALProposal Discussion Group

http://bit.ly/offheap-ml

Proposal Texthttp://bit.ly/proposal-text

FFI JEP 191http://bit.ly/ffi-jep

www.hazelcast.com

@noctarius2k@hazelcast

http://www.sourceprojects.comhttp://github.com/noctarius

THANK YOU!ANY QUESTIONS?

Images: www.clipartist.info, Gnome Nebula Theme, KDE theme, www.grabsteine-klnt.de

www.hazelcast.com