What is cool in Java 8 and new in 9cr.openjdk.java.net/~thartmann/talks/2017-Cool_in_Java_8_and_new... · Java source code int i = 0; do {i++;} while (i < f()); Bytecodes 0: iconst_0

$Page 1: What is cool in Java 8 and new in 9cr.openjdk.java.net/~thartmann/talks/2017-Cool_in_Java_8_and_new... · Java source code int i = 0; do {i++;} while (i < f()); Bytecodes 0: iconst_0$
Copyright © 2017, Oracle and/or i ts affiliates. All rights reserved.

What is cool in Java 8 and new in 9From a VM engineer‘s perspective

Tobias Hartmann

Compiler Group – Java HotSpot Virtual MachineOracle Corporation

March 2017


About me

• Software engineer in the HotSpot JVM Compiler Team at Oracle– Based in Baden, Switzerland

• Master’s degree in Computer Science from ETH Zurich

• Worked on various compiler-related projects

– Currently working on future Value Type support for Java


Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

3


Outline

• Intro: The HotSpot Java Virtual Machine

• Part 1: What's cool in Java 8

– Background: JIT compilation in HotSpot

– Tiered Compilation

• Part 2: What's new in Java 9

– Segmented Code Cache

– Compact Strings

– Ahead-of-time Compilation

4


A typical computing platform

5

Hardware

Operating system

Java Virtual Machine

User Applications

Java EEJava SE

Application Software

System Software



6

Hardware

Operating system


User Applications

Java EEJava SE


System Software



7

Hardware

Operating system


User Applications

Java EEJava SE


System Software


Programming language implementation

C

Windows

Intel x86

8

Operatingsystem

Languageimplementation

Hardware

Programminglanguage

CompilerStandard librariesDebuggerMemory management

Linux

Intel x86


Linux

ARM


Solaris

SPARC


Copyright © 2017, Oracle and/or i ts affiliates. All rights reserved. 9

(Language) virtual machine

Java

Windows

Intel x86

Operatingsystem

Virtual machine

Hardware

Programminglanguage

HotSpot VM

PPC ARM SPARC

Mac OS X SolarisLinux

JavaScript Scala Python


The VM: An application developer’s view

Java source code

int i = 0;do {

i++;} while (i < f());

Bytecodes

0: iconst_01: istore_12: iinc5: iload_16: invokestatic f9: if_icmplt 2

12: return

compileHotSpotJava VMexecute

• Ahead-of-time• Using javac

• Instructions for an abstract machine• Stack-based machine (no registers)

10


The VM: A VM engineer’s view

11

Bytecodes

0: iconst_01: istore_12: iinc5: iload_16: invokestatic f9: if_icmplt 2

12: return

HotSpot Java VM

Garbage collector

manage

Interpreter

executeHeap

Stack

access

access

Compilation system

compile

C1

C2

Compiled methodproduce

Machine code

Debug info

Object maps


Outline






– Segmented Code

– Compact Strings

– Ahead-of-Time Compilation

• Conclusion

12


Interpretation vs. compilation in HotSpot• Template-based interpreter

– Generated at VM startup (before program execution)

– Maps a well-defined machine code sequence to every bytecode instruction

• Compilation system

– Speedup relative to interpretation: ~100X

– Two just-in-time compilers (C1, C2)

– Aggressive optimistic optimizations

13

Bytecodes0: iconst_01: istore_12: iinc5: iload_16: invokestatic f9: if_icmplt 2

12: return

Machine codemov -0x8(%r14), %eaxmovzbl 0x1(%r13), %ebxinc %r13mov $0xff40,%r10jpmq *(%r10, %rbx, 8)

Load local variable 1

Dispatch next instruction


Ahead-of-time vs. just-in-time compilation

• AOT: Before program execution

• JIT: During program execution

– Tradeoff: Resource usage vs. performance of generated code

14

Performance

Amount of compilationInterpretation Compile everything

Bad performancedue to interpretation

Bad performancedue to compilation overhead

Good performancedue to good selection of compiled methods and applied optimizations


JIT compilation in HotSpot

• Resource usage vs. performance– Getting to the “sweet spot”

1. Selecting methods to compile

2. Selecting compiler optimizations

15



• Hot methods (frequently executed methods)

• Profile method execution

– # of method invocations, # of backedges

• A method’s lifetime in the VM

16

Interpreter Compiler (C1 or C2) Code cache

Gather profiling information Compile bytecode to native code Store machine code

# method invocations > THRESHOLD1

# of backedges > THRESHOLD2

Deoptimization

Compiler’s optimistic assumptionsproven wrong


False

Control flow graph Generated code

Example optimization:

S1;S2;S3;if (x > 3)

S4; S5;S6;S7;

S8;S9;

10’000 0

guard(x > 3)S1;S2;S3;S4;S8;S9;

Deoptimize

17

True

Hot path compilation


Example optimization:

18

class A {

void bar() {S1;

}

}

class B extends A {

void bar() {S2;

}

}

void foo() {

A a = create(); // return A or Ba.bar();

}

Class hierarchy Method to be compiled

loaded

not loaded

Compiler:Inline call?Yes.

Virtual call inlining


Example optimization: Virtual call inlining

• Benefits of inlining

– Virtual call avoided

– Code locality

• Optimistic assumption: only A is loaded

– Note dependence on class hierarchy

– Deoptimize if hierarchy changes

19

class A {

void bar() {S1;

}

}

class B extends A {

void bar() {S2;

}

}

void foo() {

A a = create(); // return A or BS1;

}


loaded

not loaded

Compiler:Inline call?Yes.


Example optimization: Virtual call inlining

20

class A {

void bar() {S1;

}

}

class B extends A {

void bar() {S2;

}

}

void foo() {

A a = create(); // return A or Ba.bar();

}


loaded

loaded

Compiler:Inline call?No.


Deoptimization

• Compiler’s optimistic assumption proven wrong

– Assumptions about class hierarchy

– Profile information does not match method behavior

• Switch execution from compiled code to interpretation

– Reconstruct state of interpreter at runtime

– Complex implementation

• Compiled code

– Possibly thrown away

– Possibly reprofiled and recompiled

21


Performance effect of deoptimization

• Follow the variation of a method’s performance

22

Performance

Time

Interpreted Compiled Interpreted Compiled

Compilation DeoptimizationVM Startup Compilation


Background: JIT compilation in HotSpot

• Resource usage vs. performance– Getting to the “sweet spot”



23



• C1 compiler

– Limited set of optimizations

– Fast compilation

– Small footprint

• C2 compiler

– Aggressive optimistic optimizations

– High resource demands

– High-performance code

• Graal

– Experimental compiler

– Will be part of HotSpot for AOT in JDK 9

24

Client VM

Server VM

Tiered Compilation(enabled since JDK 8)


Outline






– Segmented Code

– Compact Strings


• Conclusion

25


Tiered Compilation

• Introduced in JDK 7, enabled by default in JDK 8

• Combines the benefits of

– Interpreter: Fast startup

– C1: Fast compilation

– C2: High peak performance

• Within the sweet spot

– Faster startup

– More profile information

26


Benefits of Tiered Compilation

27

Performance

Time

VM Startup

Interpreted C1-compiled

warm-up time

Client VM (C1 only)

Compilation



28

PerformanceInterpreted C2-compiled

warm-up time

Server VM (C2 only)

Time

VM Startup Compilation



29

PerformanceInterpreted C1-compiled

warm-up time

Tiered compilation

C2-compiled

Time

VM Startup Compilation Compilation


Additional benefit: More accurate profiling

time

Interpreter C1 (profiled) C2 (non-profiled)

Interpreter

Profiling without tiered compilation

Profiling with tiered compilation

C2 (non-profiled)

300 samples

100 samples 1000 samples

100 samples 200 samples

w/ tiered compilation: 1’100 samples gatheredw/o tiered compilation: 300 samples gathered

30


Tiered Compilation

• Combined benefits of interpreter, C1, and C2

• Additional benefits

– More accurate profiling information

• Drawbacks

– Complex implementation

– Careful tuning of compilation thresholds needed

– More pressure on code cache

31


A method’s lifetime (Tiered Compilation)

Interpreter C1 C2

Code cache

Collect profiling information Generate code quicklyContinue collectingprofiling information

Generate high-quality codeUse profiling information

Deoptimization

32


Performance of a method (Tiered Compilation)

33

Performance

Time

VM Startup

Interpreted C1 compiled C2 compiled

Compilation Compilation

Interpreted C2 compiled

Deoptimization Compilation


Compilation levels (detailed view)

Interpreter

C1: no profiling

C1: limited profiling

C1: full profiling

C2

0

1

2

3

4

Co

mp

ilati

on

leve

lTypical compilation sequence

Associated thresholds:Tier3InvokeNotifyFreqLog

Tier3BackedgeNotifyFreqLogTier3InvocationThresholdTier3MinInvocationThreshold

Tier3BackEdgeThresholdTier3CompileThreshold

Associated thresholds:Tier4InvocationThreshold

Tier4MinInvocationThresholdTier4CompileThresholdTier4BackEdgeThreshold

34


• Lambda Expressions

• Language Interop• Nashorn

• Parallel operations for core collections APIs

• Improvements in functionality

• Improved type inference

• Tiered Compilation

• PermGen removal• Performance improvements

• Profiles for constrained devices

• JSR 310: Date & Time APIs

• Non-Gregorian calendars

• Unicode 6.2

• ResourceBundle

• BCP47 locale matching

• Globalization & Accessibility

• JSR 308: Annotations on Java Type

• Native app bundling

• App Store Bundling tools

• jdeps

• Deployment enhancements

• JavaFX 8

• Public UI Control API

• Java SE Embedded support

• Enhanced HTML5 support

• 3D shapes and attributes

• Printing

• Limited doPrivilege

• NSA Suite B algorithm support

• SNI Server Side support

• DSA updated to FIPS186-3

• AEAD JSSE CipherSuites

• Mission Control

• Flight Recorder

• Usage Tracker

• Advanced Management Console

• MSI Enterprise JRE Installer

35

InnovationCore libraries

Security

Java for Everyone

Client

ToolsJVM enhancements

Enterprise

JDK 8 features


Outline






• Conclusion

36


• Segmented Code Cache• Ahead-of-Time Compilation

• Store Interned Strings in CDS Archives

• Improve Contended Locking• Compact Strings• Improve Secure Application Performance

• Leverage CPU Instructions for GHASH and RSA• Tiered Attribution for javac• Javadoc Search

• Marlin Graphics Renderer• HiDPI Graphics on Windows and Linux• Enable GTK 3 on Linux

• Update JavaFX/Media to Newer Version of GStreamer

• Jigsaw – Modularize JDK

• Enhanced Deprecation• Stack-Walking API• Convenience Factory Methods for Collections

• Platform Logging API and Service• jshell: The Java Shell (Read-Eval-Print Loop)• Compile for Older Platform Versions

• Multi-Release JAR Files• Platform-Specific Desktop Features• TIFF Image I/O

• Multi-Resolution Images

• Process API Updates

• Variable Handles• Spin-Wait Hints• Dynamic Linking of Language-

Defined Object Models• Enhanced Method Handles• More Concurrency Updates

• Compiler Control

Behind the scenes

New functionality

Specialized

• Parser API for Nashorn• Prepare JavaFX UI Controls & CSS APIs for Modularization

• Modular Java Application Packaging• New Version-String Scheme

• Reserved Stack Areas for Critical Sections• Segmented Code Cache• Ahead-of-Time Compilation

• Indify String Concatenation• Unified JVM Logging• Unified GC Logging

• Make G1 the Default Garbage Collector• Use CLDR Locale Data by Default• Validate JVM Command-Line Flag Arguments

• Java-Level JVM Compiler Interface• Disable SHA-1 Certificates• Deprecate the Applet API

• Process Import Statements Correctly• Annotations Pipeline 2.0• Elide Deprecation Warnings on Import Statements

• Filter Incoming Serialization Data

• Milling Project Coin• Simplified Doclet API

• Remove GC Combinations Deprecated in JDK 8

• Remove Launch-Time JRE Version Selection• Remove the JVM TI hprof Agent• Remove the jhat Tool

Housekeeping

Removed

• HTTP 2 Client

• Unicode 8.0• UTF-8 Property Files• ECMAScript 6 Features in Nashorn

• Datagram Transport Layer Security (DTLS)• OCSP Stapling for TLS• TLS Application-Layer Protocol

Negotiation Extension• SHA-3 Hash Algorithms• DRBG-Based SecureRandom Implementations

• Create PKCS12 Keystores by Default

• Merge Selected Xerces 2.11.0 Updates into JAXP

• HarfBuzz Font-Layout Engine

• XML Catalogs• HTML5 Javadoc

New standards

JDK 9 features

• Compact Strings


Outline


• Part1: What's cool in Java 8



• Part2: What's new in Java 9

– Segmented Code

– Compact Strings


• Conclusion

38


What is the code cache?

• Stores code generated by JIT compilers

• Continuous chunk of memory

– Managed (similar to the Java heap)

– Fixed size

• Essential for performance

39


Code cache usage: JDK 6 and 7

free space

VM internals

compiled code

40


Code cache usage: JDK 8 (Tiered Compilation)

41

free space

VM internals

C1 compiled (profiled)

C2 compiled (non-profiled)


Code cache usage: JDK 9

42

free space

VM internals

C1 compiled (profiled)

C2 compiled (non-profiled)


Challenges

• Tiered compilation increases amount of code by up to 4X

• All code is stored in a single code cache

• High fragmentation and bad locality

• But is this a problem in real life?

43


Code cache usage: Reality

profiled code

non-profiled code

free space


Code cache usage: Reality

hotness

profiled code

non-profiled code

free space


Design: Types of compiled code

Optimization level Size Cost Lifetime

Non-method code optimized small cheap immortal

Profiled code (C1) instrumented medium cheap limited

Non-profiled code (C2) highly optimized large expensive long

46


• Without Segmented Code Cache • With Segmented Code Cache

47

Design

Code Cache

non-profiled methods

profiled methods

non-methods


non-profiled methodsprofiled methods

48

Segmented Code Cache: Reality

profiled code

non-profiled code

free space


Segmented Code Cache: Reality

non-profiled methodsprofiled methods hotness


Evaluation: Responsiveness

• Sweeper (GC for compiled code)

50

0

5

10

15

20

25

30

35

40

# full sweeps Cleanup pause time Sweep time

Re

du

ctio

n i

n %


Evaluation: Performance

51

0

2

4

6

8

10

12

14

SPECjbb2005 SPECjbb2013 JMH-Javac Octane (Typescript) Octane (Gbemu)

Imp

rove

me

nt

in %


What we have learned

• Segmented Code Cache helps– To reduce the sweeper overhead and improve responsiveness

– To reduce memory fragmentation

– To improve code locality

• And thus improves overall performance

• To be released with JDK 9

52


Outline






– Segmented Code

– Compact Strings


• Conclusion

53


public class HelloWorld {public static void main(String[] args) {

String myString = "HELLO";System.out.println(myString);

}}

Java Strings

public final class String {private final char value[];...

}

char value[] =

H

0x0048 0x0045 0x004C 0x004C 0x004F

2 bytes

E L L O

UTF-16 encoded


“Perfection is achieved, not when there is nothing more to add, but when there is nothing more to take away.”

– Antoine de Saint Exupéry

55


There is a lot to take away here..

• UTF-16 encoded Strings always occupy two bytes per char

• Wasted memory if only Latin-1 (one-byte) characters used:

• But is this a problem in real life?

56

char value[] =

H

0x0048 0x0045 0x004C 0x004C 0x004F

2 bytes

E L L O

0x0048 0x0045 0x004C 0x004C 0x004F


Real life analysis: char[] footprint

• 950 heap dumps from a variety of applications– char[] footprint makes up 10% - 45% of live data

– Majority of characters are single byte

• Predicted footprint reduction of 5% - 10%

57


Project Goals

• Memory footprint reduction by improving space efficiency of Strings

• Meet or beat performance of JDK 9

• Full compatibility with related Java and native interfaces

• Full platform support– x86/x64, SPARC, ARM

– Linux, Solaris, Windows, Mac OS X

58


Design

• String class now uses a byte[] instead of a char[]

• Additional 'coder' field indicates which encoding is used

59

public final class String {private final byte value[];private final byte coder;...

}

H E L L O

byte value[] = 0x00 0x48 0x00 0x45 0x00 0x4C 0x00 0x4C 0x00 0x4F

byte value[] = 0x48 0x45 0x4C 0x4C 0x4F

UTF-16 encoded

Latin-1 encoded

H E L L O


Design

• If all characters have a zero upper byte→ String is compressed to Latin-1 by stripping off high order bytes

• If a character has a non-zero upper byte→ String cannot be compressed and is stored UTF-16 encoded

60

byte value[] = 0x00 0x48 0x00 0x45 0x00 0x4C 0x00 0x4C 0x00 0x4F

byte value[] = 0x48 0x45 0x4C 0x4C 0x4F

UTF-16 encoded

Latin-1 encoded

InflationCompression

0x47


Design

• Compression / inflation needs to fast

• Requires HotSpot support in addition to Java class library changes

– JIT compilers: Intrinsics and String concatenation optimizations

– Runtime: String object constructors, JNI, JVMTI

– GC: String deduplication

• Kill switch to enforce UTF-16 encoding (-XX:-CompactStrings)

– For applications that extensively use UTF-16 characters

61


Evaluation: Performance

• SPECjbb2005

– 21% footprint reduction

– 27% less GCs

– 5% throughput improvement

• SPECjbb2015


– 11% critical-jOps improvement

• Weblogic (startup)


– 5% startup time improvement


62


Outline






– Segmented Code

– Compact Strings


• Conclusion

63


Ahead-of-Time Compilation

• Compile Java classes to native code prior to launching the VM

• AOT compilation is done by new jaotc tool

– Uses Java based Graal compiler as backend

– Stores code and metadata in shared object file

• Improves start-up time

– Limited impact on peak performance

• Sharing of compiled code between VM instances

64


Revisit: Performance of a method (Tiered Compilation)

65

Performance

Time

VM Startup

Interpreted C1 compiled C2 compiled





Performance of a method (Tiered AOT)

66

Performance

Time

VM Startup

C2 compiled




AOT compiled C1 compiled


Ahead-of-Time Compilation

• Experimental feature– Supported on Linux x64

– Limited to the java.base module

• Try with your own code - feedback is welcome!


– More to come in future releases

67


Summary

• Many cool features in Java 8– Tiered Compilation

• More features to come with Java 9 in July 2017– Segmented Code Cache, Compact Strings, Ahead-of-Time compilation

• Java – A vibrant platform

– Early access releases are available:

https://jdk9.java.net/download/

68

"Our SaaS products are built on top of Java and the Oracle DB—that’s the platform.”Larry Ellison, Oracle CTO

https://jdk9.java.net/download/



Evaluation: Code locality

• Instruction Cache (ICache)– 14% less ICache misses

• Instruction Translation Lookaside Buffer (ITLB1)– 44% less ITLB misses

• Overall performance

– 9% speedup with microbenchmark

70

1 caches virtual to physical address mappings to avoid slow page walks


public class LogLineBench {int size;

String method = generateString(size);

public String work() throws Exceptions {return "[" + System.nanoTime() + "] " +

Thread.currentThread().getName() +"Calling an application method \"" + method +"\" without fear and prejudice.";

}

Microbenchmark: LogLineBench


LogLineBench results

Performance ns/op Allocated b/op

1 10 100 1 10 100

Baseline 149 153 231 888 904 1680

CS disabled 152 150 230 888 904 1680

CS enabled 142 139 169 504 512 904

72

• Kill switch works (no regression)

• 27% performance improvement and 46% footprint reduction

Documents

What is cool in Java 8 and new in 9cr.openjdk.java.net/~thartmann/talks/2017-Cool_in_Java_8_and_new... · Java source code int i = 0; do {i++;} while (i < f()); Bytecodes 0: iconst_0