Upload
mattias-jiderhamn
View
143
Download
0
Embed Size (px)
DESCRIPTION
Are you tired of java.lang.OutOfMemoryError: PermGen space? Then this talk is for you! We'll begin with a crash course in the Java memory model in order to understand what the error message means. Then we'll look at different causes of the error and how to avoid them. We may glance at a few interesting mistakes from the Open Source world. Last but not least you'll learn how you can get rid of java.lang.OutOfMemoryError: PermGen space once and for all.
Citation preview
Mattias Jiderhamn – java.jiderhamn.se
Classloader leaks
Mattias Jiderhamn
Mattias Jiderhamn – java.jiderhamn.se
java.lang.OutOfMemoryError: PermGen space
:-(
Mattias Jiderhamn – java.jiderhamn.se
Local dev env
Mattias Jiderhamn – java.jiderhamn.se
Continuous Deploy
Mattias Jiderhamn – java.jiderhamn.se
Java Memory Model
Stack
Heap
PermGen
Mattias Jiderhamn – java.jiderhamn.se
Java Memory ModelPer threadLocal variables and
method parametersStack
Mattias Jiderhamn – java.jiderhamn.se
Java Memory Model
HeapYoung generation Old generation /
tenured spaceEdenspace
Survivorspace
Object instances
Mattias Jiderhamn – java.jiderhamn.se
Java Memory ModelPermanent Generationjava.lang.Class instances
etcNamed before JEE and
class unloadingRenamed Metaspace in Java 8
PermGen
Mattias Jiderhamn – java.jiderhamn.se
java.lang.OutOfMemoryError: PermGen space
= Too many classes are loaded
Java Memory Model
Mattias Jiderhamn – java.jiderhamn.se
Reason for OOME
1. Your application is too large-XX:MaxPermSize=256MJava 8: Metaspace auto increase by default
2. java.lang.Class instances could not be garbage collected after redeploy
java.lang.OutOfMemoryError: PermGen space
Mattias Jiderhamn – java.jiderhamn.se
RedeployApplication Server
ClassLoader
app.war
ClassLoader
app.war’
ClassLoader
app.war’’
Mattias Jiderhamn – java.jiderhamn.se
Reference types• Strong (i.e. normal) reference–Never GC:ed if reachable
• Soft reference–GC:ed before OutOfMemoryError
•Weak reference (WeakHashMap)–GC:ed when no Strong or Soft refs
• Phantom reference–… won’t prevent GC
Mattias Jiderhamn – java.jiderhamn.se
Example
Foo
WeakHashMap
Map m = new WeakHashMap();Foo myFoo = new Foo();m.put(myFoo, ”bar”);myFoo = null;
Mattias Jiderhamn – java.jiderhamn.se
GC reachability
GC roots
Mattias Jiderhamn – java.jiderhamn.se
ClassLoader refsApplication Server
ClassLoader
Class ClassClass
Instance
Mattias Jiderhamn – java.jiderhamn.se
How leaks happenApplication Server
ClassLoader
Class ClassClass
InstanceGC root
Mattias Jiderhamn – java.jiderhamn.se
Application Server• Application Server bugs• Logging frameworks
– Apache Commons LoggingUnless LogFactory.release()
– Log4j - some configsUnless LogManager.shutdown()
– java.util.logging custom Level• Bean Validation API (JSR 303)• Unified Expression Language (javax.el)
Mattias Jiderhamn – java.jiderhamn.se
GC roots• Class loaded by system ClassLoader
– static field in JDK classes (java.* etc)• Live thread
– Stack – local vars, method params–java.lang.Thread instance
• Object held as synchronization monitor• JNI references• JVM specials…
Mattias Jiderhamn – java.jiderhamn.se
System classes•java.sql.DriverManager• Bean introspection cache, shutdown hooks,
custom default Authenticator, custom security Provider, custom MBeans, custom ThreadGroup, custom property editor, …
• Reference to contextClassLoader of first caller
Mattias Jiderhamn – java.jiderhamn.se
DriverManager
Mattias Jiderhamn – java.jiderhamn.se
DriverManagerApplication Server
app.war
mysql-jdbc.jar
JRE
DriverManager
ClassLoader
com.mysql.jdbc.Driver
1) …
… or 2)DriverManager.deregisterDriver(…)
Mattias Jiderhamn – java.jiderhamn.se
Context shutdownpublic class MyCleanupListener implements
javax.servlet.ServletContextListener {
...
/** Called when application is undeployed */
public void contextDestroyed(
ServletContextEvent servletContextEvent) {
DriverManager.deregisterDriver(…);
}
}
Mattias Jiderhamn – java.jiderhamn.se
Threads• Thread stack
– Local variables– Method parameters
• MyThread extends ThreadMyRunnable implements Runnable
• contextClassLoader– Example HTTP 1.1 Keep-Alive-Timer
Mattias Jiderhamn – java.jiderhamn.se
Context shutdownpublic class MyCleanupListener implements
javax.servlet.ServletContextListener {
...
/** Called when application is undeployed */
public void contextDestroyed(
ServletContextEvent servletContextEvent) {
DriverManager.deregisterDriver(…);
// Stop threads here!
}
}
Mattias Jiderhamn – java.jiderhamn.se
Stopping Threadspublic class MyThread extends Thread {
public void run() { while(true) { // Bad idea! // Do something } }
}
Mattias Jiderhamn – java.jiderhamn.se
Stopping Threadspublic class MyThread extends Thread {
private boolean running = true;
public void run() { while(running) { // Until stopped // Do something } }
public void shutdown() { running = false; }}
private volatile boolean running = true;
Heinz Kabutz / Java Specialists’ - The Law of the Blind Spot
Mattias Jiderhamn – java.jiderhamn.se
Threads• Thread stack
– Local variables– Method parameters
• MyThread extends ThreadMyRunnable implements Runnable
• contextClassLoader– Example HTTP 1.1 Keep-Alive-Timer
• ThreadLocal
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal
WeakHashMap<Thread, ?>
ThreadLocal
Thread Foo
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocalThreadLocal JavaDoc
”Each thread holds an implicit reference to its copy of a thread-local variable as long
as the thread is alive and the ThreadLocal instance is accessible;
…”
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal
ThreadLocalMapEntry
Thread
ThreadLocal Foo
put()
set()
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocalPooled threads:•Threads may outlive ClassLoader•ThreadLocal → ThreadGlobal!
(?)
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal
ThreadLocalMapEntry
Thread
ThreadLocal Foo
ClassClassLoader
static
Class
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal
ThreadLocalMapEntry
Thread
ThreadLocal Foo
ClassClassLoader
Stale entry
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocalThreadLocalMap JavaDoc
”However, since reference queues are not used, stale entries are guaranteed
to be removed only when the table starts running out of space”
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal•Custom value (incl references)–static ThreadLocal = leak–otherwise = unpredicted GC:ing
•Custom ThreadLocal = no leak
Mattias Jiderhamn – java.jiderhamn.se
ThreadLocal
try { myThreadLocal.set(foo); …}finally { myThreadLocal.remove();}
Always clear your ThreadLocals!
Mattias Jiderhamn – java.jiderhamn.se
Known offendersApache ActiveMQApache AxisApache BatikApache Commons Pool / DBCPApache CXFAWT ToolkitBean Validation API / JSR 303CGLIB (Hibernate / Spring / JBoss /
Apache Geronimo)dom4jEclipseLinkGeoToolsGoogle GuiceGroovyGWT / javax.imageioHessianiCal4JInfinispanIntrospectionUtils
JarURLConnectionJava Advanced Imaging (JAI)JavassistJava Cryptography Architecture (JCA)Java Server Faces 2javax.security.auth.Policy/login.ConfigurationJGroupsLogbackJAXBMozilla RhinoMVELOpenOffice Java Uno RunTime (JURT)Oracle JDBCRMISerialization (<= JDK 1.4.2)Spring frameworkUnified Expression Language / javax.elURLConnection + HTTP 1.1XML parsing (DocumentBuilderFactory)
Mattias Jiderhamn – java.jiderhamn.se
Leak prevention lib• Application server independent• Covers much more than Tomcat
– System class references avoided/cleared– Threads are killed– ThreadLocals are cleared– Known offenders handled
• Logs warnings• Apache 2 licensed
– You may modify and redistribute
Mattias Jiderhamn – java.jiderhamn.se
Leak prevention lib
0 runtime dependencies1 class2 XML snippets
Mattias Jiderhamn – java.jiderhamn.se
Leak prevention libpom.xml
<dependency>
<groupId>se.jiderhamn</groupId>
<artifactId>classloader-leak-prevention</artifactId>
<version>1.9.2</version>
</dependency>
web.xml
<listener>
<listener-class>se.jiderhamn.classloader.leak.prevention.ClassLoaderLeakPreventor
</listener-class>
</listener>
Mattias Jiderhamn – java.jiderhamn.se
Tomcat
Bugzilla #48895
ThreadLocalMap
ThreadCleanupThread
remove()
Unsafe!
get()set()
remove()
Removed in 6.0.27
Tomcat 7.0.6+ renews the thread pool whenever an application is redeployed.
Disposable threads for lifecycle events.Bugzilla #49159
Mattias Jiderhamn – java.jiderhamn.se
Leak prevention lib
set(null) ThreadLocalMapEntry
Thread
ThreadLocal Foo
Stale
CleanupThread
Mattias Jiderhamn – java.jiderhamn.se
Bean Validation APIVersion 1.0.0.GA
javax.validation.Validation
Application Servervalidation-api-1.0.0.GA.jar
app.warhibernate-validator.jar
Mattias Jiderhamn – java.jiderhamn.se
javax.el APIjavax.el.BeanELResolver
Mattias Jiderhamn – java.jiderhamn.se
OpenOffice JURTcom.sun.star.lib.util.AsynchronousFinalizer
Mattias Jiderhamn – java.jiderhamn.se
GC choice matters• Parallell GC seems to have problems
collecting java.lang.Class instances• CMS needs extra parameter
-XX:+UseConcMarkSweepGC -XX:+CMSClassUnloadingEnabled
• G1-XX:+UseG1GC
Mattias Jiderhamn – java.jiderhamn.se
Linkshttp://java.jiderhamn.se
• Leak prevention library incl sources (GitHub)• Step by step guide: acquire heap
dump and analyze for leaks usingEclipse Memory Analyzer (MAT)
• List of known offenders and links to bug reports• Submit your own reports and patches!
Que?Join the fight!