Upload
c2b2-consulting
View
6.809
Download
2
Embed Size (px)
DESCRIPTION
'In the Wild'presentation by Steve Millidge
Citation preview
GlassFishMonitoring and Troubleshooting
In the Wild
Steve Millidge
C2B2 Consulting Limited
CAVEATIf this works it’ll be a miracle!
Agenda
• Setting the Scene– A Real World? Scenario
• Command Line Tools
• Graphical Tools
Setting the Scene
Troubleshooting in the Wild!
(An Apology)
C2B2 Consulting
• Fast– Performance and Scalability
• Reliable– High Availability and Recoverability
• Manageable– Monitoring and Supporting
• Secure– Security
First Day At Work
It’s All Gone to *#*@!
Meet the Project Manager
Where do You Start?
JVM Versions
• JDK 1.4– Black Box
• JDK 1.5– You can watch from outside
• JDK 1.6– You can sneak in by the back door
JVM Command Line Tools
• jps– Identified Running Machines on the Host
• jstat– Provides Statistics on Running VMs
• jmap– Provides Memory Information
• jstack– Provides Stack Trace Information
jps
• Lists the Running VMs on a Host
• -l shows you the class name
• -m The Command Line Parameters
• -v full Java Options
GlassFish Command Line Tools
• asadmin– Command line access to GlassFish
• Orientation– list-domains– list-clusters– get-health– generate-diagnostic-report – generate-jvm-report
GlassFish Monitoring
• Check Monitoring– get server.monitoring-service.module-
monitoring-levels.*
• Switch Monitoring On– set server.monitoring-service.module-
monitoring-levels.jvm=LOW
• Monitor– Monitor –type jvm
jstack
• Dumps a stack trace
• Adds monitors and Synchronizers
• C stack as well on Linux
jstat
• Provides Memory Stats– Perm Gen, New Gen, Old Gen
• Provide Class Loading Stats
• Provide Compilation Stats
jinfo
• Provides Information on the Running VM– Command Line– System Properties– VM flags
• Changes the flags in the VM
• Limited use on Windows
jmap
• Gives you visibility into the memory
• See the Class Histogram
• Dump the Memory to a File
jhat
• Heap Analyzer
• Works from a Dump File
• Browse the Heap via a Browser
Graphical Tools
GlassFish AdminConsole
Admin Console
Visual VM
• New in 1.6u7
• Replaces JConsole
• Number of plugins
• Enables Heap Dumps
• Thread Dumps
• Simple Profiling– Pretty Dodgy (Often Crashes VM)
Third Party Tool
Memory Analyzer Tool
• Open Source
• Developed by SAP Support
• Analyses Heap Dumps– Finds Culprits Easily– Indexes the dump for fast navigation
Analysis
• We are Leaking Memory in a static– Class name and variable
• We have a rogue thread spinning the cpu– Full stack trace
Exotica
When things get really tough!
Exotica
• Oracle JRockit
• Attach API
• Instrumentation API
• BTrace
Oracle JRockit
• Supports 1.4, 1.5, 1.6• Ships with Many Advanced Tools• Mission Control• Leak Analyzer
– Looks for Increasing Dominators in Real Time
• Latency Analyzer– Records cause of Thread Latency in Real Time
• Instrumentation in the VM itself
Attach API
• Enables tools to Dynamically Attach to a JVM
// attach to target VM VirtualMachine vm = VirtualMachine.attach("2177");
// construct path to management String agent = "management-agent.jar"; // load agent into target VM vm.loadAgent(agent)
vm.loadAgent(agent);// detach vm.detach();
Agents
• Agent is a JAR file
• Define Premain Class in the Manifest– Has method premain(String args)
• Define Agent Class in the Manifest– Has Method agentmain(String args)
Instrumentation
• JDK 6 support in place Class Replacement
• Agents implementClassFileTransformer
• Register themselves as a Transformer– Instrumentation.addTransformer
• Call restransform– Instrumentation.retransform
BTrace
• Uses Attach API and Instrumentation
• Attaches an Agent into the VM
• Injects “scripts” into a running JVM
• Script transform class files and inject code
• http://btrace.dev.java.net
Example Script
// import all BTrace annotations import com.sun.btrace.annotations.*;
// import statics from BTraceUtils class import static com.sun.btrace.BTraceUtils.*; // @BTrace annotation tells that this is a BTrace program @BTrace public class HelloWorld {
@OnMethod( clazz="java.lang.Thread", method="start" ) public static void func() { println("about to start a thread!"); } }
Thank you
Questions?