Pimp my gc - Supersonic Scala

  • View
    1.727

  • Download
    0

Embed Size (px)

DESCRIPTION

How to tune the Garbage Collector for Scala Applications ? Is immutability an issue for the GC ? How to tune the heap ? Well, the answers are here !

Text of Pimp my gc - Supersonic Scala

  • 1. Pimp my GC Supersonic Scala ! @pingtimeout Scala.IO 24&25 oct 13
  • 2. /me Pierre Laporte Java Performance Tuning Trainer Perfs issues, logs GC eye-compliant http://www.pingtimeout.fr @pingtimeout pierre@pingtimeout.fr @pingtimeout Scala.IO 24&25 oct 13
  • 3. Agenda 42 minutes of Fun (Practice) Fun (Feedbacks) Fun (Questions/Answers) Fun (Theory) Fun (Trolls) Because performance is fun ! @pingtimeout Scala.IO 24&25 oct 13
  • 4. Disclaimer Be critical with the information contained in this talk JVM Tuning is always made on a case-by-case basis. There is no magic, no special set of flags that produces good results on every project. The resemblance of any opinion, recommendation or comment made during this presentation to performance tuning advice is merely coincidental. @pingtimeout Scala.IO 24&25 oct 13
  • 5. Weak Generational Hypothesis 101 @pingtimeout Scala.IO 24&25 oct 13
  • 6. Theory Weak Generational Hypothesis @pingtimeout Scala.IO 24&25 oct 13
  • 7. Theory Weak Generational Hypothesis @pingtimeout Scala.IO 24&25 oct 13
  • 8. Theory Weak Generational Hypothesis Most objects die young Possible scales : MB, GB, TB Minutes, hours, days @pingtimeout Scala.IO 24&25 oct 13
  • 9. Examples Weak Generational Hypothesis Total : 145 GB Avg : 48 GB/j GB 3j @pingtimeout Scala.IO 24&25 oct 13
  • 10. Examples Weak Generational Hypothesis Total : 30 TB Avg : 3TB/j TB 10j @pingtimeout Scala.IO 24&25 oct 13
  • 11. Examples Weak Generational Hypothesis 35 GB/j Play 2 Scala Akka 3 TB/j Java Tomcat Jax-RS / Spring / Hibernate... @pingtimeout Scala.IO 24&25 oct 13
  • 12. Examples Weak Generational Hypothesis Don't forget ! Be critical Case-by-case analysis Please don't do that ----> @pingtimeout Scala.IO 24&25 oct 13
  • 13. JVM Heap 101 @pingtimeout Scala.IO 24&25 oct 13
  • 14. Theory Memory pools Java Heap 2 memory pools (Except for G1 GC) Young Generation for... young objects Old Generation for... old objects !!! Amazing, right ? @pingtimeout Scala.IO 24&25 oct 13
  • 15. Theory Memory pools @pingtimeout Scala.IO 24&25 oct 13
  • 16. Theory Memory pools Young Generation = Eden + Survivors Every object is created in Eden* * : except when it is too big to fit in Eden * : except in special cases for G1 GC @pingtimeout Scala.IO 24&25 oct 13
  • 17. Theory Memory pools @pingtimeout Scala.IO 24&25 oct 13
  • 18. Why memory pools ?! Always 2 GC per JVM* * Except for G1 GC Young GC Cheap Duration mostly O(Live data in YG) Old GC Expensive Duration mostly O(Live data in OG) @pingtimeout Scala.IO 24&25 oct 13
  • 19. Why memory pools ?! Common GC Name Young Gen GC Old Gen GC Parallel GC PSYoungGen ParOldGen ParNew CMS CMS G1 GC @pingtimeout G1 Scala.IO 24&25 oct 13
  • 20. GC Duration ?! Prove it! @pingtimeout Scala.IO 24&25 oct 13
  • 21. App with small live set @pingtimeout Scala.IO 24&25 oct 13
  • 22. App with big live set @pingtimeout Scala.IO 24&25 oct 13
  • 23. Experiment 1 1st run (SmallLiveSet) 50 GB heap (-ms50g -mx50g) 49.9GB Young Gen (-Xmn49900m) GC logs @pingtimeout Scala.IO 24&25 oct 13
  • 24. Experiment 1 1st run (SmallLiveSet) 50 GB heap 49.9GB Young Gen -ms50g -mx50g -Xmn49900m GC logs Result : 6ms YGC pauses to free 38GB of memory @pingtimeout Scala.IO 24&25 oct 13
  • 25. Experiment 1 : Result [PSYoungGen: 38329728K->6496K(44710400K)] 38329744K->6512K(46041600K), 0.0067050 secs] //... 38.329.728K data before GC in YG, 6.496K after YG size is 44.710.400K 38.329.744K data before GC in heap, 6.512K after Heap size is 46.041.600K Total pause time : 6.7ms @pingtimeout Scala.IO 24&25 oct 13
  • 26. Experiment 2 2nd run (SmallLiveSet) 50 GB heap (-ms50g -mx50g) 10MB Young Gen (-Xmn10m) GC logs @pingtimeout Scala.IO 24&25 oct 13
  • 27. Experiment 2 1st run (SmallLiveSet) 50 GB heap 10MB Young Gen -ms50g -mx50g -Xmn10m GC logs Result : 322ms Full GC pauses to free 52GB of memory @pingtimeout Scala.IO 24&25 oct 13
  • 28. Experiment 2 : Result [Full GC [PSYoungGen: 3072K->0K(7168K)] [ParOldGen: 52418151K->30287K(52418560K)] 52421223K->30287K(52425728K)//... 0.3229410 secs] 52.418.151K data before GC in OG, 30.287K after OG size is 52.418.560K 52.421.223K data before GC in heap, 30.287K after Heap size is 52.425.728K Total pause time : 322.9ms @pingtimeout Scala.IO 24&25 oct 13
  • 29. Experiments 1->4, Wrap up 1st and 2nd runs with BigLiveSet Ran out of time* :-( *: Stopped measuring at Heap occupancy 22GB GC Pauses : Live set Small Big @pingtimeout 6 millis 55 secs (Full GC)* Scala.IO 24&25 oct 13 322 millis (Full GC) 250 secs (Full GC)*
  • 30. Immutability Is immutability a problem ? @pingtimeout Scala.IO 24&25 oct 13
  • 31. Immutability What does this code do ? (GC point of view ?) @pingtimeout Scala.IO 24&25 oct 13
  • 32. Immutability @pingtimeout Scala.IO 24&25 oct 13
  • 33.