Memory Management
Kathryn McKinley
Isn’t GC a bit retro?
“Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.”
Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008
“Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.”
Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008
Mark-CompactStyger, 1967
Mark-SweepMcCarthy, 1960
Semi-SpaceCheney, 1970
2McKinley, UT 395T
MM
Course Logistics
• Syllabus• Critical reading & writing• Presentations• Discussion• Critiques• Schedule• Volunteers for next week?
3McKinley, UT 395T MM
Outline
• Briefly introduce the challenges and key ideas in memory management– Context – modern VMs– Explicit vs automatic– Memory organization– Allocation– Garbage Identification– Reclamation
4McKinley, UT 395T MM
5
Basic VM Structure
Executing Program
Program/Bytecode
Interpreter &/or
Dynamic Compiler
Class Loader Verifier, etc. Heap
Thread Scheduler
Garbage Collector
McKinley, UT 395T MM
Dynamic memory allocation and reclamation
• Heap contains dynamically allocated objects
• Object allocation: malloc, new• Deallocation:
– Manual/explicit: free, delete– automatic: garbage collection
6McKinley, UT 395T MM
Memory Management
• Objects/data in heap memory• How does the runtime system efficiently create and recycle memory on behalf of the program?– What makes this problem important?– What makes this problem hard?– Why are researchers still working on
it?
7McKinley, UT 395T MM
Explicit memory managementchallenges
• More code to maintain• Correctness
– Free an object too soon - core dump– Free an object too late - waste space– Never free - at best waste, at worst
fail• Efficiency can be very high• Gives programmers “control”
8McKinley, UT 395T MM
Garbage collection:Automatic memory
management
• Reduces programmer burden• Eliminates sources of errors
– which ones?• Integral to modern object-oriented
languages – Java, C#, PHP, JavaScript
• Mainstream• Challenge: performance
9McKinley, UT 395T MM
10
Why use Garbage Collection?
• Software engineering benefits– Less user code compared to expilict
memory management (MM)– Less user code to get correct– Protects against some classes of memory
errors• No free(), thus no premature free(), no double
free(), or forgetting to free()
• Not perfect, memory can still leak– Programmers still need to eliminate all
pointers to objects the program no longer needs
McKinley, UT 395T MM
11
Why use Garbage Collection?
• Performance: space/time tradeoff– Time proportional to dead objects
(explicit mm, reference counting) or live objects (semi-space, mark-sweep)
– Throughput versus pause time• Less frequent collection typically reduces
total time but increases space requirements and pause times
– Hidden locality benefits?• Layer of abstraction• What else can the collector do
when it visits all objects?McKinley, UT 395T
MM
Key Issues
• For both– Fast allocation– Fast reclamation– Low fragmentation (wasted space)– How to organize the memory space
• Garbage Collection– Discriminating live objects and
garbage
12McKinley, UT 395T MM
13
What is Garbage?
McKinley, UT 395T MM
Perfect live object detection
• Live object has a future use• Prove that object is not live, and
deallocate it• Deallocate as soon as possible
after last use
14McKinley, UT 395T MM
Estimating liveness in practice
• Approximate liveness by reachability from outside the heap
• An unreachable object cannot ever be used---it is garbage
• Once dead always dead!• Find and preserve reachable objects
– Tracing or reference counting • Recycle the space of garbage
objects
15McKinley, UT 395T MM
How does the GC implement
reachability?
McKinley, UT 395T MM
16
How does the GC implement
reachability?• Tracing• Counting
McKinley, UT 395T MM
17
18
How does the GC find the pointers to trace or
count? • Managed languages couple GC
with safe pointers– Programs may not access arbitrary
addresses in memory– Compiler can identify and provide the
GC with all the pointers….enforcing:•“Once garbage, always garbage”
– Runtime system can move objects by updating pointers
– Unsafe languages can do non-moving GC by assuming anything that looks like a pointer is one.
McKinley, UT 395T MM
19
Reachability with tracing
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
• Compiler produces a stack-map at GC safe-points and Type Information Blocks
• GC safe points: new(), method entry, method exit, & back-edges (thread switch points)
• Stack-map: enumerate global variables, stack variables, live registers -- This code is hard to get right! Why?
• Type Information Blocks: identify reference fields in objects
McKinley, UT 395T MM
20
Reachability with tracing
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
• Compiler produces a stack-map at GC safe-points and Type Information Blocks
• Type Information Blocks: identify reference fields in objects for each type i (class) in the program, a map
30 2TIBi
McKinley, UT 395T MM
21
Reachability with tracing
• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and
then performs a transitive closure over them
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
mark
McKinley, UT 395T MM
22
Reachability with tracing
• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and
then performs a transitive closure over them
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
mark
McKinley, UT 395T MM
23
Reachability with tracing
• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and
then performs a transitive closure over them
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
mark
McKinley, UT 395T MM
24
Reachability with tracing
• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and
then performs a transitive closure over them• All unmarked objects are dead, and can be
reclaimed
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
mark
McKinley, UT 395T MM
25
Reachability with tracing
• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and
then performs a transitive closure over them• All unmarked objects are dead, and can be
reclaimed
stackglobals registersheap
ABC{ ....
r0 = objPC -> p.f = obj ....
sweep
McKinley, UT 395T MM
Taxonomy of GC design choices
• Heap Organization• Incrementality• Composability• Concurrency• Parallelism• Distribution
26McKinley, UT 395T MM
Heap organization & basic algorithmic components
Allocation ReclamationIdentification
Bump Allocation
Free List
`
Tracing(implicit)
Reference Counting(explicit)
Sweep-to-Free
Compact
Evacuate
3 1
Sweep-to-Region
McKinley, UT 395T MM
27
28
One Big Heap?Incrementality
Pause times– it takes too long to trace the whole heap at once
Throughput– the heap contains lots of long lived objects, why collect
them over and over again?Incremental collection
– divide up the heap into increments and collect one at a time.
Increment 1 Increment 2
to space from space to space from space
McKinley, UT 395T MM
29
Incremental Collection
Ideally• perfect pointer knowledge of live pointers between
increments• requires scanning whole heap, defeats the purpose
to space from space to space from space
Increment 1 Increment 2McKinley, UT 395T
MM
30
Incremental Collection
Ideally• perfect pointer knowledge of live pointers between
increments• requires scanning whole heap, defeats the purpose
to space from space to space from space
Increment 1 Increment 2McKinley, UT 395T
MM
31
Incremental Collection
to space from space to space from space
Increment 1 Increment 2
Ideally• perfect pointer knowledge of live pointers between
increments• requires scanning whole heap, defeats the purpose
McKinley, UT 395T MM
32
Incremental Collection
Ideally• perfect pointer knowledge of live pointers between
increments• requires scanning whole heap, defeats the purposeMechanism: Write barrier• records pointers between increments when the mutator
installs them, conservative approximation of reachability
to space from space to space from space
Increment 1 Increment 2McKinley, UT 395T
MM
33
Write barrier
compiler inserts code that records pointers between increments when the mutator installs them
// original program // compiler support for incremental collectionp.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) U p.f; } p.f = o;
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
34
Write barrier
Install new pointer d -> v
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) U p.f; } p.f = o;
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
35
Write barrier
Install new pointer d -> v, then update d-> y
// original program // compiler support for incremental collectionp.f = o; if (incr(p) != incr(o) {
remembered set (incr(o)) = p.f; } p.f = o;
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g,d}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
36
Write barrier
Install new pointer d -> v, then update d-> y
// original program // compiler support for incremental collection
p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) =
p.f; } p.f = o;
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g,d,d}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
37
Write barrier
At collection time• collector re-examines all entries in the remset
for the increment, treating them like roots• Collect Increment 2
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g,d,d}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
38
Write barrier
At collection time• collector re-examines all entries in the remset
for the increment, treating them like roots• Collect Increment 2
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g,d,d}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
39
Summary of the costs of
incremental collection• write barrier to catch pointer stores crossing
boundaries• remsets to store crossing pointers• processing remembered sets at collection time• excess retention
to space from space to space from space
Increment 1 Increment 2
remset1 ={w} remset2 ={f,g,d,d}
a b c d e f g t u v w x y z
McKinley, UT 395T MM
Taxonomy of Design Choices
• Incrementality• Composability• Concurrency• Parallelism• Distribution
40McKinley, UT 395T MM
41
GC Ideas
• Generational collection - Young objects die fast
• Older first - The longer you wait• Garbage first• Immix• Reference counting - Deferred & Ulterior
• Concurrent collection [Steele, Djkistra et al.’78]– at the same time as the mutator
• Parallel collection• Real time
– make pause times tiny [Metronome, Petrank et al.]• What else can you do when you visit all the objects?• Memory utilization & fragmentation• Leaks
McKinley, UT 395T MM
Questions?