Memory Management

Memory Management

Kathryn McKinley

Isn’t GC a bit retro?

“Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.”

Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008

“Languages without automated garbage collection are getting out of fashion. The chance of running into all kinds of memory problems is gradually outweighing the performance penalty you have to pay for garbage collection.”

Paul Jansen, managing director of TIOBE Software, in Dr Dobbs, April 2008

Mark-CompactStyger, 1967

Mark-SweepMcCarthy, 1960

Semi-SpaceCheney, 1970

2McKinley, UT 395T

MM

Course Logistics

• Syllabus• Critical reading & writing• Presentations• Discussion• Critiques• Schedule• Volunteers for next week?

3McKinley, UT 395T MM

Outline

• Briefly introduce the challenges and key ideas in memory management– Context – modern VMs– Explicit vs automatic– Memory organization– Allocation– Garbage Identification– Reclamation


5

Basic VM Structure

Executing Program

Program/Bytecode

Interpreter &/or

Dynamic Compiler

Class Loader Verifier, etc. Heap

Thread Scheduler

Garbage Collector

McKinley, UT 395T MM

Dynamic memory allocation and reclamation

• Heap contains dynamically allocated objects

• Object allocation: malloc, new• Deallocation:

– Manual/explicit: free, delete– automatic: garbage collection


Memory Management

• Objects/data in heap memory• How does the runtime system efficiently create and recycle memory on behalf of the program?– What makes this problem important?– What makes this problem hard?– Why are researchers still working on

it?


Explicit memory managementchallenges

• More code to maintain• Correctness

– Free an object too soon - core dump– Free an object too late - waste space– Never free - at best waste, at worst

fail• Efficiency can be very high• Gives programmers “control”


Garbage collection:Automatic memory

management

• Reduces programmer burden• Eliminates sources of errors

– which ones?• Integral to modern object-oriented

languages – Java, C#, PHP, JavaScript

• Mainstream• Challenge: performance


10

Why use Garbage Collection?

• Software engineering benefits– Less user code compared to expilict

memory management (MM)– Less user code to get correct– Protects against some classes of memory

errors• No free(), thus no premature free(), no double

free(), or forgetting to free()

• Not perfect, memory can still leak– Programmers still need to eliminate all

pointers to objects the program no longer needs


11

Why use Garbage Collection?

• Performance: space/time tradeoff– Time proportional to dead objects

(explicit mm, reference counting) or live objects (semi-space, mark-sweep)

– Throughput versus pause time• Less frequent collection typically reduces

total time but increases space requirements and pause times

– Hidden locality benefits?• Layer of abstraction• What else can the collector do

when it visits all objects?McKinley, UT 395T

MM

Key Issues

• For both– Fast allocation– Fast reclamation– Low fragmentation (wasted space)– How to organize the memory space

• Garbage Collection– Discriminating live objects and

garbage


13

What is Garbage?


Perfect live object detection

• Live object has a future use• Prove that object is not live, and

deallocate it• Deallocate as soon as possible

after last use


Estimating liveness in practice

• Approximate liveness by reachability from outside the heap

• An unreachable object cannot ever be used---it is garbage

• Once dead always dead!• Find and preserve reachable objects

– Tracing or reference counting • Recycle the space of garbage

objects


How does the GC implement

reachability?


16

How does the GC implement

reachability?• Tracing• Counting


17

18

How does the GC find the pointers to trace or

count? • Managed languages couple GC

with safe pointers– Programs may not access arbitrary

addresses in memory– Compiler can identify and provide the

GC with all the pointers….enforcing:•“Once garbage, always garbage”

– Runtime system can move objects by updating pointers

– Unsafe languages can do non-moving GC by assuming anything that looks like a pointer is one.


19

Reachability with tracing

stackglobals registersheap

ABC{ ....

r0 = objPC -> p.f = obj ....

• Compiler produces a stack-map at GC safe-points and Type Information Blocks

• GC safe points: new(), method entry, method exit, & back-edges (thread switch points)

• Stack-map: enumerate global variables, stack variables, live registers -- This code is hard to get right! Why?

• Type Information Blocks: identify reference fields in objects


20



ABC{ ....

r0 = objPC -> p.f = obj ....

• Compiler produces a stack-map at GC safe-points and Type Information Blocks

• Type Information Blocks: identify reference fields in objects for each type i (class) in the program, a map

30 2TIBi


21


• Tracing collector (semispace, marksweep)– Marks the objects reachable from the roots live, and

then performs a transitive closure over them


ABC{ ....

r0 = objPC -> p.f = obj ....

mark


22





ABC{ ....

r0 = objPC -> p.f = obj ....

mark


23





ABC{ ....

r0 = objPC -> p.f = obj ....

mark


24



then performs a transitive closure over them• All unmarked objects are dead, and can be

reclaimed


ABC{ ....

r0 = objPC -> p.f = obj ....

mark


25



then performs a transitive closure over them• All unmarked objects are dead, and can be

reclaimed


ABC{ ....

r0 = objPC -> p.f = obj ....

sweep


Taxonomy of GC design choices

• Heap Organization• Incrementality• Composability• Concurrency• Parallelism• Distribution


Heap organization & basic algorithmic components

Allocation ReclamationIdentification

Bump Allocation

Free List

`

Tracing(implicit)

Reference Counting(explicit)

Sweep-to-Free

Compact

Evacuate

3 1

Sweep-to-Region


27

28

One Big Heap?Incrementality

Pause times– it takes too long to trace the whole heap at once

Throughput– the heap contains lots of long lived objects, why collect

them over and over again?Incremental collection

– divide up the heap into increments and collect one at a time.

Increment 1 Increment 2

to space from space to space from space


29

Incremental Collection

Ideally• perfect pointer knowledge of live pointers between

increments• requires scanning whole heap, defeats the purpose


Increment 1 Increment 2McKinley, UT 395T

MM

30






MM

31







32



increments• requires scanning whole heap, defeats the purposeMechanism: Write barrier• records pointers between increments when the mutator

installs them, conservative approximation of reachability



MM

33

Write barrier

compiler inserts code that records pointers between increments when the mutator installs them

// original program // compiler support for incremental collectionp.f = o; if (incr(p) != incr(o) {

remembered set (incr(o)) U p.f; } p.f = o;



remset1 ={w} remset2 ={f,g}

a b c d e f g t u v w x y z


34

Write barrier

Install new pointer d -> v

// original program // compiler support for incremental collection

p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) U p.f; } p.f = o;



remset1 ={w} remset2 ={f,g}



35

Write barrier

Install new pointer d -> v, then update d-> y

// original program // compiler support for incremental collectionp.f = o; if (incr(p) != incr(o) {

remembered set (incr(o)) = p.f; } p.f = o;



remset1 ={w} remset2 ={f,g,d}



36

Write barrier

Install new pointer d -> v, then update d-> y

// original program // compiler support for incremental collection

p.f = o; if (incr(p) != incr(o) { remembered set (incr(o)) =

p.f; } p.f = o;



remset1 ={w} remset2 ={f,g,d,d}



37

Write barrier

At collection time• collector re-examines all entries in the remset

for the increment, treating them like roots• Collect Increment 2






38

Write barrier

At collection time• collector re-examines all entries in the remset

for the increment, treating them like roots• Collect Increment 2






39

Summary of the costs of

incremental collection• write barrier to catch pointer stores crossing

boundaries• remsets to store crossing pointers• processing remembered sets at collection time• excess retention






Taxonomy of Design Choices

• Incrementality• Composability• Concurrency• Parallelism• Distribution


41

GC Ideas

• Generational collection - Young objects die fast

• Older first - The longer you wait• Garbage first• Immix• Reference counting - Deferred & Ulterior

• Concurrent collection [Steele, Djkistra et al.’78]– at the same time as the mutator

• Parallel collection• Real time

– make pause times tiny [Metronome, Petrank et al.]• What else can you do when you visit all the objects?• Memory utilization & fragmentation• Leaks


Questions?

Documents

Memory Management