Joint work with Mike Barnett, Robert DeLine, Manuel Fahndrich, and Wolfram Schulte Verifying...

Joint work with Mike Barnett, Robert DeLine, Manuel Fahndrich, and Wolfram Schulte

Verifying invariants inVerifying invariants in object-oriented programs object-oriented programs Verifying invariants inVerifying invariants in object-oriented programs object-oriented programs

K. Rustan M. Leino Microsoft Research, Redmond, WA, USA

Computer Science ColloquiumETH Zurich24 Nov 2003

‥

Vision

Record design decisions+ Utilize automatic checking

= Detect errors and improve maintainability

A# and Boogie:Programmer's view

Boogie

C# compiler

A# compiler

A#

C#

MSIL

Warnings

Other .NETcompilers

Boogie: under the hood

Theoremprover

weakest-preconditiongenerator

translator

MSIL

BoogiePL

verification condition

Warnings

Inferenceengine

Boogie technologyand research

• Programming methodology– model for writing code and specifications

• Inference– abstract domains over heap structures

• Verification-condition generation– formulas that are “efficient” for theorem

prover

Invariants: straw man

class T {// field declarations ...

invariant J ;

T(...)requires Pmodifies vensures Q

{…

}

method m(...)requires Rmodifies wensures T

{…

}

…

class T {// field declarations ...

T(...)requires Pmodifies vensures Q ∧ J

{…

}

method m(...)requires R ∧ J

modifies wensures T ∧ J

{…

}

…

Object invariants hold on public method boundaries and are shorthands for pre/post-conditions

Invariants, exampleclass T {

private int x, y ;invariant 0 ≦ x < y ;

public T(){

x = 0 ; y = 1 ;}

public method m()modifies x, y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

class T {

private int x, y ;

public T()ensures 0 ≦ x < y

{x = 0 ; y = 1 ;

}

public method m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

Invariants, problemsclass T {

private int x, y ;invariant 0 ≦ x < y ;

public T(){

x = 0 ; y = 1 ;}

public method m()modifies x, y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

class T {

private int x, y ;

public T()ensures 0 ≦ x < y

{x = 0 ; y = 1 ;

}

public method m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

callers are expected to establish property about internal data structure

Invariants, problemsclass T {

private int x, y ;invariant 0 ≦ x < y ;

public T(){

x = 0 ; y = 1 ;}

public method m()modifies x, y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

class T {

private int x, y ;

public T()ensures 0 ≦ x < y

{x = 0 ; y = 1 ;

}

public method m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

possible solution (?): callers don’t need to be checked for this precondition—it holds automatically!

Invariants, problemsclass T {

private int x, y ;invariant 0 ≦ x < y ;

public T(){

x = 0 ; y = 1 ;}

public method m()modifies x, y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

class T {

private int x, y ;

public T()ensures 0 ≦ x < y

{x = 0 ; y = 1 ;

}

public method m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y

{assert y-x ≧ 0 ;x = x + 3 ;

y = 4 * y ;}

…

invariant does not hold here

Invariants, problemsclass T {

private int x, y ;invariant 0 ≦ x < y ;

public T(){

x = 0 ; y = 1 ;}

public method m()modifies x, y

{assert y-x ≧ 0 ;x = x + 3 ;y = 4 * y ;

}

…

class T {

private int x, y ;

public T()ensures 0 ≦ x < y

{x = 0 ; y = 1 ;

}

public method m()requires 0 ≦ x < y modifies x, y ensures 0 ≦ x < y

{assert y-x ≧ 0 ;x = x + 3 ;p(...) ;y = 4 * y ;

}

…

invariant does not hold here, so what if p calls m?!

The problem of specifying modifications

class Kitchen {private Dishes d ;private bool hasGarbage ;private Stove s ;private Light l ; ...public method SpiffUp()

modifies hasGarbage, s.foodPieces, s.big.sufaceColor,

…{

d.Clean() ;hasGarbage = false ;s.Wipe() ;l.TurnOff() ;...

}

class Stove {private Burner big ;private Burner small ;private int foodPieces ;private Knob[] knobs ;...public method Wipe()

modifies foodPieces, big.surfaceColor, …

{big.Polish() ;small.Polish() ;foodPieces = 0 ;...

}these lists are long, and they mention private state

The problem of specifying modifications

class Kitchen {private Dishes d ;private bool hasGarbage ;private Stove s ;private Light l ; ...public method SpiffUp()

modifies hasGarbage, s.foodPieces, s.big.sufaceColor,

…{

d.Clean() ;hasGarbage = false ;s.Wipe() ;l.TurnOff() ;...

}

class Stove {private Burner big ;private Burner small ;private int foodPieces ;private Knob[] knobs ;...public method Wipe()

modifies foodPieces, big.surfaceColor, …

{big.Polish() ;small.Polish() ;foodPieces = 0 ;...

}possible solution (?): don’t need to declare modifications of private state—it can’t be observed anyhow

The problem of specifying modifications

class Kitchen {private Dishes d ;private bool hasGarbage ;private Stove s ;private Light l ; ...public method SpiffUp()

modifies

{d.Clean() ;hasGarbage = false ;s.Wipe() ;assert ￢ hasGarbage ;l.TurnOff() ;...

}

class Stove {private Burner big ;private Burner small ;private int foodPieces ;private Knob[] knobs ;...public method Wipe()

modifies

{big.Polish() ;small.Polish() ;foodPieces = 0 ;...

}

SpiffUp treats Wipe as if Wipe modified nothing, so what if Wipe calls a method in Kitchen that sets hasGarbage to true?!

Soundness of verification• Soundness = verification finds all

errors• Soundness follows from:

– pre- and postconditions are the same for caller and callee

• Note: In addition to soundness, we want something usable

Methodology• object invariant declaration

– class T {int x, y ;invariant x < y ;

• special variable st: {Invalid, Valid}• Idea: program invariant

(∀o ･ o.st = Invalid ∨ Inv(o))• st is changed by commands pack and

unpack

holds at everyprogram point!

for any o: T, we writeInv(o) ≡ o.x < o.y

pack and unpack• pack o ≡

assert o.st = Invalid ;assert Inv(o) ;o.st := Valid

• unpack o ≡ assert o.st = Valid ;o.st := Invalid

Exampleclass T {

int x, y ;invariant 0 ≦ x < y ;method init(t)

requires t.st = Invalidmodifies t.st, t.x, t.yensures t.st = Valid

{t.x := 0 ; t.y := 1 ;pack t

}method m(t)

requires t.st = Validmodifies t.x, t.y

{unpack t ;t.x := t.x + 3 ; t.y := 4 * t.y ;pack t

}

receiver parameter(“this”, “self”, “current”)

Program invariant(∀o ･ o.st = Invalid ∨ Inv(o))

• x := new(T) ≡... ; assume x.st = Invalid

• pack o ≡... ; assert Inv(o) ; o.st := Valid

• unpack o ≡ ... ; o.st := Invalid

• o.f := E ≡assert o.st = Invalid ; ...

• Inv(o) can mention only the fields of o

Methodology, summary• invariant ...• st: {Invalid, Valid}• pack, unpack• modifications of o.f require

o.st=Invalid• Inv(o) can mention only the fields of o• (∀o ･ o.st = Invalid ∨ Inv(o))

Methodology, extended• component declarations

– class Kitchen {component Stove s ;

• st: {Invalid, Valid, Committed}• Idea: program invariant

(∀o ･ o.st=Invalid ∨ (Inv(o) ∧ (∀p∈Comp(o) ･ p.st=Committed)))

• pack o and unpack o change st for o and o's components

for any k: Kitchen, we havek.s ∈ Comp(k)

pack and unpack, extended• pack o ≡

assert o.st = Invalid ∧ Inv(o) ;assert (∀p ∈ Comp(o) ･ p.st=Valid) ;o.st := Valid ;foreach p ∈ Comp(o) do

p.st :=Committed ;

• unpack o ≡ assert o.st = Valid ;foreach p ∈ Comp(o) do p.st=Valid ;o.st := Invalid ;

Exampleclass Kitchen {

method SpiffUp(k)requires k.st=Valid …

{unpack k ;k.d.Clean() ;k.s.Wipe() ;pack k

}

class Stove {method Wipe(s) requires s.st=Valid …

sd

Kitchenk

Stove

Dishes

Valid

Committed

Committed

Committed

Committed

Committed

ValidInvalid

Exampleclass Kitchen {

method SpiffUp(k)requires k.st=Valid …

{unpack k ;k.d.Clean() ;k.s.Wipe() ;pack k

}

class Stove {method Wipe(s) requires s.st=Valid …

sd

Kitchenk

Stove

Dishes

Committed

Committed

Committed

Committed

Committed

Valid

Valid

Program invariant

(∀o ･ o.st=Invalid ∨ (Inv(o) ∧ (∀p∈Comp(o) ･

p.st=Committed)))

• x := new(T) ≡ ... ; assume x.st=Invalid• pack o, unpack o• o.f := E ≡ assert o.st=Invalid ; ...• Inv(o) can mention only the fields of o and

of o.p for any component field p

Extended methodology, summary• invariant ...• component ...• st: {Invalid, Valid, Committed}• pack, unpack• modifications of o.f require o.st=Invalid• Inv(o) can mention only the fields of o and

of o.p for any component field p• (∀o ･ o.st=Invalid ∨

(Inv(o) ∧ (∀p∈Comp(o) ･p.st=Committed)))

Verification system• We let st be used in method

specifications (requires, modifies, ensures)

• We must address the Problem of Specifying Modifications

A heap model

• Heap is a two-dimensional “array”

class T { f: U; g: V; ... }• x := o.f = x := Heap[o, f]• o.f := E = Heap[o, f] := E

Meaning of modifies

• modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] =

Heap0[o,f] ∨ (o,f) ∈ w0 )

viewed as set of object/field-name pairs

Meaning of modifies

• modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] =

Heap0[o,f]∨ (o,f) ∈ w0 ∨ ￢ Heap0[o,alloc] )

Meaning of modifies

• modifies w =modifies Heapensures (∀o,f ･ Heap[o,f] = Heap0[o,f]

∨ (o,f) ∈ w0 ∨ ￢ Heap0[o,alloc] ∨

Heap0[o,st]=Committed )

Example

class Kitchen {method SpiffUp(k)

requires k.st=Validmodifies k.hasGarbage

{unpack k ;k.d.Clean() ;k.hasGarbage := false ;k.s.Wipe() ;assert ￢ k.hasGarbage ;pack k

}

class Stove {method Wipe(s) requires s.st=Valid modifies s.foodPieces

Another example

method m(p)requires p.st=Committed

{var y, z in

y := p.x ;z := sqrt(49) ;assert y = p.x

end}

Soundness• Pre- and postconditions are the

same for callers and callees, so verification system is sound!

Related work• rep types in CLU• dynamically checked invariants in Eiffel• valid idiom in ESC/Modula-3• universe types and invariants in Muller's

thesis• invariant declarations in ESC/Java and JML• (implicit) pack/unpack operations in Vault and

Fugue• capability calculus• ownership types• locking and monitor disciplines in concurrent

programming

‥

Conclusions• Invariants different from pre/post-conditions• Resulting program invariants hold at every

program point• Uses pack/unpack commands, but good

defaults can be constructed for these• No linear type system• Components are not unique references—

objects (pointers) can freely be copied• Fields can freely be read• No additional features of abstraction needed

to support the specification of modifications• Sound

‥

Further research challenges• experience, understanding of

limits• extensions to support more good

programs(joint work with Peter Muller)

‥