Finite State Verification: An Emerging Technology for Validating Software Systems

Laboratory for Advanced Software Engineering ResearchLaboratory for Advanced Software Engineering ResearchUMASSUMASS

Lori A. ClarkeUniversity of Massachusetts

[email protected]://laser.cs.umass.edu/

Finite State Verification:

An Emerging Technology for Validating

Software Systems


Outline of Presentation

• Lay of the Land:– Testing, Theorem-proving based verification, Finite

state verification(FSV)

• Overview of FSV

• Look at 3 Different Approaches to FSV– Model Checking– Flow Equations– Data Flow Analysis

• Major Challenges to be Addressed


Sorry State of Affairs

• Testing consumes about half the cost of s/w development

• Maintenance consumes about 80% of the full life cycle costs--much of that devoted to testing

• Most companies use ad hoc QA practices• Unhappy with the results; Unhappy with the cost

– Failed projects

– Delayed product releases


Testing

• can:– Uncover failures– Show specifications are (not) met for specific

test cases– Be an indication of overall reliability

• cannot:– Prove that a program will/will not behave in a

particular way


Must do better!

• Increasing number of high assurance applications– Medical applications– Flight control software– Electronic commerce

• Increasing number of complex systems– Systems of systems– Distributed systems


Distributed SystemsDistributed Systems

• Better performance, better flexibility,

but there is a cost• distributed systems are more difficult

to test than sequential systems– number of execution paths can grow exponentially

with the number of processes– Testing can not even demonstrate that a system

works on the selected/executed test data


3

4

2

9

T1 T2

5

8

1 6

1,6

2,6

5,9

3,6

4

1,7

2,7 1,8

3,7 2,8

3,8

7

Complexity of Distributed Systems


3

4

2

9

T1 T2

5

8

1 6

1,6

2,6

5,9

3,6

4

1,7

2,7 1,8

3,7 2,8

3,8

7

Uncertainty of Testing

X:=1

X: =2

X==?


Formal Verification: An Alternative to Testing

• Theorem Proving Based Verification– Use mathematical reasoning– Prove properties about all possible executions – Difficult and error prone

• Finite State Verification– Reason about a finite model of the system– Prove properties about all possible executions, but not

as powerful as theorem proving– Almost a totally automated process


Spectrum of Difficulty

Ad-hoc Testing

Systematic Testing

Theorem Proving

Finite State Verification

•Arbitrary testcases

•Reqts based test planning

•Requirements captured as properties

•Properties guaranteed on all possible executions


Finite State Verification (FSV)

• Holds the promise of providing a cost effective way of verifying important properties about a system– Not all faults are created equal

– Invest effort into most important properties

• Several promising prototypes– Reachability Based

• SPIN or Symbolic Model Checking (SMV)

– Flow Equations• Integer Necessary Conditions (INCA)

– Data Flow Analysis• FLAVERS


Property

System

Property Translator

SystemTranslator

ReasoningEngine

System ModelProperty Verified

Property Representation

High-Level Architecture of High-Level Architecture of FSV SystemsFSV Systems

Counter Examples for Model


Conservative Analysis • If property verified, property holds for all possible

executions of the system• If property not verified:

– An error OR

– A spurious result• System model abstracts information to be tractable

• Conservative abstractions over-approximate behavior

• If inconsistency relies upon over-approximations, then a spurious result

– e.g. counter example corresponds to an infeasible path


System Model

• Depends on property being verified

• Eliminate information that does not impact the proof

• Abstraction techniques allows “states” in the model to be reduced/collapsed


Some Properties of Properties

• State-based versus event-based• Once temperature is greater than 100 degrees,

lock is true

• Elevator door closes before elevator moves

• Single locations versus (sub)paths – Deadlock or race conditions– Sequences of states or events

• Safety versus Liveness


A quick look at three approaches to FSV

• Model Checking

• Flow Equations

• Data Flow Analysis

Big Disclaimer!


Model Checking: some history

• Originally proposed for hardware • Early 80’s: E. Clarke and Emerson;

Quielle and Sifakis• Late 80’s: Improved algorithms and property

notations (E. Clarke, Emerson, Sistla)• 90’s: Symbolic Model Checking (SMV) and

other optimizations (Burch, E. Clarke, Dill, Long, and McMillan)

• Current: Hybrid approaches


Model Checking

• Properties usually expressed in a temporal logic

• System represented as a (possibly “abstracted”) reachability graph– State based

• Reasoning engine propagates valid subformulas through the graph


Temporal Logic Property

System

Property Translator

SystemTranslator

Subformula propagation

State-based Reachability

GraphProperty Verified

Property Representation

High-Level Architecture of High-Level Architecture of Model CheckingModel Checking



Representing Properties

• CTL operators– G - globally– F - future– X- next– U - until

• At a state in the model: – AG p means that for all paths from this state, p is

true and will remain true

– EF p means that for some path from this state, p will eventually be true


Propagating Propositions

p AF p

AF p

AF p

AF p


Example: mutual exclusion protocol*

reachability graph n1,n2,turn=0

t1,n2,turn=1

c1,n2,turn=1

t1,t2,turn=1

c1,t2,turn=1

n1,t2,turn=2

n1,c2,turn=2

t1,t2,turn=2

t1,c2,turn=2

*McMillan


Example Property

• AG(t1=>AF c1)

• If process1 tries (t1) to get the lock then eventually it gets into its critical region (c1)

• Note, would like to prove this for all processes but FSV approaches usually must instantiate property (and system)


Example: propagation

n1,n2,turn=0

t1,n2,turn=1

c1,n2,turn=1

t1,t2,turn=1

c1,t2,turn=1

n1,t2,turn=2

n1,c2,turn=2

t1,t2,turn=2

t1,c2,turn=2

AF c1

AG(t1=>AF c1)

AF c1

AF c1

AF c1

AF c1

t1=>

t1=>

t1=>

t1=>

AF c1

AF c1

AF c1

AF c1


Formula Propagation

• Propagate until no change– propagate from smaller to larger subformulas

– “smart” algorithm: linear in the size of model and size of the formula

• Many optimization techniques– Symbolic model checking– Use efficient algorithms that propagate subformula for sets

of values


Symbolic Model Checking

• With abstraction, nodes may represent sets of values– BDD

– Worst case bound exponential in size of the model– For some examples, able to deal with 10120 states

a

b b

c

0

0 0

1

1 1

ab+c

0 1

ab

0

0

1

1

c0 1

0 1

c0 1

0 1

c0 1

0 1

c0 1

0 1 1

1 1


Some observations: Model Checking

• Worst case bound linear in size of the model– Model exponential

• Experimentally often very effective

• Not clear if model checking or symbolic model checking is superior– Depends on the problem


Flow Equations: some history

• Originally proposed for designs

• Early 80’s: Initial development (Avrunin, Dillon, and Wileden)

• 90’s: Optimized and extended to real-time (Avrunin, Buy, Corbett, Dillon, and Wileden)

• Current: INCA prototype (Avrunin, Corbett, and Siegel)


Flow Equations

• Model system as finite state automata

• Use extended network flow inequalities to capture legal flow through a concurrent system

• Represent negation of the property as a set of inequalities


Solving the Set of Inequalities

• Determine if combined system of inequalities is consistent– Use integer linear programming

• If consistent, there is a set of flows through automata that violate the property

• Provides guidance for trace through the model (but may not be executable)


PropertySystem

Property Translator

FSATranslator

Integer Linear Programming System

Set of Inequalities Property Verified

(no solution)

Set ofInequalities

High-Level Architecture of High-Level Architecture of INCAINCA

Counter Examples for Model (solution)

SystemTranslator

FSA’s


Example: Process Flow Equations

x1 = x0 + x2x1 = x2 + x3x0 = 1; x3 = 1

x9 = x8 + x10x9 = x10 + x11x8 = 1; x11 = 1

x5 + x7 = x4 + x6 x5 = x6x4 = 1; x7 = 1

x1x2

x4

x5x6 x9x10

x0 x8

x3

x7

x11

a a’b’ b


Example: Inter-process Flow Equations

x1x2

x4

x5x6 x9x10

x0 x8

x3

x7

x11

a a’b’ b

x1 = x5

x9 = x6


Solving for a property

Property: For all paths, event a occurs more than event b

represent complement ¬(x1 > x9) = = (x1 ≤ x9)

x1 = x0 + x2x1 = x2 + x3x0 = 1; x3 = 1x5 + x7 = x4 + x6 x5 = x6x4 = 1; x7 = 1x9 = x8 + x10x9 = x10 + x11x8 = 1; x11=1x1 = x5x9 =x6j: 0 ≤ xj

Solution exists e.g., x2, x10 = 0, all other xi = 1 => property does not hold


Seeing the counter example

Property: For all paths, event a occurs more than event b

x1x2

x4

x5x6 x9x10

x0 x8

x3

x7

x11

a a’b’ b

x2, x10 = 0, all other xi = 1


Some Limitations

• Integer Linear Programming has an exponential worst case bound

• Inter-process order information is not preserved– only checks whether event counts are

consistent– Like most static techniques, may produce

spurious results


Some Benefits

• Does not enumerate the state space!

• Integer linear Programming is often very efficient– Empirical evidence: linear inequality systems

usually grow linearly and take sub-exponential times to solve

• In practice, INCA is usually an effective technique


Data Flow Based Verification: some history

• Mid-70’s: Originally proposed for def-ref anomalies in FORTRAN (Osterweil and Fosdick)

• Early 80’s: Extended to general properties (Olender and Osterweil) & concurrency (Taylor and Osterweil)

• 90’s: Deadlock detection (Masticola and Ryder); Efficient representation of concurrency & incremental precision improvement (Dwyer and L. Clarke)

• Recent: Optimizations, Java (Avrunin, L. Clarke, Cobleigh, Naumovich, and Osterweil)


Data Flow Analysis: FLAVERS

• Represents property as a finite state automaton

• System model is collection of annotated control flow graphs– Inter-process communication and interleavings are

represented with additional edges– does not enumerate all reachable states– over-approximates relevant executable behaviors

• Reasoning engine based on data flow analysis


Property

System

Property Translator

SystemTranslator

State Propagation

Collection of annotated CFG’s

Property Verified

FSA

High-Level Architecture of High-Level Architecture of FSV SystemsFSV Systems

Counter Examples from Model


Modeling the System

3

4

2

9

T1 T2

5

7

1,6

2,6

5,9

3,6

4

8

1,71 6

2,7 1,8

3,7 2,8

3,8

•State explosion


Modeling the System

3

4

2

9

T1 T2

5

7

8

1 6 •Automatically creates the program model from source code

•Instead of the state space, explicitly represents interleaved execution via edges

•Smaller model

•Loss of precision


Representing PropertiesRepresenting Properties

Example:

close,open,move

0

1

openclose

2

move

closemove

open


State Propagation

• States of the property are propagated through the model

• The property is proved if only accepting (non-accepting) states are contained in the final node of the model


Example

public static void main (String [] args){ … if (elevatorStopped) {... openDoors(); } recordState(); if (elevatorStopped) {... closeDoors(); } moveToNextFloor();}

if

open

close

if

move


Example

if

open

close

if

move close,open,move

0

1

openclose

2

move

closemove

open

{0}

{1}

{0,1}

{0}

{0,2}


Property

System

Property Translator

SystemTranslator

State Propagation

System model

Property Verified

...

Constraints

FSA

Incrementally Improving Incrementally Improving PrecisionPrecision



Example with Constraints

if

open

close

if

move

S==true S==false

S==falseS==true

0

21

viol

S==true S==false

S==true

S==true

S==false

S==false

S==falseS==true

Constraint

close,open,move

0

1

openclose

2

move

closemove

open

Property(0,0)

(0,1)

(1,1) (1,1)

(1,viol)


Example with Constraints

if

open

close

if

move

S==true S==false

S==falseS==true

0

21

viol

S==true S==false

S==true

S==true

S==false

S==false

S==falseS==true

Constraint

close,open,move

0

1

openclose

2

move

closemove

open

Property(0,0)

(0,1)

(1,1){(1,1), (0,2)}

(0,2)

{(1,1), (0,viol)} {(1,viol), (0,2)}

{(0,1)}

{(0,1), (0,2)}


Some Observations: Data Flow Analysis• Overall complexity is O(N2S)

– N is the # nodes in the model – S is the number of states: property x constraints– Experimentally: performance subexponential

• Usually requires several iterations to determine needed constraints

• Constraints– Many automatically generated on request– Can be used to model other information


Experimental Comparisons

• All these approaches are:– very effective on some problems– disappointing on some problems

• Hard to predict how they will perform

• Experimental results– George S. Avrunin, James C. Corbett, Matthew B. Dwyer, Corina S.

Pasareanu, and Stephen F. Siegel, Comparing Finite-State Verification Techniques for Concurrent Software

Very Big Disclaimer!


url: laser.cs.umass.edu


Can we move beyond academic prototypes to practitioners’ tools?

• Yes, but there is more work to be done– Optimization, optimization, optimization– Process support– Better support for specifying properties– Better support for generating, selecting, visualizing

counter example traces – Better approaches for dealing with dynamism– Full support for real languages– Full lifecycle support

• Integration with testing


Specifying Properties

• It is very hard to specify properties precisely– E.g., open and close file repeatedly

• Must file always be opened?Or, IF it is opened, then it must be closed?

• Can file be opened repeatedly before it is closed?

• Need notations that are easy to use– Specification patterns

• Need tools to help understand properties– need to test the properties


Counter Example Traces

• Want “short” but “useful” counter examples

• How to select the “next” counter example?

• How to incorporate user guidance?

• How to go from traces in the model to traces in the program?


Dynamism

• FSV is a static analysis approach that deals with static models– Must create a specific instance of the model

• E.g., N philosophers => 5 philosphers

– Can not handle • dynamic objects

• dynamic process creation

• Need hybrid techniques that integrate theorem proving with FSV


Support for Real Languages

• Many language features have not been addressed– Aliasing – Exception handling– Event based notification


Lifecycle-based Verification

• High-level architectural design– Extremely important for distributed systems

• Detect problems early

– Need to support heterogeneous interaction models

• Low-level design– Additional detail leads to additional properties– Need to maintain consistency with the HLA


Lifecycle-based Verification (continued)

• Coding– Partial systems– Incremental, compositional

development/verification

• Debugging– Hypothesize fault in terms of a property– FSV provides a counter example trace or

invalidates hypothesis


• Testing– Generalize test cases to their corresponding

property– Test planning via requirements based property

specification

• Regression testing– re-verify properties that should not have changed

• Need efficient re-verification techniques

Lifecycle-based verification (continued)


Integrating Testing and Verification

• Testing and verification complement one another– verification makes assumptions that should

be monitored dynamically– testing finds problems that should then be

examined globally

• Need to develop integrated techniques


Synergy between Testing and Verification

Properties

Faults

Testing

Assumptions/constraints

Counter examples

Assertions

Verification

Test plans/cases


Conclusions

• Testing alone can not provide the assurance that is needed for many applications– especially distributed systems

• FSV a promising technology– Applicable to a wide range of properties

– Applicable throughout the lifecycle

– Initial empirical results promising


ConclusionConclusion

• Finite State Verification is a major paradigm shift– More difficult than testing,

but not that much more difficult – Cultural resistance to doing anything different

• Is the pain worth the gain?

• Grand challenge: Can we lower the obstacles to adoption?

Documents

Finite State Verification: An Emerging Technology for Validating Software Systems