Upload
uriah
View
55
Download
13
Embed Size (px)
DESCRIPTION
Christelle Scharff http://www.csis.pace.edu/~scharff May 2004. Formal Software Verification. DPS, Pace University. References. Use of the slides of Dr. Clark Barrett (with permission) ITR NSF Little Engines of Proof with Stanford Research Institute and Clarkson University, 2003 - PowerPoint PPT Presentation
Citation preview
Formal Software Verification
Christelle Scharff
http://www.csis.pace.edu/~scharffMay 2004
DPS, Pace University
References
Use of the slides of Dr. Clark Barrett (with permission) ITR NSF Little Engines of Proof with Stanford
Research Institute and Clarkson University, 2003 Automated Deduction with Constraints and
Simplification in Equational Theories, Christelle Scharff, PhD, 1999
ICS (I can Solve or Integrated Canonizer and Solver), Harald Ruess, 2002
Abstract Congruence Closure, Leo Bachmair, Ashish Tiwari, and Laurent Vigneron, 2003
Unsound Theorem Proving, Christopher Lynch, 2004 Direct Combination of Completion and Congruence
Closure, Christelle Scharff and Leo Bachmair, 2002 and 2004
On prototyping Deduction with Constraints and Simplification in a Rewriting Language, Christelle Scharff, 2004
Therac-25
Between 1985 and 1987, at least 6 accidental radiation overdoses were administered.
All the victims were injured, and 3 of them later died.
Ariane 5 Rocket
On June 4, 1996, an unmanned Ariane 5 rocket launched by the European Space Agency exploded just 40 seconds after its lift-off.
Value of rocket and cargo: $500 million
Blackout
In August, 2003, the largest blackout in our country’s history occurred.
Estimated cost to New York City alone: $1.1 billion.
What do these events have in common?
Caused by Software Bugs!
Each of the overdoses from the Therac-25 was the result of a bug in the controlling software.
The Ariane 5 explosion was the result of an unsafe floating point to integer conversion in the rocket’s software system.
A software bug caused an alarm system failure at FirstEnergy in Akron, Ohio. An early response to those alarms would likely have prevented the blackout.
More horror stories
Software Reliability: Principles and Practice, p 25, by G. J. Myers Appolo 8 spacecraft erased part of the
computer's memory. Eighteen errors were detected during the
10-day flight of Apollo 14. An error in a single FORTRAN statement
resulted in the loss of the first American probe to Venus.
Why must we suffer so? Engineers are supposed to be good at building
things.
When we build a building, we don’t expect it to crumble and have to be rebuilt twice a week.
Why don’t we have software engineers who build better software? It’s not all Microsoft’s fault. Software may be the most complex thing ever
created. Getting software right is a very, very difficult
task. Existing engineering techniques are inadequate.
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (Eclipse, ICS, GACC)
Conclusions and Future Work
What is Formal Verification? “[Formal] software verification … has been the Holy Grail
of computer science for many decades” – Bill Gates
Formal Verification (Proving) versus Validation (Testing)
Create a mathematical model of the system An inaccurate model can introduce or mask bugs. Fortunately, this can often be done automatically.
Specify formally what the properties of the system should be
Prove that the model has the desired properties Much better than any testing method Covers all possible cases This is the hard part
There are a variety of tools and techniques
Proof techniques Model Checking
Typically relies on low-level Boolean logic Proof is automatic Does not always scale to large systems
Theorem Proving Typically uses more expressive logic (higher
order logic) Proof is automatic in first order logic Proof is manually directed (in higher order
logic) Unlimited scalability
Advanced techniques combine elements of both
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (ICS, GACC, Eclipse)
Conclusions and Future Work
Formal Models
Typically, a formal model is a graph in which each vertex represents a state of the program, and each edge represents a transition from one state to another.
int x, y;x = 0;y = 0;while (x < 4) { x++; y = y + x;}
Consider this simple program:
The states of this program are all possible pairs of the variables x and y.
Fortunately, we can restrict our attention to the reachable states.
x ≥ y
Reachable States and Checking Properties
0,0
1,0
1,1
int x, y;x = 0;y = 0;while (x < 4) { x++; y = y + x;}
Initial State
2,1
2,3
3,3
3,6 Final State
x ≥ 0 0,0
1,0
x ≥ 0
We can check a property by verifying that it is true in every reachable state. If the property is false, then there is a bug.
2
1
xxy
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (Eclipse, ICS, GACC)
Conclusions and Future Work
Theorem Proving
Theorem proving relies on human ingenuity and symbolic manipulations to prove that a program satisfies some properties.
Typically, proving a single property about a program will require proving many other properties as well.
Theorem Prover = Software based on Inference Rules + Strategy Inference Rules: How to deduce new data?
How to remove data? Strategy: How to apply the inference rules?
Theorem Proving
What are the properties of theorem provers? Soundness (Mandatory Property)
Do not prove that True = False Prove only true formulae
Completeness (Optional Property) If a formula is true, it can be proved.
Termination (Undecidable Property) Does the application of the rules with the strategy terminate?
Theorem Proving
One approach is to annotate the program with theorems to be proved (e.g. pre-conditions, post-conditions, invariants, assertions), and then prove that each theorem really does hold. Hoare Logic
Consider a simple program:
Suppose we wish to prove that, if x = y before the execution of this program, then at the end of its execution, x = y. We can annotate the beginning and the end of the program with these properties.
[x=y]x = x + 1y = y + 1[x=y]
Theorem Proving
What is the condition that will guarantee that x = y after executing y = y + 1 ?
To find out, propagate the assertions upward through the program. We imagine trying to prove x = y using primed variables to represent the values after the execution of y = y + 1, and unprimed variables for the values before it:
(?) y’ = y + 1 x’ = x x’ = y’ (?) x = y + 1
[x=y]x = x + 1[x = y + 1]y = y + 1[x=y]
Theorem Proving
What is the condition that will guarantee that x = y + 1 after executing x = x + 1 ?
To find out, we imagine trying to prove x = y + 1 using primed variables to represent the values after the execution of x = x + 1, and unprimed variables for the values before it:
(?) x’ = x + 1 y’ = y x’ = y’ + 1 (?) x + 1 = y + 1
[x=y][x + 1 = y + 1]x = x + 1[x = y + 1]y = y + 1[x=y]
Theorem Proving
We prove that the program is correct by proving that: x = y implies x + 1 = y + 1 using a theorem prover.
[x=y][x + 1 = y + 1]x = x + 1[x = y + 1]y = y + 1[x=y]
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (Eclipse, ICS, GACC)
Conclusions and Future Work
Systems and Tools Model checkers
SMV SPIN
Theorem provers Little Engines of proof
Automated domain-specific theorem provers (uninterpreted functions, linear arithmetic, arrays, bitvectors...)
ICS, CVC lite GACC (Graph-based Abstract
Congruence Closure) Other Engines of proof
PVS, Isabelle, HOL BaCCS (Basic Completion with
Constraint and Simplification)
Microsoft SLAM Clever combination of model checking and
automated theorem proving An abstract program is created in which all
conditions are replaced with Boolean variables
Resulting Boolean program is model checked If model checking fails, the potential error
path is checked in the original program using an automated theorem prover
Successfully used to find bugs in Windows drivers. …reducing the frequency of “Blue Screens of
Death”!
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (Eclipse, ICS, GACC)
Conclusions and Future Work
Eclipse
Eclipse is an open source IDE developed by IBM
Eclipse is a “kind of universal tool platform – an open extensible IDE for anything and nothing in particular.”
Eclipse can be used to create applications as diverse as web sites, java programs, C++ programs...
An extension to Eclipse is called a plugin.
http://www.eclipse.org
Demo of ICS (I can solve or Integrated Canonizer and Solver)
Developed at SRI. http://www.ics.com
Decides formulae in a useful combination of theories (non interpreted formulae, bitvectors, reals, arrays…)
& means AND, | means OR, and ~ means NOT.
Example 1: Ics> sat x = 2 & x = 3; Answer: unsat
Example 2: Ics> sat [x = 2 | x = 3] & x < 3; Answer: sat, model [x = 2; 3 + - 1 * x > 0]
Demo of ICS (I can solve or Integrated Canonizer and Solver)
If we want to prove: p q, then we use refutational theorem proving. In fact, we assume that the hypotheses are true, and that the conclusion is false. Then, if p q is true, we will derive a contradiction (unsat). Otherwise, the resulting model provides us with a counter-example.
Example 3: If x = 2 and y = 4, then x + y = 6 ????
ics> sat x = 2 & y = 4 & ~ x + y = 6;Answer: unsat
If x = 2 or x = 5, then x < 4 ????ics> sat [x = 2 | x = 5] & ~ x < 4;Answer: sat, model [-4 + x >= 0; x = 5]
Demo of GACC (Graph-based Abstract Congruence Closure)
Congruence Closure is a method for deciding ground equalities. Example: From f(a) = b and a = c, we can
deduce f(c) =b.
The system GACC implements a new algorithm of congruence closure based on the use of graph with labeled vertices and edges.
GACC was implemented by Eugene Kipnis from Pace University under my supervision, and is available for online experimentation at: http://www.csis.pace.edu/~scharff/SOFTWARE
Outline
What is Formal Verification? Model Checking Theorem Proving Systems and Tools Demonstrations (Eclipse, ICS, GACC)
Conclusions and Future Work
Conclusions Formal Software Verification is starting to
become practical
Still lots of work to be done
How can it make you a better programmer? Document your code with the properties and
invariants that you think should be true When you modify code, convince yourself
that you are not breaking any invariants
Hopefully, someday software will be as safe and reliable as the other objects built by engineers!
Environment
CAFME Center create by Dr. Skevoulis
CS 851: Software Validation and Verification New course offered in the Fall 2004 Part of the degree in Software Design and
Development Validation = Testing Verification = Proving
ITR NSF Grant “Little Engines of Proof” with Stanford Research Institute, and Clarkson University
Future Work 1: Eclipse Plugin
A java program is annotated with assertions to be proved
Assertions are written in the ICS syntax
Some assertions are computed. Assertions are propagated up through the program line by line
ICS will be called from the java program, and the assertions will be checked
Parts of the program will be proved correct, or bugs will be discovered
Future Work 2: Strategies
Rewriting languages (e.g. ELAN, MAUDE, OBJ, ASF+SDF) are very expressive, and very convenient to prototype theorem provers
Rewriting languages share many features with functional programming languages (no side-effect, use of recursion, pattern matching…), but they are based on rewriting instead of lambda calculus
Can we develop a Java API that would be used to prototype theorem provers? Can we define the strategy on the rules in an XML document?
Future Work 3: Correctness of Abstract Congruence Closure
Abstract congruence closure is a framework that defines a set of inference rules and different strategies for congruence closure. These strategies include strategies that simulate Nelson-Oppen, Shostak and Downey-Sethi-Tarjan algorithms
The rules and the strategies can be implemented easily using a rewriting language such as ELAN
This implementation would be a formal framework to prove the correctness of the abstract congruence closure algorithm, and, besides, would permit intensive benchmarking on criteria to be defined
Future Work 4: Unsound Theorem Proving
Many problems given to a theorem prover will be false conjectures. It is important to be able to find quickly bugs in programs.
An inference system for unsound theorem proving (or theorem (dis)proving) has been defined by Christopher Lynch.
The theorem (dis)prover can be prototyped in a rewriting language such as ELAN.
This framework could be used to show the correctness of the procedure, and benchmark the method on criteria to be defined