Www.adacore.com A Pragmatic View of Formal Methods The Hi-Lite Project Robert B K Dewar SSS ‘11 President & CEO, AdaCore Emeritus Professor of Computer

www.adacore.com

A Pragmatic View of Formal MethodsThe Hi-Lite Project

Robert B K Dewar

SSS ‘11

President & CEO, AdaCoreEmeritus Professor of Computer Science, New York University

Feb 10th 2011

Slide: 2 Copyright © 2009 AdaCore

An observation from Yogi Berra

• In theory there is no difference between theory and practice. In practice there is.

• Variously attributed to Yogi Berra, Einstein etc.



Correctness and Reliability

• Not really the same thing

• Often merged in people’s thinking

• The issue of large reliable programs

• Correctness means adherence to some formal model

• Reliability means that a program works

• Many reasons why a correct program might be unreliable.

• And a reliable program may well be incorrect


We can do Pretty Well in Practice

• No one has ever died on a commercial airplane as a result of a software bug.

• We have however had some hair raising close calls

• But a close call is a close call

• When it comes to avionics, software is not the weak link in the chain.

• But we could do better

• And most certainly best practices are not widely applied

– Cars

– Medical Equipment

– Critical financial applications


Safety And Security

• Safety means we don’t have bugs that kill people

• Security means bad guys can’t cause chaos

• We have lots of bad guys around who know a lot

• Any safety critical system is security critical now

• How vulnerable are we

• TV thinks we are very vulnerable– Alias and Avionics systems

– 24 and Air Traffic Control

• Don’t be too quick to dismiss fiction– Clancy crashed a plane long before 9/11


Proving Entire Programs Correct

• A nice conceptual goal

• But definitely a case where theory and practice are not on the same page

• Fundamental problems stand in the way


Producing a Full Formal Specification

• No use unless we find an appropriate level of abstraction.

• Every program is a full formal specification in some sense, but that’s not useful

• Every generation of programming languages has been sold as high level specification eliminating the need for programmers

– Fortran, mathematicians will enter formulas directly

– COBOL, people will stick business models into the computer directly

– 4G languages, no, honestly, they really will stick them in directly

– Modeling languages, no more need for expensive programmers

• Expensive, but might be worth the cost if it was a panacea, but it is not.


Difficulties in Producing Full Specifications

• No one understands formal specifications

• There are things that can’t be formally specified– Good error messages in a compiler

– Intuitive user interfaces

• Deliberate incompleteness

• Difficulties of proving termination

• Difficulties of specifying programming languages

• Anyway, they will have errors

• A specification is basically a large program

• And often one that has never been tested


Proving Properties of Programs

• Definitely a promising approach

• Not just promises, we can deliver– Freedom from run-time errors

– Proof of security properties

• Security a particular important target– Secure programs do not have to work

– But they do have to be secure


The Role of Testing

• Right now, we largely rely on testing, e.g. in DO-178B

• Thorough testing definitely improves our confidence

• But testing is fundamentally incomplete

• DO-178B has no formal model, it just works

• So can we eliminate testing?

• Almost no one thinks we can eliminate 100% of testing

• I don’t care how much you have proved, I don’t want to fly on a plane with avionics that has never been tested.

• But if we can’t eliminate ANY testing, then bleak times ahead for formal methods, certainly in the safety area.

• So the idea (enshrined in DO-178C) is to figure out how to give some credit for formal methods, eliminating some testing requirements.


Proofs Replacing Testing

• Specific Unit tests might be replaced by proofs, e.g. of the correctness of an algorithm, eliminating the need for complete domain testing.

• Object Oriented Programs and Dynamic Dispatch

• Thorough testing impractical (every call to every method).

• Require LSP in OO hierarchies

• OK, we can require it, but how to verify?

• Unit testing???

• This is a place where proof techniques do make sense


Reasoning About Existing Programs

• The use of static analysis

• Finding bugs is useful, but not so useful

• Guaranteeing absence of bugs is definitely useful

• But eliminating all false negatives can generate too many false positives

• Helps to start with an appropriate language, e.g. in Ada, range declarations add a focus for analysis.

• The Microsoft Vista story– Added 500,000 annotations

– Found 100,000 potential buffer overflows

• Finding pre and post conditions

• Incremental Analysis


Putting it All Together, the Hi-Lite Project

• In practice we will make use of a combination of techniques and tools

– Proof of correctness and formal specification

– Thorough testing

– Use of static analysis tools

• All these tools can benefit from the addition of annotations about what should be going on.

– Postconditions

– Preconditions

– Invariants

• But different tools have different views– Is a precondition something to be proved?

– Or is it something to be tested?

– Or is it there to give hints to a static analyzer


Preconditions and Postconditions

• At one level, they are just formalizations of comments. Comments are a good thing if

– They are precise

– They are complete

– They are kept up to date

• But out of date or incorrect comments are a menace

• So formalizing them can only help

• Not that much work once you get used to it

• But what language should be used

• Ada 2012 adds extensive capabilities for annotations including preconditions, postconditions, invariants, and predicates.


Trying to Unify the Language Requirements

• If annotations/contracts are useful for proof, testing, and static analysis, how about we use the same annotation language, so we can integrate the different approaches within a common framework?

• Multiple approaches are problematic

• In AdaCore technology, four syntaxes for preconditions– Precondition pragma for Ada 95/2005

– Pre aspect in Ada 2012

– SPARK annotation for preconditions

– CodePeer syntax for discovered precondition


Hi-Lite

• One important goal of Hi-Lite is to create a unified language for annotations that can be used in all contexts.

• Then the second step is to create a unified development environment in which different tools can use the same source base in different contexts.

• The goal: to facilitate (i.e. make cheaper) the integration of formal methods and testing regimes.

• We need to create industrial workflows that incorporate verification on a routine basis

• These tools need to integrate different programming approaches as well (different languages, modeling …)


Problems in the Unification

• The requirements for mathematical specifications and programming language expressions are different.

• Does A+B mean A+B or does it mean apply some weird programming notion of + with overflows or wrap around?

– with Pre => A + B <= Integer’Last

• Does A or B mean A or B, or does it mean do some strange evaluation in which an overflow in B causes an exception even if A is True?

• Not insuperable, but such issues stand in the way of this being a simple job of getting people to agree on syntax.


Conclusion

• We have a lot of industrial experience to feed in here

• We have creators of tools and users very interested in participating in creating a workable framework

– AdaCore

– Praxis

– CEA

– INRIA

• We have mportant industrial partners who need this technology now, so this is much more than just research.

– Altran

– Astrium

– Thales

– Rockwell

Documents

Www.adacore.com A Pragmatic View of Formal Methods The Hi-Lite Project Robert B K Dewar SSS ‘11 President & CEO, AdaCore Emeritus Professor of Computer