Testing safety-critical software systems

Marcos Mainar Lalmolda

Quality Assurance and Testing

20th November 2009

Contents What a safety-critical software

system isStandardsProgramming features and

languagesApproaches on designTestingConclusion

What a safety-critical software system is A safety-critical software system is a computer

system whose failure or malfunction may severely harm people's lives, environment or equipment.

Some fields and examples: Medicine (patient monitors) Nuclear engineering (nuclear power station

control) Transport (railway systems, cars anti-lock

brakes) Aviation (control systems: fly-by-wire) Aerospace (NASA space shuttle) Civil engineering (calculate structures) Military devices Etc.

Safety-critical Standards Industries specific

Medical device software: IEC 62304Nuclear power stations: IEC 60880Aerospace: AS9100AAirbone: DO178B…

Scale of 5 safety integrity levels: 4 is very high, 0 not safety related.

Safety engineering

Programming features and languages (I)

General principle: Try to keep the system as simple as possible.

Programming features not recommended:Pointers and dynamic memory

allocation/deallocation.Unstructured programming (gotos)Variant data Implicit declaration and initialisationRecursionConcurrency and interrupts

Programming features and languages (II)Features which increase reliability:

Strong typingRun time constraint checkingParameter checking

Language to be avoided: CLanguage recommended: AdaAda subset for safety-critical

software: SPARKOther languages: increased

overhead

Approaches on design Formal methods

Assume that errors exist and design prevention and recovery mechanisms.

“Program verification does not mean error-proof programs […]. Mathematical proofs can also be faulty. So whereas verification might reduce the program-testing load, it cannot eliminate it” (F.P. Brooks, No Silver Bullet, 1987).

Testing safety-critical software systems (I) Basic idea: Identify hazards as early as possible in the

development life-cycle and try to reduce them as much as possible to an acceptable level.

Remember: Always test software against specifications! Independent verification required If formal methods have been used then formal

mathematical proof is a verification activity. Already known techniques used for typical systems

White box testing Black box testing Reviews Static analysis Dynamic analysis and coverage

Testing safety-critical software systems (II) Specific procedures and techniques from

safety engineering:Probabilistic risk assessment (PRA)Failure modes and effects analysis (FMEA)Fault trees analysis (FTA)Failure mode, effects and criticality analysis

(FMECA)Hazard and operatibility analysis (HAZOP)Hazard and risk analysisCause and effect diagrams (aka fishbone

diagrams or Ishikawa diagrams)

10

Probability Risk Assessment

Hazard

Severity Probability

Risk

Risk Criteria

Tolerable?No Risk

Reduction MeasuresYes

*From Safety-Critical Computer Systems – Open Questions and Approaches presentation, Andreas Gerstinger,February 16, 2007, Institute of Computer Technology, Wien

Fault tree analysis (FTA) A graphical technique that provides a systematic

description of the combinations of possible occurrences in a system which can result in an undesirable outcome (failure).

An undesired effect is taken as the root of a tree of logic

Each situation that could cause that effect is added to the tree as a series of logic expressions.

Events are labelled with actual numbers about failure probabilities.

The probability of the top-level event can be determined using mathematical techniques.

An example of a Fault tree

*From http://syque.com/quality_tools/toolbook/FTA/how.htm

Conclusions Complex subject

Suitably trained and experienced people are key to the success of any software development.

Main objective of testing techniques: minimise risk of implementation errors.

Above all, the best way to minimise risk both to safety, reliablity and to the timescale of a software project is to keep is simple.

Questions

¿?

References Wikipedia. http://en.wikipedia.org IPL Information Processing Ltd, An Introduction to Safety Critical Systems,

Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0820

IPL Information Processing Ltd, An Introduction to Software Testing, Testing Papers. http://www.ipl.com/include/download/CookieRequestPage.php?FileID=p0826

Evangelos Nikolaropoulos, Testing safety-critical software, Hewlett-Packard Journal, June 1997. http://findarticles.com/p/articles/mi_m0HPJ/is_n3_v48/ai_19540814/?tag=content;col1

Frederick P. Brooks, Jr. , No Silver Bullet: Essence and Accidents of Software Engineering, 1986.

Andreas Gerstinger, Safety-Critical Computer Systems – Open Questions and Approaches presentation, February 16, 2007, Institute of Computer Technology, Wien.

Fault Tree Analysis: How to understand it.

http://syque.com/quality_tools/toolbook/FTA/how.htm