Forensic Software Engineering: Are Software Failures Symptomatic of Systemic Problems?

Chris Johnson,

University of Glasgow

My name is Elisabeth

Software Induced Failures

Failure of software to perform an intended function– Includes elicitation and specification problems

Includes– Guam crash– Therac 25– Ariane 5– London Ambulance Computer-Aided Dispatch

Problem of Supporting Systemic Approaches to Software Failure Theorem proving

– Specification can be wrong– Environment can be wrong– Many possible paths– Model checking can help

Other Problems

Human factors– KA 801 captain/crew

Organizational issues– FAA oversight

SE techniques say what to do, not what happened

Problems of Framing Any Analysis of Software Failure Stopping point?

– CFIT– Aircraft was below minimum altitude– ATC personnel failed to notice altitude– Warning system was misconfigured– FAA did not ensure proper system function– FAA doesn’t certify ground-based systems– Public doesn’t pressure FAA to certify them– Researchers don’t inform public well enough

Existing Techniques

Fault trees– Specific software failure

Requirements engineering– Problems with requirements capture

Organizational issues– ?

Therac 25– Flaws in software or bug-fixing process?

Further Problems

So, use all of them– Chooser may be biased– Tools find causal factors suited to them– Tools may be selectively deployed to show

certain things

Problems of Assessing Intention in Software Development Why did the software fail? Why was the software erroneous? Why was the whole island inhibited? Why did Dulles look just like Tampa?

Intent Specifications

Developers include why as well as what Extension of safety case (?) External certification vs. internal development Shows why sw is built this way Shows why changes were made Helps match code to design Possibly better than current maintenance

certification procedures

Problems of Assessing Human and Environmental Factors Environment often hard to simulate

– Mars It is true because the experts say so? SW often proprietary, can’t check Can’t learn from mistakes

And Operator Error…

Who knows what people are thinking Can’t recreate situation very well People react differently, same people

may not be available Knowing what people did does not show

why they did it London Ambulance Report: what was or

wasn’t intentional?

Problems of Making Adequate Recommendations What about software process?

– What in the process is bad?– Is other code built by this process okay?

Recommendations should be feasible & should include guidance– “Identify all implicit assumptions made by the

code”

Shouldn’t set silly goals– “Totally reliable software”

Summary

No techniques involving systemic factors

No agreement about scope of cause No guidance about analytical tools All assumptions must be questioned for

reused software Want to know why as well as what

Summary (cont.)

Can’t always simulate conditions Must consider human factors Problems with scope because of

consequences of recommendations Must educate investigators & public

And a Few More Things…

Problem paper, not solution paper Intent specifications can help Maintaining safety cases & design docs

– Especially for reuse NTSB’s accident investigation academy

– What will we teach them?

Mars Climate Orbiter & Mars Polar Lander Faster, Better, Cheaper Decided not to collect telemetry data Signal was lost (expectedly) Causes included

– Long working hours– Communications problems– Deadline pressure

Goldin’s speech

Acknowledges environmental issues Points to emergent problems “System failed them”