Software Quality Assurance Design and implementation

Software Quality AssuranceDesign and implementation

slide 2COMP319 S Coope 2011

ObjectivesObjectives

• At the end of this lecture you should be able to;– list and describe factors which contribute to software quality– name and explain the processes used to ensure software quality


QualityQuality“Characteristics of the software and/or its supporting documentation which make it suitable for its

purpose".


Product design and conformance

Product design and conformance

• Design– characteristics of the materials and processes which have been specified for the system– includes all aspects of specification and software design

• Conformance– degree to which design specifications are followed during manufacturing– code implementation


DeMarco’s QualityDeMarco’s Quality“A products quality is a function of how much it changes the world for the better”


Quality controlQuality control

Productiondesign

process 1

Productiondesign

process 2

ReviewSample

TestMeasure


Software production quality cost

Software production quality cost

• Each individual product needs producing only once• Benefits

– economy of scale (sometimes)

• Drawbacks– all products from one release will contain same defects– implementation details may change radically from product to product– statistical sampling techniques not applicable


Software verses other manufacturing

Software verses other manufacturing

• Manufacturing industry– design once, make many– production process refined– design faults fed back on-going basis

• Software industry– design once, make once


Component quality conceptComponent quality concept• In manufacturing

– electronics, cars, aviation– components made to specification and determined to be of high quality– build product out of blackbox components

• In software– components conform to interface specification– MS Component Object Model– Sun Java Beans


Quality and reuseQuality and reuse

High reuse

should

improve quality

as code needs

to only be

tested or

designed once

only name : stringaddress : stringdob : date

Person

setAddress(String)print()

account_number :Long

Customer

pin : Integer

Bank accountholder

checkPin()

points : Integer

Clubcardholder

printStatement()

allergies: String

Patient

prescription()

Bank Supermarket Clinic


Need for quality control Cost of correcting errorsNeed for quality control

Cost of correcting errors• IBM research

– 7053 hours inspecting 200,000 lines of code, 3112 potential defects discovered– total cost $282,000– cost per defect (at $40/hour) $91/defect– IBM estimate for correcting defects in the field is $25,000– total cost of 3112 defects $5 million


Reliability EstimationReliability Estimation• Calculating probability of fault over time• Why do we need it?

– schedule maintainance– commercial (costing for warrantees, support contracts)– scheduling software release dates– determining safety of critical systems


Reliability EstimationReliability Estimation


Reliability ProblemsReliability Problems• Therac 25• Airbus 320

– altimeter reading issues

• Patriot defence missile failure– Scud missile killed 28

– 0.000000095/0.1 second over 100 hours


Reliability modellingReliability modelling

• Defect density models– analyses structure/size of software– input/outputs/complexity– can be determined before execution

• Reliability growth models– test performance– execution time– correlates defect density over time


Reliability growth modelReliability growth model• Parameters

– Total bugs in software– Total bugs discovered– Time– Residual bugs


Reliability growth modelReliability growth model• Exponential


Growth model typesGrowth model types


Growth model examplesGrowth model examplesA = a(1-e-bt)

– Goel-Okumoto or Musa model– Concave a=total defects

A = a( 1-(1+bt)e-bt) – Yamada,83 Modification of G-O model

• A= (l/c)ln(ct+l)– Infinite poisson Musa


Reliability growth model assumptions

Reliability growth model assumptions

• Defects are repaired immediately– Defects are not repaired immediately

• Defect repair doesn't introduce new defects– Generally not true

• Test cases represent operation profile of software– hard to duplicate in practise, need for alpha release phase

• Failures are independent


Using a modelUsing a model• Parameter estimation

– maximum likelihood– least squares

• Maximum likelihood– Given a set of data– Which distribution gives us the highest probability of achieving this data


Maximum liklihood exampleMaximum liklihood example• A coin is tossed 10 times• Chance of getting N heads is

– (10!/(N!(10-N)!) x (phead)^N x (1-phead)^(10-N)

• So given we get 3 heads– P=(10!/(3!(7)!) x (phead)^3 x (1-phead)^7

• What is the chance of– phead being 0.1, 0.2, 0.3 etc.

• We can define these as – P(3 | 0.1) P(3 | O.2) P(3 | 0.3) or generally– P(3 | p)


Maximum liklihood exampleMaximum liklihood example• Now imagine we have the following data (number of heads)

– 2,2,2,3,3,5

• We end up with the following data– P(2 | p) P(2 | p) P(2 | p) P(3 | p) P(3 | p) P(5 | p)

• We can multiply all those probabilities together (since they are independent)– P(2 | p) x P(2 | p) x P(2 | p) x P(3 | p) x P(3 | p) x P(5 | p)

• We then solve for max prob. Against p


Least Squares (regression)Least Squares (regression)• In this approach the parameters are chosen which minimise the error squares• (F(a,b,t)-f(t))^2


Useful estimationUseful estimation• Stable

– not vary widely in terms of total bugs estimated week per week

• Give a good indication of residual errors after release– release conditions can be very different than test conditions


Case study (Tandem)Case study (Tandem)• Tandem

– fault tolerant computer systems (now part of HP)

• Results– Release Predicated faults Faults year 1– 1 33 34– 1/2 33 28– 3 10 9


Tandem conclusionsTandem conclusions• Simplest models are the best• Execution time is the only time measure worth using• Reliability growth is a useful measure• Problem reports are a good substitute for actual defects counts (problem reports can include duplicates)• Data can be analysed on a weekly basis


McCall's software quality factors

McCall's software quality factors

ProductRevision

Producttransition

Product operations


Product operationsProduct operations• Correctness

– conformance to specification (verification)– conformance to user requirements (validation)– conformance to module specification

• Reliability– results produced to expected precision in all circumstances

• Efficiency– efficient use of hardware resource


Product operationsProduct operations

• Integrity– keeps data secure– commonly conflicts with functionality requirements

• Useability– easy to use– intuitive interface


Product revisionProduct revision• Maintainability

– can the software be fixed at a later date

• Flexibility– can the software be modified to perform new or different functions

• Testability– can the software be tested


Product transitionProduct transition• Portability

– can the software be used on another hardware or software platform

• Reusability– can the software be used to produce new software

• Interoperability– will we be able to interface to another system


Evaluating and assuring quality

Evaluating and assuring quality

• Product operations– testing (user, system, security, module, stress, usability)

• Possible metrics– faults per line of code/module– mean time between software failure– time to perform operations (usability)– mean time between break ins (security)– mean response time under stress


Product revisionquality control

Product revisionquality control

• Maintainability– review (documentation, code complexity)

• Flexibility– where to put parameters– review internationalisation issues

• Testability– reduce complexity– encapsulate data (OO paradigm)


Flexibilityconfiguration parameters

Flexibilityconfiguration parameters

• Hardcoding– very bad practise

• Using #defines or constants– better, good for system constants

– more control than configuration file

– requires recompilation

• Configuration file– highest flexibility

– allows users to break software !!


ExamplesExamplesExample 1 Hard coding

Account accounts[]=new Account[20];

for (j=0;j<20;j++) {

accounts[j]=file.readAccount();

}

Example 2 pre-processor definition

#define MAX_ACCOUNTS 20

Account accounts[]=new Account[MAX_ACCOUNTS];

for (j=0;j<MAX_ACCOUNTS;j++) {


}


Example (configuration file)Example (configuration file)Example 3 configuration file

int max_accounts;

readConfig(&max_accounts,"config.dat","maxaccounts");

Account accounts[]=new Account[max_accounts]

for (j=0;j<max_accounts;j++) {


}


TransitionTransition• Portability

– effected by language, software library support– note C does not have a standard size for int

• Reusability– code should be generalised as much as possible (including variable naming)– e.g. replace pin with authorisation_code– important to de-couple I/O from processing


PortabilityPortability• Approaches

– use wrapper classes– define UI using XML– use plain old Java classes– use model/view/controller seperation


TransitionTransition• Interoperability

– use standards e.g. Internet standards, JDBC, HTTP, XML– test against standard products e.g. browsers– next use in-house protocols if standard technique available


Software reviewsSoftware reviews• Review functionality of system

– by development team– specification team– with customer (validation)– beta release to customer


FTRsFormal technical reviews

FTRsFormal technical reviews

• Purpose– uncover defects (code, spec, design)

• Before– agenda, attendance list, documentation

• While– walkthrough, code inspection

• After– hit list, next review meeting scheduled


Specification reviewSpecification review• Is it

– clear and unambiguous– understandable– free of errors– complete– what the customer wants


Design reviewDesign review• Check

– is it broken up enough– in agreement with specification– uses correct notation– makes sense


Code reviewCode review• Is it

– bug free– well document and formatted– uses meaningful names– flexible (easy to modify)– easy to understand/test


Defect amplification(importance of review)

Defect amplification(importance of review)

• Error in specification– errors in design– errors in code– errors in product

• Error in design– errors in code– errors in product


Quality factors(environment)Quality factors(environment)

• Staff turnover• Training• Size of organisation• Resources (time and money)

Documents

Software Quality Assurance Design and implementation