OPR

E 63

641

Soft

ware

Qua

lity

OPR

E 63

642

Soft

war

e D

isas

ters

►G

reen

Par

ty C

onve

ntio

n fa

ils

(By

roun

ding

erro

r and

erro

neou

s us

e of

Exc

el th

e w

rong

num

ber

of d

eleg

ates

is c

ompu

ted,

200

2)

►M

ars

Clim

ate

Orb

iters

, Los

s (M

ixtu

re o

f pou

nds

and

kilo

gram

s, 1

999)

►Ar

iane

5, E

xplo

sion

(d

ata

conv

ersi

on o

f a to

o la

rge

num

ber,

1996

)►

Den

ver A

irpor

t(C

ompu

teriz

ed B

agga

ge H

andl

ing

fails

, 199

5)

►Pe

ntiu

m P

roce

ssor

, Div

isio

n Al

gorit

hm

(inco

mpl

ete

entri

es in

a lo

ok-u

p-ta

ble,

199

4)►

Patri

ot-S

cud

(roun

ding

erro

r, 19

91)

►N

ASA

Mar

iner

1 ,

Venu

s pr

obe

(per

iod

inst

ead

of c

omm

a in

FO

RTR

AN D

O-L

oop,

196

2)

OPR

E 63

643

Erro

r Ca

tego

ries

10 D

ocum

enta

tion

pro

blem

20 S

ynta

x30

Bui

ld40

Ass

ignm

ent

50 I

nter

face

60 E

rror

Han

dlin

g70

Dat

a80

Fun

ctio

n90

Sys

tem

100

Envi

ronm

ent

OPR

E 63

644

Def

ect c

lass

ifica

tion

•D

efec

ts a

re c

lass

ified

on

the

basi

s of

def

ect t

ype,

cla

ssan

d se

verit

y

•Type

is th

e ki

nd o

f def

ect c

ondi

tion.

–D

A (d

ata)

a de

fect

in in

tern

al d

ata

use

or s

peci

ficat

ion

–D

C (d

ocum

ent)

inad

equa

te, i

rrele

vant

or i

ncor

rect

des

crip

tion

–FN

(fun

ctio

nalit

y)an

inco

rrect

spe

cific

atio

n–

HF

(hum

an fa

ctor

s)a

defe

ct in

ope

ratio

nal p

roce

dure

or h

uman

inte

rface

–IF

(int

erfa

ce)

a de

fect

in th

e co

mm

unic

atio

n be

twee

n co

mpo

nent

s–

LO (l

ogic

)a

defe

ct in

pro

cedu

ral,

algo

rithm

ic, o

r con

trol l

ogic

–M

N (m

aint

aina

bilit

y) th

e co

mpo

nent

can

not b

e m

aint

aine

d ea

sily

–PF

(per

form

ance

)op

erat

iona

l effi

cien

cy m

ay g

et s

acrif

iced

–SN

(syn

tax)

a de

fect

in la

ngua

ge u

sage

–ST

(sta

ndar

ds)

a de

partu

re fr

om re

pres

enta

tiona

l sta

ndar

ds–

OT

(oth

er)

a de

fect

con

ditio

n th

at h

as n

ot b

een

spec

ified

OPR

E 63

645

Softw

are

Qua

lity

Assu

ranc

e•

Softw

are

Qua

lity

invo

lves

–En

surin

g co

nfor

man

ce w

ith u

ser r

equi

rem

ents

–Id

entif

ying

def

ects

in e

arly

pha

ses

of d

evel

opm

ent

–M

onito

ring

the

prod

ucts

thro

ugh

its p

hase

s of

de

velo

pmen

t•

Qua

lity

assu

ranc

e du

ring

the

deve

lopm

ent

–Ac

hiev

ed th

roug

h St

ruct

ured

Wal

kthr

ough

s(a

lso

know

n as

Sof

twar

e In

spec

tion)

•Q

ualit

y As

sura

nce

afte

r dev

elop

men

t–

Achi

eved

thro

ugh

pre-

plan

ned

Test

ing

of th

e en

d pr

oduc

t

OPR

E 63

646

Tota

l Q

ual

ity

Man

agem

ent

princi

ple

s an

d t

ools

apply

!Cont

inuous

Process I

mprovemen TQ

M

Cust

omer

Sat

isfa

ctio

n:10

0% C

ompl

ianc

e wi

th E

xpec

tati

ons

Statisti

cal Pr

ocess

Control

t

OPR

E 63

647

“So

Wha

t?”

•Be

caus

e a

SW is

bui

lt us

ing

proc

esse

s•

Proc

esse

s ar

e no

t alw

ays

perfe

ct•

We

can

impr

ove

whi

le a

void

ing

the

Dilb

ert

synd

rom

e•

Rem

embe

r, w

e ca

n’t t

est q

ualit

y in

to o

ur

softw

are,

we

have

to d

esig

n it

in!

OPR

E 63

648

Dem

ing’s

PD

CA C

ycle

Plan

Do

Che

ckA

ct

Wha

t’s th

e ne

xt p

robl

em?

Wha

t can

we

do?

Try

impr

ovem

ent

Smal

l tes

t

Gat

her d

ata

And

ana

lyze

Ado

pt n

ew p

roce

ss

OPR

E 63

649

Dilb

ert

Cyc

le

Ado

pt n

ewU

np

roven

pr

oces

sA

ctM

ake

wil

d g

uess

At w

hat i

s wro

ng

OPR

E 63

6410

Moder

n Q

ual

ity

Man

agem

ent

•M

oder

n qu

ality

man

agem

ent

–re

quire

s cu

stom

er s

atis

fact

ion

–pr

efer

s pr

even

tion

to in

spec

tion

–re

cogn

izes

man

agem

ent r

espo

nsib

ility

for

qual

ity•

Not

ewor

thy

qual

ity e

xper

ts in

clud

e D

emin

g,

Jura

n, C

rosb

y, Is

hika

wa,

Tag

uchi

, and

Fe

igen

baum

OPR

E 63

6411

Rec

all Q

ual

ity

Guru

s…

Dem

ing

was

fam

ous

for h

is w

ork

in re

build

ing

Japa

n an

d fo

r his

14

poin

tsJu

ran

wro

te th

e Q

ualit

y C

ontro

l Han

dboo

k an

d th

e 10

ste

ps to

qua

lity

impr

ovem

ent

Cro

sby

wro

te “Q

ualit

y is

Fre

e” a

nd s

ugge

sted

that

or

gani

zatio

ns s

houl

d st

rive

for z

ero

defe

cts

Ishi

kaw

a de

velo

ped

the

conc

ept o

f qua

lity

circ

les

and

the

use

of fi

shbo

ne d

iagr

ams

Tagu

chi d

evel

oped

met

hods

for o

ptim

izin

g th

e pr

oces

s of

eng

inee

ring

expe

rimen

tatio

nFe

igen

baum

dev

elop

ed th

e co

ncep

t of t

otal

qua

lity

cont

rol

OPR

E 63

6412

Lead

ersh

ip c

an’t b

e es

caped

!

•“It

is m

ost i

mpo

rtant

that

top

man

agem

ent b

e qu

ality

-m

inde

d. In

the

abse

nce

of s

ince

re m

anife

stat

ion

of

inte

rest

at t

he to

p, li

ttle

will

happ

en b

elow

.” (J

uran

, 194

5)•

A la

rge

perc

enta

ge o

f qua

lity

prob

lem

s ar

e as

soci

ated

w

ith m

anag

emen

t, no

t tec

hnic

al is

sues

OPR

E 63

6413

More

Soft

war

e D

isas

ters

…►

Rad

io T

eles

cope

VLA

, cal

ibra

tion

(roun

ding

erro

r, 19

90-1

995)

►El

ectio

ns in

Sch

esw

igH

olst

ein/

Ger

man

y (R

ound

ing

lifts

the

Gre

en P

arty

from

4.9

7% to

5.0

%, 1

992)

►

Slei

pner

Offs

hore

Pla

tform

(Sin

king

cau

sed

by th

e w

rong

use

of F

E-co

de N

ASTR

AN, 1

991)

►AT

&T lo

ng d

ista

nce

serv

ice

fails

for n

ine

hour

s(W

rong

BR

EAK

stat

emen

t in

C-C

ode,

199

0)►

Phob

os1,

Rus

sian

Mar

s Pr

obe

lost

(Wro

ng c

omm

and

lead

s to

rota

tion,

198

8)►

Airb

us d

owni

ng d

urin

g Ira

n-co

nflic

t(P

atte

rn re

cogn

ition

sof

twar

e, 1

988)

►Fa

lse

alar

m in

Sov

iet e

arly

-war

ning

mon

itorin

g sy

stem

(Pat

tern

reco

gniti

on, 1

983)

OPR

E 63

6414

The

Cost

of Q

ual

ity

Cos

t of q

ualit

y is

–

The

cost

of c

onfo

rman

ce o

r del

iver

ing

prod

ucts

that

mee

t req

uire

men

ts a

nd fi

tnes

s fo

r use

–Th

e co

st o

f non

conf

orm

ance

or t

akin

g re

spon

sibi

lity

for f

ailu

res

or n

ot m

eetin

g qu

ality

exp

ecta

tions

OPR

E 63

6415

Cost

s Pe

r H

our

of

Dow

ntim

e Cau

sed

by

Soft

war

e D

efec

ts

Bus

ines

sC

ost p

er H

our

Dow

ntim

e

Aut

omat

ed te

ller m

achi

nes (

med

ium

-siz

ed b

ank)

$14,

500

Pack

age

ship

ping

serv

ice

$28,

250

Tele

phon

e tic

ket s

ales

$69,

000

Cat

alog

sale

s cen

ter

$90,

000

Airl

ine

rese

rvat

ion

cent

er (s

mal

l airl

ine)

$89,

500

In a

dditio

n,

poor

qual

ity

may

cost

you

fore

goin

g r

epea

t busi

nes

s

OPR

E 63

6416

Five

Cost

Cat

egories

Rel

ated

to Q

ual

ity

•Pr

even

tion

cost

: the

cos

t of p

lann

ing

and

exec

utin

g a

proj

ect s

o it

is e

rror-f

ree

or w

ithin

an

acce

ptab

le e

rror

rang

e•

App

rais

al c

ost:

the

cost

of e

valu

atin

g pr

oces

ses

and

thei

r ou

tput

s to

ens

ure

qual

ity•

Inte

rnal

failu

re c

ost:

cost

incu

rred

to c

orre

ct a

n id

entif

ied

defe

ct b

efor

e th

e cu

stom

er re

ceiv

es th

e pr

oduc

t•

Exte

rnal

failu

re c

ost:

cost

that

rela

tes

to a

ll er

rors

not

de

tect

ed a

nd c

orre

cted

bef

ore

deliv

ery

to th

e cu

stom

er•

Mea

sure

men

t and

test

equ

ipm

ent c

osts

: cap

ital c

ost o

f eq

uipm

ent u

sed

to p

erfo

rm p

reve

ntio

n an

d ap

prai

sal

activ

ities

OPR

E 63

6417

Tw

o a

spec

ts o

f Soft

war

e Q

ual

ity

Proc

ess

Qua

lity

–En

surin

g co

nfor

man

ce w

ith u

ser r

equi

rem

ents

–Id

entif

ying

def

ects

–M

onito

ring

the

prod

uct t

hrou

gh it

s ph

ases

of

deve

lopm

ent

Prod

uct Q

ualit

y–

Iden

tifyi

ng u

ser s

peci

fied

qual

ity n

eeds

–Pr

iorit

izin

g qu

ality

nee

ds–

Res

olvi

ng q

ualit

y co

nflic

ts, i

f any

–Bu

ildin

g th

em in

to th

e de

velo

pmen

t pro

cess

–Al

loca

ting

effo

rt an

d tim

e fo

r the

m

OPR

E 63

6418

Mal

colm

Bal

drige

Aw

ard a

nd

ISO

9000

•Th

e M

alco

lm B

aldr

ige

Qua

lity

Awar

d w

as s

tarte

d in

19

87 to

reco

gniz

e co

mpa

nies

with

wor

ld-c

lass

qu

ality

•IS

O 9

000

prov

ides

min

imum

requ

irem

ents

for a

n or

gani

zatio

n to

mee

t the

ir qu

ality

cer

tific

atio

n st

anda

rds

•H

owev

er, n

one

assu

re th

e qu

ality

of t

he fi

nal

prod

uct!

OPR

E 63

6419

Qual

ity

Plan

nin

g

•It

is im

porta

nt to

(a)

“des

ign

in” q

ualit

y an

d (b

) co

mm

unic

ate

with

in th

e or

gani

zatio

n th

e im

porta

nt

fact

ors

that

dire

ctly

con

tribu

te to

mee

ting

the

cust

omer

’s re

quire

men

ts•

Tool

s su

ch a

s D

esig

ned

Expe

rimen

tatio

n ca

n id

entif

y fa

ctor

s ha

ve th

e m

ost i

nflu

ence

on

the

over

all

outc

ome

of a

pro

cess

•M

any

“sco

pe” a

spec

ts o

f IT

proj

ects

als

o af

fect

cu

stom

er s

atis

fact

ion

or q

ualit

y, s

uch

as fu

nctio

nalit

y,

feat

ures

, sys

tem

out

puts

, per

form

ance

, rel

iabi

lity,

and

m

aint

aina

bilit

y

OPR

E 63

6420

Qual

ity

Ass

ura

nce

•Q

ualit

y as

sura

nce

incl

udes

all

the

activ

ities

rela

ted

to

satis

fyin

g th

e re

leva

nt q

ualit

y st

anda

rds

for a

pro

ject

•An

othe

r goa

l of q

ualit

y as

sura

nce

is c

ontin

uous

qu

ality

impr

ovem

ent

•Be

nchm

arki

ng c

an b

e us

ed to

gen

erat

e id

eas

for

qual

ity im

prov

emen

ts•

Qua

lity

audi

ts h

elp

iden

tify

less

ons

lear

ned

that

can

im

prov

e pe

rform

ance

on

curre

nt o

r fut

ure

proj

ects

OPR

E 63

6421

Qual

ity

Contr

ol

•Th

e m

ain

outp

uts

of q

ualit

y co

ntro

l are

–ac

cept

ance

dec

isio

ns–

rew

ork

–pr

oces

s ad

just

men

ts•

Som

e to

ols

and

tech

niqu

es in

clud

e–

pare

to a

naly

sis

–st

atis

tical

sam

plin

g–

qual

ity c

ontro

l cha

rts–

test

ing

OPR

E 63

6422

Freq

uen

tly

sought

afte

r qual

ity

fact

ors

Use

r's N

eed

Use

r's C

once

rnQ

ualit

y Fa

ctor

FUN

CTI

ON

ALH

ow s

ecur

e is

it ?

INTE

GR

ITY

How

ofte

n w

ill it

fail

?R

ELIA

BILI

TYC

an it

sur

vive

dur

ing

failu

re ?

SUR

VIVA

BILI

TYH

ow e

asy

is it

to u

se ?

USA

BILI

TYPE

RFO

RM

ANC

EH

ow m

uch

reso

urce

s ar

e ne

eded

?EF

FIC

IEN

CY

Doe

s it

com

ply

with

requ

irem

ents

?C

OR

REC

TNES

SD

oes

it pr

even

t haz

ards

?SA

FETY

Doe

s it

inte

rface

eas

ily ?

INTE

RO

PER

ABIL

ITY

CH

ANG

EH

ow e

asy

is it

to re

pair

?M

AIN

TAIN

ABIL

ITY

How

eas

y is

it to

exp

and

?EX

PAN

DAB

ILIT

YH

ow e

asy

is it

to c

hang

e ?

FLEX

IBIL

ITY

How

eas

y is

it to

tran

spor

t ?PO

RTA

BILI

TYIs

it re

usab

le in

oth

er s

yste

ms

?R

EUSA

BILI

TYM

ANAG

EMEN

TIs

per

form

ance

ver

ifica

tion

easy

?VE

RIF

IABI

LITY

Is th

e so

ftwar

e ea

sily

man

aged

?M

ANAG

EABI

LITY

OPR

E 63

6423

Req

uirem

ents

Anal

ysis

Ana

lyze

nee

dfo

r qua

lity

Con

vert

qual

ityne

eds t

ore

quire

men

ts

Doc

umen

t SW

Qua

lity

requ

irem

ents

Use

r opi

nion

s

Crit

eria

for g

ood

requ

irem

ents

Req

spec

Der

ived

req’

s

Qua

lity

fact

ors

Nee

ds d

ata

base

Qua

lity

conf

licts

Cos

t of q

ualit

y

Fact

or a

nd c

riter

ia d

efin

ition

s

Leve

l of q

ualit

y m

atrix

Qua

lity

need

s dat

a ba

se

Engi

neer

ing

crite

ria

Trac

eabi

lity

mat

rix

Softw

are

qrs

Qua

lity

spec

ifica

tion

guid

elin

e

OPR

E 63

6424

The

Qual

ity

Nee

ds

dat

abas

e

•N

EED

Text

ual d

escr

iptio

n of

the

need

•SO

UR

CE

Prec

ise

desc

riptio

n of

the

sour

ce•

IMPO

RTA

NC

EPe

rcei

ved

impo

rtanc

e of

the

need

•R

EQU

IREM

ENT

Keyw

ord

for e

ach

need

•SU

BJEC

TKe

ywor

d fo

r the

sub

ject

of t

he n

eed

OPR

E 63

6425

Exa

mple

of

a U

ser

Nee

ds

Dat

abas

e

Nee

dSo

urce

Im

port

ance

R

equi

rem

ent

S

ubje

ct

Prec

isio

n an

d ac

cura

cy o

f the

P. S

mith

8

prec

ise

arith

met

icin

puts

shou

ld b

e pr

eser

ved

in10

/9/9

9al

l cal

cula

tions

.

Emph

asis

shou

ld b

e on

faul

tR

P, p

g 3

5

faul

tde

sign

tole

ranc

e.pa

ra2

tole

rant

Cod

e sh

ould

be

rege

nera

ble

SDD

,

7

com

plet

esu

ppor

tus

ing

deliv

ered

softw

are.

Pg 1

0,pa

ra 4

softw

are

Softw

are

to b

e us

ed b

yR

P, p

g 2

8

user

desi

gnN

on-te

chni

cal p

eopl

epa

ra4

frie

ndlin

ess

OPR

E 63

6426

Prio

ritizi

ng Q

ual

ity

Nee

ds

•O

ne p

ossi

ble

met

ric c

ould

be:

–E

(Exc

elle

nt)

G

(Goo

d)

A

(Ave

rage

)

NI (

Not

an

issu

e)•

Clie

nt’s

cho

ice

is b

ased

on

–Po

ssib

le q

ualit

y co

nflic

ts–

Cos

t•

Qua

lity

does

not

com

e fre

e an

d re

quire

s ad

ditio

nal e

ffort

and

cost

Rep

ercu

ssio

n on

revi

ew a

ctiv

ity, f

or e

xam

ple,

Qua

lity

Nee

d

Rev

iew

s ne

eded

Not

an

issu

e

int

erna

l rev

iew

cyc

le, i

nter

nal w

alkt

hrou

ghs

Aver

age

inte

rnal

revi

ew c

ycle

, for

mal

des

ign

revi

ew (s

ubse

t)G

ood

inte

rnal

revi

ew c

ycle

, ins

pect

ions

, for

mal

des

ign

revi

ews

(full

set)

Exce

llent

inte

rnal

revi

ew c

ycle

, ins

pect

ions

, for

mal

des

ign

revi

ew, I

V &

V

OPR

E 63

6427

-Ve

rifia

bilit

y (1

5)+

+-

Usa

bilit

y (1

4)+

--

+-

--

-Su

rviv

abilit

y (1

3)+

++

+-

Safe

ty (1

2)-

-+

++

--

Reu

sabi

lity

(11)

+-

Rel

iabi

lity

(10)

-Po

rtabi

lity

(9)

++

+M

anag

eabi

lity

(8)

++

++

-M

aint

aina

bilit

y (7

)-

-In

tero

pera

bilit

y (6

)-

Inte

grity

(5)

--

+-

-Fl

exib

ility

(4)

--

+-

-Ex

pand

abilit

y (3

)-

--

Effic

ienc

y (2

)+

++

++

Cor

rect

ness

(1)

1514

1312

1110

98

76

54

32

1Q

ualit

y N

eeds

in

tera

ct

OPR

E 63

6428

Res

olvi

ng Q

ualit

y C

onfli

cts

Alo

ng w

ith t

he

clie

nt,

go b

ack

to t

he

leve

l-of

-im

port

ance

and a

dju

st it

by

usi

ng t

he

follo

win

g p

roce

dure

:1.

For

the

each

soft

war

e co

mponen

t st

arting f

rom

the

firs

t co

lum

n,

2.

For

each

conflic

ting r

elat

ionsh

ip (

-) b

etw

een f

acto

r A a

nd f

acto

r B,

3.

Adju

st t

he

two lev

els

of

qual

ity

usi

ng t

he

follo

win

g t

able

(this

tab

le is

for

guid

ance

only

).If

spec

ifie

d f

or

Fact

or

AThen

at

most

for

Fact

or

BE

NI

GA

AG

NI

E4.

Rev

isit p

revi

ousl

y ad

just

ed r

elationsh

ip t

o m

ake

sure

that

they

hav

e not

bee

n c

han

ged

OPR

E 63

6429

Build

ing q

ual

ity

in d

evel

opm

ent

pro

cess

The

resu

lting

qua

lity

fact

ors,

alo

ng w

ith t

heir

leve

ls,

ente

r th

e So

ftwar

e R

equi

rem

ent S

peci

ficat

ion

docu

men

t as

attr

ibut

es.

For

exam

ple,

if

we

are

follo

win

g IE

EE S

RS

Doc

umen

tatio

n St

anda

rd

(Pro

toty

pe

outli

ne

1)

as

the

SRS

docu

men

tatio

n pr

actic

e, th

e qu

ality

req

uire

men

ts w

ill ge

t pla

ced

in s

ectio

n 3.

5 as

giv

en b

elow

:1.

. . .

2.. .

.3.

Spec

ific

requ

irem

ents

3.1

Func

tiona

l req

uire

men

ts3.

2Ex

tern

al in

terfa

ce re

quire

men

ts3.

3Pe

rform

ance

requ

irem

ents

3.4

Des

ign

cons

train

ts3.

5A

ttrib

utes

3.6

Oth

er re

quire

men

ts

OPR

E 63

6430

Docu

men

tation

•R

EQU

IREM

ENT

Text

of a

ctua

l req

uire

men

t•

SOU

RC

ESo

urce

of r

equi

rem

ent

•C

RIT

ERIO

NC

riter

ion

the

requ

irem

ent b

elon

gs to

•D

ESIG

N‘x

’ if r

equi

rem

ent i

s ve

rifia

ble

in th

ede

sign

pha

se•

CO

DE

‘x’ i

f req

uire

men

t is

verif

iabl

e in

the

codi

ng p

hase

•TE

ST‘x

’ if r

equi

rem

ent i

s ve

rifia

ble

in th

ete

stin

g ph

ase

OPR

E 63

6431

Adju

stin

g ef

fort

and

sche

dule

CO

CO

MO

stu

dy h

as b

een

exte

nded

and

can

now

be

used

to

giv

e an

idea

of

the

extra

cos

t ne

eded

for

the

qua

lity

conc

erns

. The

var

iatio

n in

the

CO

CO

MO

cos

t an

d tim

e sc

hedu

le c

an b

e as

giv

en in

the

tabl

e be

low

.

Qua

lity

leve

lR

elat

ive

Cos

tR

elat

ive

Sche

dule

Not

an

issu

e0.

69 to

1.0

00.

89 to

1.0

0Av

erag

e0.

90 to

1.1

60.

97 to

1.0

5G

ood

1.10

to 1

.30

1.03

to 1

.09

Exce

llent

1.48

to 2

.06

1.13

to 1

.26

OPR

E 63

6432

Qua

lity

by S

truct

ured

Wal

kthr

ough

•Is

use

d to

ver

ify in

telle

ctua

l pro

duct

s by

man

ually

ex

amin

ing

the

deve

lope

d pr

oduc

t, a

piec

e at

a ti

me,

by

sm

all g

roup

s of

pee

rs•

Expl

oits

the

syne

rgy

crea

ted

by a

sm

all g

roup

of

peop

le w

orki

ng to

geth

er•

Perfo

rms

exam

inat

ion

of w

ork

prod

ucts

at d

efin

ed

chec

kpoi

nts

•U

ses

a de

fined

six

ste

p pr

oced

ure

•Ke

eps

reco

rd o

f erro

rs d

etec

ted

for q

ualit

y co

ntro

l an

d pr

oces

s m

anag

emen

t•

Is n

ot a

foru

m fo

r des

ign

deci

sion

s, n

or it

is a

br

ains

torm

ing

sess

ion

OPR

E 63

6433

Stru

ctur

ed W

alkt

hrou

gh In

spec

tion

Req

uire

s m

etic

ulou

s pl

anni

ng a

bout

•W

hat c

an b

e in

spec

ted

•W

hen

it ca

n be

insp

ecte

d•

Who

can

insp

ect i

t•

Wha

t pre

para

tion

is n

eede

d•

How

the

insp

ectio

n is

to b

e co

nduc

ted

•W

hat d

ata

need

s to

be

colle

cted

•W

hat f

ollo

w u

p ac

tion

is re

quire

d

OPR

E 63

6434

The

Insp

ectio

n te

am

•Pr

oced

ural

role

s–

Auth

or–

Mod

erat

or–

Rea

der

–R

ecor

der

–In

spec

tor

•Fu

nctio

nal v

iew

poin

ts–

Lead

er–

Req

uest

er–

Peer

–R

ecei

ver

–Va

lidat

or–

stan

dard

izer

OPR

E 63

6435

Gui

delin

es fo

r tea

m

•Th

e m

inim

um s

ize

shou

ld b

e th

ree

: a

mod

erat

or/re

cord

er, a

read

er a

nd a

n au

thor

. Al

l tea

m

mem

bers

are

insp

ecto

rs.

•Te

am s

houl

d no

t hav

e m

ore

than

sev

en p

erso

ns.

•Th

e in

spec

tion

is a

pee

r pro

cess

. No

man

ager

s sh

ould

be

invo

lved

. The

se a

re te

chni

cal a

nd n

ot

man

agem

ent m

eetin

gs.

•O

ther

mem

bers

cho

sen

shou

ld b

e fa

milia

r with

the

insp

ectio

n pr

oces

s as

wel

l as

the

wor

k ar

ea.

OPR

E 63

6436

Step

1 :

Plan

ning

Purp

ose

: Org

aniz

atio

nR

oles

: Mod

erat

or, A

utho

rTa

sks

: App

rove

ent

ry c

riter

iaEs

tabl

ish

sche

dule

sD

esig

nate

par

ticip

ants

Det

erm

ine

over

view

Dis

tribu

te m

ater

ial

Prep

are

insp

ectio

n m

eetin

g no

tice

Inpu

ts: C

ompl

eted

wor

k pr

oduc

t, In

spec

tion

rate

sO

utpu

ts: M

eetin

g sc

hedu

le, I

nspe

ctio

n m

ater

ial

OPR

E 63

6437

Insp

ectio

n R

ates

Dev

elop

men

t Sta

geO

VIEW

Rat

ePR

EP R

ate

INSP

Rat

e

Req

uire

men

ts50

025

025

0Pr

elim

inar

y de

sign

500

200

200

Det

aile

d de

sign

500

150

150

Sour

ce c

ode

300

150

150

Test

pla

ns50

020

020

0Te

st c

ases

300

150

150

Rat

es a

re in

line

s pe

r hou

r. A

page

is ro

ughl

y eq

uiva

lent

to 5

0 lin

es.

The

abov

e ra

tes

wer

e fo

und

effe

ctiv

e in

sys

tem

con

trol a

pplic

atio

ns.

OPR

E 63

6438

Step

2 :

Ove

rvie

w

Purp

ose

: Tut

oria

lR

oles

: Mod

erat

or, A

utho

r, In

spec

tors

Task

: Pre

sent

atio

nIn

put

: Wor

k pr

oduc

t, vi

sual

aid

sO

verv

iew

is o

ptio

nal.

Rea

sons

for s

ched

ulin

g it

may

be:

–Th

e w

ork

prod

uct i

s cr

itica

l to

the

proj

ect

–Th

e w

ork

prod

uct i

s la

rge

or c

ompl

ex–

The

wor

k pr

oduc

t rep

rese

nts

use

of a

tech

nolo

gy th

at

is n

ew o

r inf

requ

ently

use

d–

The

wor

k pr

oduc

t is

the

resu

lt of

a ‘o

ne-p

erso

n’

proj

ect

OPR

E 63

6439

Step

3 :

Prep

arat

ion

Purp

ose

: Und

erst

andi

ng; i

dent

ifyin

g po

tent

ial d

efec

tsR

oles

: Ins

pect

ors

Task

s: S

tudy

mat

eria

l to

be in

spec

ted

Inpu

t: W

ork

Prod

uct

Insp

ectio

n ch

eckl

ist

Rec

omm

ende

d pr

epar

atio

n tim

eO

utpu

t: N

oted

def

ects

Not

ed p

repa

ratio

n tim

e

OPR

E 63

6440

Step

4 :

Insp

ectio

n M

eetin

g

Purp

ose

: Ver

ify p

rodu

ctR

oles

: Mod

erat

or, a

utho

r, re

ader

, rec

orde

r, in

spec

tors

Task

s: I

ntro

duce

insp

ectio

n m

eetin

gEs

tabl

ish

prep

arat

ion

Rea

ding

and

reco

rdin

g de

fect

sR

evie

w li

sted

def

ects

Det

erm

ine

prod

uct d

ispo

sitio

nIn

put

: Wor

k pr

oduc

tIn

spec

tor’s

pre

para

tion

time

Insp

ecto

r’s p

repa

ratio

n no

tatio

nsO

utpu

t: L

ist o

f ide

ntifi

ed d

efec

tsM

eetin

g re

port

and

defe

ct s

umm

ary

OPR

E 63

6441

Def

ect c

lass

ifica

tion

•D

efec

ts a

re c

lass

ified

on

the

basi

s of

def

ect t

ype,

cla

ssan

d se

verit

y

•Type

is th

e ki

nd o

f def

ect c

ondi

tion.

–D

A (d

ata)

a de

fect

in in

tern

al d

ata

use

or s

peci

ficat

ion

–D

C (d

ocum

ent)

inad

equa

te, i

rrele

vant

or i

ncor

rect

des

crip

tion

–FN

(fun

ctio

nalit

y)an

inco

rrect

spe

cific

atio

n–

HF

(hum

an fa

ctor

s)a

defe

ct in

ope

ratio

nal p

roce

dure

or h

uman

inte

rface

–IF

(int

erfa

ce)

a de

fect

in th

e co

mm

unic

atio

n be

twee

n co

mpo

nent

s–

LO (l

ogic

)a

defe

ct in

pro

cedu

ral,

algo

rithm

ic, o

r con

trol l

ogic

–M

N (m

aint

aina

bilit

y) th

e co

mpo

nent

can

not b

e m

aint

aine

d ea

sily

–PF

(per

form

ance

)op

erat

iona

l effi

cien

cy m

ay g

et s

acrif

iced

–SN

(syn

tax)

a de

fect

in la

ngua

ge u

sage

–ST

(sta

ndar

ds)

a de

partu

re fr

om re

pres

enta

tiona

l sta

ndar

ds–

OT

(oth

er)

a de

fect

con

ditio

n th

at h

as n

ot b

een

spec

ified

OPR

E 63

6442

Def

ect c

lass

ifica

tion

•Class

is th

e w

ay th

e de

fect

isex

pres

sed

orad

dres

sed.

–

M (m

issi

ng)

the

pro

duct

is m

issi

ng m

ater

ial t

hat n

eeds

to b

e ad

ded

–W

(wro

ng)

th

e pr

oduc

t con

tain

s in

corre

ct o

r unc

lear

mat

eria

l–

E (e

xtra

)th

e pr

oduc

t con

tain

s ex

tra m

ater

ial

•Severity

is th

eim

pact

that

the

defe

ct is

exp

ecte

d to

hav

e on

the

wor

k pr

oduc

t or t

he d

evel

opm

ent p

roce

ss.

–J

(maj

or)a

def

ect t

hat i

s ex

pect

ed to

cau

se p

rodu

ct fa

ilure

, dep

artu

refro

m s

peci

ficat

ions

, or p

reve

nt fu

rther

cor

rect

dev

elop

men

t.–

N (m

inor

)a d

efec

t tha

t red

uces

the

effe

ctiv

enes

s, o

r con

fuse

s a

prod

uct's

repr

esen

tatio

n or

form

at, b

ut is

not

exp

ecte

d to

impa

ct th

e op

erat

ion

or fu

rther

dev

elop

men

t of t

he p

rodu

ct.

OPR

E 63

6443

Insp

ectio

n D

efec

t Lis

t

Mee

ting

Dat

e :

Proj

ect :

Rel

ease

:Ac

tivity

:D

ocum

ent :

Com

pone

nt :

Mod

erat

or :

Mee

ting

Type

:In

spec

tion

(I)R

eins

pect

ion

(R)

Mai

nten

ance

(M)

Insp

ectio

n Ty

pe :

Hig

h-le

vel d

esig

n(H

D)

Det

aile

d de

sign

(DD

)C

ode

(CD

)

Loca

tion

Def

ect D

escr

iptio

nTy

peC

lass

Seve

rity

------

-----

------

------

----

----

-----

------

------

-----

------

------

------

----

----

-----

------

------

-----

------

------

----

----

-----

------

---

OPR

E 63

6444

Insp

ection D

efec

t Sum

mar

yM

eetin

g D

ate

:Pr

ojec

t :R

elea

se :

Activ

ity :

Doc

umen

t :C

ompo

nent

:M

oder

ator

:M

eetin

g Ty

pe :

Insp

ectio

n (I)

Rei

nspe

ctio

n(R

)M

aint

enan

ce (M

)In

spec

tion

Type

:H

DD

DC

DD

ispo

sitio

n :

Acce

pt (A

)C

ondi

tiona

l (C

)R

eins

pect

(R)

Min

or D

efec

tsM

inor

Def

ects

Def

ect

M

W

E T

otal

M

W

E T

otal

DA

DC

FN

… OT

Tota

l

OPR

E 63

6445

Product

Dis

posi

tion

Acce

pt(A

) :

Acce

pt t

he w

ork

prod

uct

as c

ompl

ete,

with

out

any

furth

er

verif

icat

ion

of r

ewor

k. T

his

does

not

req

uire

the

wor

k pr

oduc

t to

be

defe

ct fr

ee. B

ut it

doe

s re

quire

that

ther

e be

no

defe

cts

that

cau

se th

e pr

oduc

t to

devi

ate

from

its

spec

ifica

tions

, and

that

ther

e ar

e on

ly v

ery

few

triv

ial d

efec

ts.

Con

ditio

nal(

C) :

Con

ditio

nally

acc

ept t

he w

ork

prod

uct,

with

ver

ifica

tion

of

the

rew

ork

by th

e m

oder

ator

. In

this

cas

e th

ere

are

som

e m

ajor

def

ects

, bu

t the

y ar

e fe

w, r

elat

ive

to th

e w

ork

prod

uct,

and

thei

r re

wor

kis

not

ex

pect

ed to

cre

ate

any

subs

tant

ial c

hang

es in

the

'des

ign

prem

ise'

of

the

wor

k pr

oduc

t.

Rei

nspe

ct(R

) :

Rei

nspe

ctth

e au

thor

's r

ewor

k. T

his

disp

ositi

on r

equi

res

that

the

rew

ork

be e

xam

ined

by

the

mod

erat

or, t

he a

utho

r and

at l

east

on

e ot

her

mem

ber

of t

he i

nspe

ctio

n te

am.

For

this

cas

e, t

here

are

ei

ther

a s

ubst

antia

l num

ber

of m

ajor

def

ects

, re

wor

k th

at w

ill ch

ange

th

e de

sign

pre

mis

e of

the

wor

k pr

oduc

t.

OPR

E 63

6446

Step

5 :

Rew

ork

Purp

ose

: Mee

t exi

t crit

eria

Rol

es: A

utho

rTa

sks

: Res

olve

all

iden

tifie

d de

fect

sIn

put

: Ins

pect

ion

defe

ct li

stW

ork

prod

uct d

ispo

sitio

nSc

hedu

le fo

r mod

erat

or re

view

Out

put

: Wor

ked

upon

wor

k pr

oduc

tD

ocum

enta

tion

of d

efec

t res

olut

ion

OPR

E 63

6447

Step

6 :

Follo

w-u

p

Purp

ose

: Cer

tify

insp

ectio

nR

oles

: Mod

erat

or, A

utho

r, (In

spec

tors

)Ta

sks

: Ver

ify a

ll re

wor

k; R

epor

t res

ults

Inpu

t: R

evis

ed w

ork

prod

uct

Doc

umen

tatio

n of

def

ect r

esol

utio

nO

utpu

t: C

ompl

eted

insp

ectio

n re

port

OPR

E 63

6448

NASA S

hutt

le s

atel

lite

link

snag

s co

mpute

r pro

ble

ms

The

NAS

A sa

tellit

e lin

k th

at n

orm

ally

tran

smits

vid

eo o

f the

sta

tion

to E

arth

ha

s be

en in

terru

pted

by

the

com

pute

r pro

blem

s. T

he s

pace

age

ncy

cons

ider

ed th

e re

stor

atio

n of

the

vide

o lin

k a

low

prio

rity

in th

e re

cove

ry o

f th

e co

mpu

ters

. Th

ose

diffi

culti

es s

urfa

ced

Tues

day

whe

n th

e st

atio

n's

thre

e cr

itica

l co

mm

and

and

cont

rol c

ompu

ters

beg

an to

fail

inex

plic

ably

. The

dev

ices

co

ntro

l the

gui

danc

e an

d st

eerin

g of

the

stat

ion,

pro

duct

ion

of e

lect

rical

po

wer

, com

mun

icat

ions

with

Mis

sion

Con

trol a

s w

ell a

s th

e op

erat

ion

of th

e ne

w ro

bot a

rm.

By S

unda

y, e

xper

ts b

elie

ved

the

prob

lem

s co

uld

have

bee

n ca

used

by

near

-si

mul

tane

ous

failu

res

in th

e ha

rd d

rives

, or f

ile s

erve

rs, i

n tw

o of

the

mac

hine

s. T

he s

erve

rs s

tore

the

softw

are

appl

icat

ions

for m

any

of th

e st

atio

n's

criti

cal f

unct

ions

. H

owev

er, a

sof

twar

e pr

oble

m h

ad n

ot b

een

rule

d ou

t. ht

tp://

ww

w.c

hron

.com

/con

tent

/inte

ract

ive/

spac

e/m

issi

ons/

sts-

100/

stor

ies/

2001

0430

.htm

l

OPR

E 63

6449

Som

e in

spec

tion

expe

rienc

esJP

L Ex

perie

nce

Aver

ages

per

insp

ectio

nM

ajor

def

ects

foun

d4

Min

or d

efec

ts fo

und

12Pa

ges

of m

ater

ial

38N

umbe

r of p

artic

ipan

ts5

Tota

l sta

ff ho

urs

28Ap

prox

imat

e sa

ving

s$2

5,00

0

Bell

Nor

ther

n R

esea

rch

Expe

rienc

e2.

5 m

illion

s LO

C o

ver 8

sof

twar

e re

leas

esD

etec

ted

37 d

efec

ts p

er 1

000

lines

of c

ode

Foun

d on

e de

fect

for e

ach

staf

f-hou

r inv

este

dD

isco

vere

d de

fect

s 2

to 4

tim

es fa

ster

than

test

ing

Foun

d 80

% o

f all

defe

cts.

OPR

E 63

6450

Softw

are

Test

ing

•Th

e pu

rpos

e of

test

ing

is to

dem

olis

h th

e so

ftwar

e th

at h

as ju

stbe

en c

ompl

eted

•Te

stin

g ca

nnot

dem

onst

rate

abs

ence

of d

efec

ts•

Test

ing

unco

vers

erro

rs o

nly

if yo

u ar

e w

illing

to d

etec

t the

m•

If a

new

ly d

evel

oped

sof

twar

e do

es n

ot h

ave

erro

rs th

en th

e so

ftwar

e w

as to

o tri

vial

to b

e de

velo

ped

•So

ftwar

e er

rors

follo

w P

aret

o Pr

inci

ple

–ab

out 8

0% o

f the

er

rors

occ

ur in

abo

ut 2

0% o

f the

cod

e•

Exha

ustiv

e te

stin

g is

impo

ssib

le•

A ty

pica

l sof

twar

e de

velo

pmen

t pro

ject

ear

mar

ks a

bout

25%

of

the

tota

l effo

rt fo

r tes

ting

OPR

E 63

6451

Test

ing

phas

es•

Unit T

est

–Tes

ts t

he code

of ea

ch u

nit d

evel

oped

by

a pro

gra

mm

er.

Usu

ally

done

by

the

pro

gra

mm

er

him

(her

)sel

f.•

Inte

gra

tion T

est

–Tes

ts t

he design

of

the

syst

em b

y te

stin

g t

he

module

le

vel in

terf

aces

. U

sual

ly d

one

by

the

per

son in

char

ge

of th

e co

rres

pondin

g s

ubsy

stem

•Val

idation T

est

–Tes

ts t

he requirements

of th

e sy

stem

–U

sual

ly d

one

by

the

ITG

, In

dep

enden

t Tes

t G

roup

•Acc

epta

nce

Tes

t–

Tes

ts t

he

entire

system

acco

rdin

g t

o p

re-s

pec

ifie

d

criter

ia.

Usu

ally

done

by

the

use

r

OPR

E 63

6452

Test

cas

e de

sign

•Bl

ack

box

test

ing

•Te

sts

the

func

tiona

l req

uire

men

ts o

f the

uni

t. Te

st c

ases

ar

e de

sign

ed k

eepi

ng in

min

d w

hat t

his

porti

on o

f the

so

ftwar

e w

as s

uppo

sed

to d

o–

Inco

rrect

or m

issi

ng fu

nctio

ns–

Inte

rface

erro

rs–

Erro

rs in

ext

erna

l dat

a ac

cess

–Pe

rform

ance

erro

r

•G

lass

box

test

ing

–Te

sts

the

cont

rol s

truct

ure

of th

e un

it•

Exec

ute

each

inde

pend

ent p

aths

•Ex

erci

se a

ll lo

gica

l dec

isio

ns•

Exec

ute

all l

oops

at t

heir

boun

darie

s•

Exer

cise

inte

rnal

dat

a st

ruct

ures

OPR

E 63

6453

Inte

grat

ion

test

ing

•A

syst

emat

ic te

chni

que

for c

onst

ruct

ing

the

prog

ram

stru

ctur

e w

hile

at t

he s

ame

time

cond

uctin

g te

sts

to u

ncov

er e

rrors

as

soci

ated

with

inte

rfaci

ng•

Top-

dow

n in

tegr

atio

n–

Mod

ules

are

inte

grat

ed b

y m

ovin

g do

wnw

ard

thro

ugh

the

cont

rol h

iera

rchy

–U

ses

stub

s to

repr

esen

t low

er le

vel m

odul

es•

Botto

m-u

p in

tegr

atio

n–

Low

leve

l mod

ules

are

inte

grat

ed fi

rst i

nto

clus

ters

and

the

clus

ters

are

inte

grat

ed b

y m

ovin

g up

the

cont

rol s

truct

ure

–U

ses

driv

ers

to re

pres

ent u

pper

leve

l mod

ules

•R

egre

ssio

n te

stin

g

OPR

E 63

6454

Stu

bs

and D

rive

rs

Stub

ASt

ub B

Stub

CSt

ub D

Dis

pla

y a

trac

e m

essa

ge

Dis

pla

y pas

sed

par

amet

er

Ret

urn

a v

alue

from

a t

able

or

exte

rnal

file

Do a

table

sea

rch f

or

the

input

and r

eturn

an

outp

ut

Driv

er A

Driv

er B

Driv

er C

Driv

er D

Invo

ke

subord

inate

Sen

d a

par

amet

erD

ispla

y a

par

amet

erA c

om

bin

atio

n

of

Drive

rs B

an

d C

S T U

B S D

R I V

E R S

OPR

E 63

6455

Crit

eria

for t

est c

ompl

etio

n•

How

do

we

know

we

have

test

ed e

noug

h?•

Use

som

e st

atis

tical

mod

el to

pre

dict

the

num

ber o

f de

fect

s af

ter t

he s

oftw

are

has

been

test

ed fo

r tun

its o

f tim

e•

Whe

n th

e nu

mbe

r of d

efec

ts fo

und

in e

ach

of la

st n

cons

ecut

ive

hour

s of

test

ing

fall

belo

w a

pre

-spe

cifie

d lim

it•

Whe

n a

pre-

spec

ified

per

cent

age

of p

lant

ed e

rrors

get

di

scov

ered

OPR

E 63

6456

Valid

atio

n Te

stin

g•

Answ

ers

the

ques

tion

‘Are

we

deve

lopi

ng th

e rig

ht p

rodu

ct?’

rath

er th

an‘A

re w

e de

velo

ping

the

prod

uct r

ight

?’•

Alph

a te

stin

g–

Con

duct

ed a

t dev

elop

er’s

site

by

a cu

stom

er. C

ondu

cted

in

a co

ntro

lled

envi

ronm

ent i

n a

natu

ral s

ettin

g w

ith th

e de

velo

per “

look

ing

over

the

shou

lder

” of t

he u

ser a

nd

reco

rdin

g er

rors

•Be

ta te

stin

g–

Con

duct

ed a

t one

or m

ore

cust

omer

’s s

ites

by th

e en

d-us

er

of th

e so

ftwar

e. It

is a

‘liv

e’ te

st o

f the

sof

twar

e in

an

envi

ronm

ent n

ot c

ontro

lled

by th

e de

velo

per

OPR

E 63

6457

Acce

ptan

ce T

estin

g•

It is

a c

ompl

ete

test

of t

he e

ntire

sys

tem

by

the

end-

user

ac

cord

ing

to p

re-d

eter

min

ed c

riter

ia

–“T

he s

oftw

are

will

run

cont

inuo

usly

for 4

8 ho

urs”

–“A

vera

ge q

uery

tim

e sh

ould

not

exc

eed

1.5

seco

nd w

orki

ng

on a

dat

abas

e of

siz

e no

t exc

eedi

ng 1

000

reco

rds”

–“P

rodu

ctio

n pl

anni

ng w

ill be

don

e us

ing

real

dat

a of

last

four

m

onth

s”•

Syst

em te

stin

g–

Rec

over

y te

stin

g–

Secu

rity

test

ing

–St

ress

test

ing

–Pe

rform

ance

test

ing

OPR

E 63

6458

When

to T

est

Man

y IT

pro

fess

iona

ls th

ink

of te

stin

g as

a s

tage

that

co

mes

nea

r the

end

of I

T pr

oduc

t dev

elop

men

tTe

stin

g sh

ould

be

done

dur

ing

alm

ost e

very

pha

seof

th

e IT

pro

duct

dev

elop

men

t life

cyc

le

OPR

E 63

6459

Tes

ting T

asks

in t

he

Soft

war

e D

evel

opm

ent

Life

Cyc

le

OPR

E 63

6460

Typ

es o

f Tes

ts

•A

unit

test

is d

one

to te

st e

ach

indi

vidu

al c

ompo

nent

(o

ften

a pr

ogra

m) t

o en

sure

it is

as

defe

ct fr

ee a

s po

ssib

le•

Inte

grat

ion

test

ing

occu

rs b

etw

een

unit

and

syst

em

test

ing

to te

st fu

nctio

nally

gro

uped

com

pone

nts

•Sy

stem

test

ing

test

s th

e en

tire

syst

em a

s on

e en

tity

•U

ser a

ccep

tanc

e te

stin

g is

an

inde

pend

ent t

est

perfo

rmed

by

the

end

user

prio

r to

acce

ptin

g th

e de

liver

ed s

yste

m

OPR

E 63

6461

Sta

tist

ical

Tools

for

test

dat

a an

alys

is

•Q

uant

itativ

e Te

chni

ques

that

sup

port

PDC

A cy

cle

•C

ontro

l (R

un) C

harts

•Pa

reto

Cha

rts•

Ishi

kaw

a D

iagr

ams

•H

isto

gram

s, S

catte

r Cha

rts, F

low

Cha

rts,

Che

cksh

eets

OPR

E 63

6462

Pare

to A

nal

ysis

•Pa

reto

ana

lysi

s in

volv

es id

entif

ying

the

vita

l few

co

ntrib

utor

s th

at a

ccou

nt fo

r the

mos

t qua

lity

prob

lem

s in

a s

yste

m•

Also

cal

led

the

80-2

0 ru

le, m

eani

ng th

at 8

0% o

f pr

oble

ms

are

ofte

n du

e to

20%

of t

he c

ause

s•

Pare

to d

iagr

ams

are

hist

ogra

ms

that

hel

p id

entif

y an

d pr

iorit

ize

prob

lem

are

as

OPR

E 63

6463

Pare

to A

nal

ysis

•Th

e Pa

reto

Prin

cipl

e (8

0/20

rule

)•

Put o

ur e

ffort

on m

ost s

igni

fican

t pro

blem

•C

olle

ct d

ata

usin

g ch

ecks

heet

s

OPR

E 63

6464

Pare

to E

xam

ple

Time Worked (Hours)

42.2

icon

gen

40.8

18.5

157

hud_

ipc

hudd

sply

N=1

23.5

Percent of Total

34%

67%

hud_

proc

hud_

boot

82%

96%

100%

OPR

E 63

6465

Sam

ple

Par

eto D

iagra

m

OPR

E 63

6466

Exer

cise

Par

t I

•Ta

ke y

our d

efec

t dat

a•

Gro

up b

y de

fect

cat

egor

y (n

ext s

lide)

•D

raw

a P

aret

o C

hart

of y

our d

efec

ts

OPR

E 63

6467

Sort

dat

a by

Err

or C

ateg

orie

s10

Doc

umen

tati

on p

robl

em20

Syn

tax

30 B

uild

40 A

ssig

nmen

t50

Int

erfa

ce60

Err

or H

andl

ing

70 D

ata

80 F

unct

ion

90 S

yste

m10

0 En

viro

nmen

t

OPR

E 63

6468

Root

Cau

se A

nal

ysis

by

Ishik

awa

Dia

gra

ms

•Al

so c

alle

d C

ause

-Effe

ct o

r fis

hbon

e D

iagr

am•

Hel

ps R

oot C

ause

Ana

lysi

s•

Don

’t tre

at s

ympt

oms!

Dig

out

the

root

cau

se--t

he

fact

oraf

fect

ing

qual

ity•

Use

s di

agra

m p

lus

brai

nsto

rmin

g•

4 M

’s: M

anpo

wer

, Mac

hine

s, M

etho

ds,M

ater

ials

•4

P’s:

Pol

icie

s, P

roce

dure

s, P

eopl

e, P

lant

OPR

E 63

6469

The

Dia

gra

m loca

tes

Cau

ses

OPR

E 63

6470

Exa

mple

of Is

hik

awa

in S

W

Icongen

use

d 3

4%

of al

l ef

fort

People

Proce

ssEquip

men

t

Poor

IDE

Inex

per

ience

d m

anag

er

Har

d t

o m

odify

Bad

des

ign

Inex

per

ience

d d

evel

oper

s

Mea

sure

men

t

Poor

chan

ge

trac

king

OPR

E 63

6471

OPR

E 63

6472

Exe

rcis

e Pa

rt I

I

•N

ow s

elec

t you

r mos

t crit

ical

def

ect c

ateg

ory

from

the

Pare

to A

naly

sis

•Fo

rmul

ate

a pr

oble

m s

ympt

om•

Dra

w Is

hika

wa

Dia

gram

•Se

lect

app

ropr

iate

cat

egor

ies

(M,E

,P)

•Br

ains

torm

cau

ses

(try

to g

o at

leas

t 2 d

eep

on a

few

)

OPR

E 63

6473

Your

Ishi

kaw

a

OPR

E 63

6474

Exe

rcis

e Pa

rt I

II

•N

ow a

naly

ze y

our I

shik

awa

diag

ram

to p

ick

the

mos

t lik

ely

caus

e•

Form

ulat

e a

shor

t pla

n (P

DC

A) to

add

ress

cor

rect

ing

the

caus

e•

Wha

t met

rics

(che

cksh

eet)

wou

ld y

ou s

elec

t to

dete

rmin

e w

heth

er th

is s

olut

ion

wor

ked?

OPR

E 63

6475

Contr

ol Char

ts

•To

impr

ove

a pr

oces

s, it

mus

t be

“in c

ontro

l”•

In c

ontro

l = re

peat

able

•H

ow d

o w

e kn

ow?

•C

olle

ct d

ata,

dra

w c

ontro

l cha

rt•

Plot

any

def

ects

that

you

can

cou

nt o

r mea

sure

(nex

t sl

ide)

•C

hart

show

s ch

ange

s ov

er ti

me

•Ea

ch o

ut-o

f-con

trol p

oint

requ

ires

inve

stig

atio

n

http

://w

ww

.sei

.cm

u.ed

u/pu

b/do

cum

ents

/92.

repo

rts/p

df/tr

22.9

2.pd

f

OPR

E 63

6476

Con

trol C

hart

(mor

e lik

e p

Cha

rt)

Rang

e of

Va

lues

Ave

rage

Upp

er C

ontr

ol L

imit

(UCL

)

Lowe

r Co

ntro

l Lim

it (L

CL)

Spec

ific

atio

n Li

mit

(Opt

iona

l)

OPR

E 63

6477

Con

trol C

hart

Valu

es

Spec

Lim

its

are

prov

ided

(nor

mal

ly a

s go

als

or s

tand

ards

)Ce

nter

line

= s

um(v

alue

s)/n

wher

e n

is s

ampl

e si

ze

UCL

= A

vg+

3*sq

rt(A

vg)

LCL

= A

vg-

3 *s

qrt(

Avg

) (if

impo

ssib

le v

alue

the

n ze

ro)

Proc

ess

in c

ontr

ol w

hen

all p

oint

s be

twee

n U

CLan

d LC

L. M

ay s

till

be p

oor

and

outs

ide

spec

, ju

st m

eans

tha

t it

is p

redi

ctab

le.

OPR

E 63

6478

Contr

ol Char

t D

ata

MO

DU

LE B

YD

EFEC

TS/K

LOC

15Te

am 6

b

10Te

am 6

40Te

am 4

25Te

am 3

OPR

E 63

6479

Sam

ple

Contr

ol Char

t

X

Defects per KLOC

050

X

X

X

Avg

=(15

+10+

40+2

5)/4

=25

UCL

=25+

3(SQ

RT(2

5))=

LCL=

25-3

(SQ

RT(2

5))=

UCL

LCL

SPEC

OPR

E 63

6480

Sam

ple

Qual

ity

Contr

ol Char

t

OPR

E 63

6481

A p C

har

t of D

efec

ts

http

://w

ww

.sei

.cm

u.ed

u/pu

b/do

cum

ents

/97.

repo

rts/p

df/9

7hb0

03.p

df

OPR

E 63

6482

http

://w

ww

.sei

.cm

u.ed

u/pu

b/do

cum

ents

/97.

repo

rts/p

df/9

7hb0

03.p

df

OPR

E 63

6483

Sta

tist

ical

Sam

plin

g a

nd S

tandar

d

Dev

iation

•St

atis

tical

sam

plin

g in

volv

es c

hoos

ing

part

of a

po

pula

tion

of in

tere

st fo

r ins

pect

ion

•Th

e si

ze o

f a s

ampl

e de

pend

s on

how

repr

esen

tativ

e yo

u w

ant t

he s

ampl

e to

be

•Sa

mpl

e si

ze fo

rmul

a:Sa

mpl

e si

ze =

.25

X (c

erta

inty

Fac

tor/a

ccep

tabl

e er

ror)2

OPR

E 63

6484

Com

monly

Use

d C

erta

inty

Fac

tors

Desir

ed C

erta

inty

Cert

aint

y Fa

ctor

95%

1.96

0

90%

1.64

5

80%

1.28

1

95%

cer

tain

ty: S

ampl

e si

ze =

0.2

5 X

(1.9

60/.0

5)2

= 38

490

% c

erta

inty

: Sam

ple

size

= 0

.25

X (1

.645

/.10)

2=

6880

% c

erta

inty

: Sam

ple

size

= 0

.25

X (1

.281

/.20)

2=

10

OPR

E 63

6485

Sta

ndar

d D

evia

tion

•St

anda

rd d

evia

tion

mea

sure

s ho

w m

uch

varia

tion

exis

ts in

a d

istri

butio

n of

dat

a•

A sm

all s

tand

ard

devi

atio

n m

eans

that

dat

a cl

uste

r cl

osel

y ar

ound

the

mid

dle

of a

dis

tribu

tion

and

ther

e is

littl

e va

riabi

lity

amon

g th

e da

ta•

A no

rmal

dis

tribu

tion

is a

bel

l-sha

ped

curv

e th

at is

sy

mm

etric

al a

bout

the

mea

n or

ave

rage

val

ue o

f a

popu

latio

n

OPR

E 63

6486

Norm

al D

istr

ibution a

nd S

tandar

d

Dev

iation

OPR

E 63

6487

Sig

ma

and N

o.

of

Def

ective

Units

Spec

ifica

tion

Ran

ge

(in +

/- Si

gmas

) Pe

rcen

t of

Popu

latio

n W

ithin

Ran

ge

Def

ectiv

e U

nits

Pe

r B

illio

n

1 68

.27

317,

300,

000

2 95

.45

45,4

00,0

00

3 99

.73

2,70

0,00

0

4 99

.993

7 63

,000

5 99

.999

943

57

6 99

.999

9998

2

No

te:

“S

ix s

igm

a”

oft

en

refe

rs t

o +

/-3

sig

ma,

mean

ing

2.7

mil

lio

n d

efe

cts

per

bil

lio

n u

nit

s p

rod

uce

d,

or

2.7

defe

cts

per

mil

lio

n.

OPR

E 63

6488

Qual

ity

Contr

ol Char

ts,

Six

Sig

ma,

and

the

Sev

en R

un R

ule

•A

cont

rol c

hart

is a

gra

phic

dis

play

of d

ata

that

illu

stra

tes

the

resu

lts o

f a p

roce

ss o

ver t

ime.

It h

elps

pre

vent

de

fect

s an

d al

low

s yo

u to

det

erm

ine

whe

ther

a p

roce

ss

is in

con

trol o

r out

of c

ontro

l•

Ope

ratin

g at

a h

ighe

r sig

ma

valu

e, li

ke 6

sig

ma,

mea

ns

the

prod

uct t

oler

ance

or c

ontro

l lim

its h

ave

less

va

riabi

lity

•Th

e se

ven

run

rule

sta

tes

that

if s

even

dat

a po

ints

in a

ro

w a

re a

ll be

low

the

mea

n, a

bove

,the

mea

n, o

r in

crea

sing

or d

ecre

asin

g, th

en th

e pr

oces

s ne

eds

to b

e ex

amin

ed fo

r non

-rand

om p

robl

ems

OPR

E 63

6489

Red

ucin

g D

efec

ts w

ith S

ix S

igm

a

OPR

E 63

6490

Sum

mar

y of Tools

Use

•Pr

oces

ses

need

to b

e co

ntin

ually

impr

oved

. To

ols

mak

e yo

ur e

ffort

effe

ctiv

e an

d ef

ficie

nt•

Don

’t fo

llow

the

Dilb

ert M

odel

•Th

e pe

ople

who

do

the

wor

k us

ually

kno

w b

est h

ow

to im

prov

e it

•C

olle

ct d

ata

to s

uppo

rt de

cisi

ons

•U

se q

uant

itativ

e m

etho

ds w

hen

poss

ible

to s

uppo

rt de

cisi

ons

•Pl

an th

e fli

ght,

Fly

the

plan

OPR

E 63

6491

Qual

ity

Man

agem

ent

of

IT P

roje

cts

•M

any

peop

le jo

ke a

bout

the

poor

qua

lity

of IT

pr

oduc

ts

•Pe

ople

see

m to

acc

ept s

yste

ms

bein

g do

wn

occa

sion

ally

or n

eedi

ng to

rebo

ot th

eir P

Cs

•Th

ere

are

man

y ex

ampl

es in

the

new

s ab

out q

ualit

y pr

oble

ms

rela

ted

to IT

(Wha

t Wen

t Wro

ng??

)•

Qua

lity

is c

ritic

al in

eve

ry IT

pro

ject

man

agem

ent

also

!

OPR

E 63

6492

What

Is

Pro

ject

Qual

ity

Man

agem

ent?

•Th

e In

tern

atio

nal O

rgan

izat

ion

for S

tand

ardi

zatio

n (IS

O)

defin

es q

ualit

y as

the

tota

lity

of c

hara

cter

istic

s of

an

entit

y th

at b

ear o

n its

abi

lity

to s

atis

fy s

tate

dor

impl

ied

need

s•

Oth

er e

xper

ts d

efin

e qu

ality

bas

ed o

n–

conf

orm

ance

to re

quire

men

ts: m

eetin

g w

ritte

n sp

ecifi

catio

ns–

fitne

ss fo

r use

: ens

urin

g a

prod

uct c

an b

e us

ed a

s it

was

inte

nded

OPR

E 63

6493

Proje

ct Q

ual

ity

Man

agem

ent

Proce

sses

•Q

ualit

y pl

anni

ng: i

dent

ifyin

g w

hich

qua

lity

stan

dard

s ar

e re

leva

nt to

the

proj

ect a

nd h

ow to

sat

isfy

them

•Q

ualit

y as

sura

nce:

eva

luat

ing

over

all p

roje

ct

perfo

rman

ce to

ens

ure

the

proj

ect w

ill sa

tisfy

the

rele

vant

qua

lity

stan

dard

s•

Qua

lity

cont

rol:

mon

itorin

g sp

ecifi

c pr

ojec

t res

ults

to

ensu

re th

at th

ey c

ompl

y w

ith th

e re

leva

nt q

ualit

y st

anda

rds

whi

le id

entif

ying

way

s to

impr

ove

over

all

qual

ity

OPR

E 63

6494

Gan

tt C

hart

for B

uild

ing

Test

ing

into

a

Syst

ems

Dev

elop

men

t Pro

ject

Pla

n

OPR

E 63

6495

Impro

ving I

nfo

rmat

ion T

echnolo

gy

Proje

ct Q

ual

ity

•Se

vera

l sug

gest

ions

for i

mpr

ovin

g qu

ality

for I

T pr

ojec

ts

incl

ude

–Le

ader

ship

that

pro

mot

es q

ualit

y–

Und

erst

andi

ng th

e co

st o

f qua

lity

–Fo

cusi

ng o

n or

gani

zatio

nal i

nflu

ence

s an

d w

orkp

lace

fa

ctor

s th

at a

ffect

qua

lity

–Fo

llow

ing

mat

urity

mod

els

to im

prov

e qu

ality

OPR

E 63

6496

Org

aniz

atio

nal

Influen

ces,

Work

pla

ce

Fact

ors

, an

d Q

ual

ity

•St

udy

by D

eMar

co a

nd L

iste

r sho

wed

that

org

aniz

atio

nal

issu

es h

ad a

muc

h gr

eate

r inf

luen

ce o

n pr

ogra

mm

er

prod

uctiv

ity th

an th

e te

chni

cal e

nviro

nmen

t or

prog

ram

min

g la

ngua

ges

•Pr

ogra

mm

er p

rodu

ctiv

ity v

arie

d by

a fa

ctor

of o

ne to

ten

acro

ss o

rgan

izat

ions

, but

onl

y by

21%

with

in th

e sa

me

orga

niza

tion

•St

udy

foun

d no

cor

rela

tion

betw

een

prod

uctiv

ity a

nd

prog

ram

min

g la

ngua

ge, y

ears

of e

xper

ienc

e, o

r sal

ary

•A

dedi

cate

d w

orks

pace

and

a qu

iet w

ork

envi

ronm

ent

wer

e ke

y fa

ctor

s to

impr

ovin

g pr

ogra

mm

er p

rodu

ctiv

ity

OPR

E 63

6497

Good a

nd B

ad S

W p

roce

sses

:

“Mat

urity

Model

s”

•M

atur

ity m

odel

s ar

e fra

mew

orks

for h

elpi

ng o

rgan

izat

ion

impr

ove

thei

r pro

cess

es a

nd s

yste

ms

–So

ftwar

e Q

ualit

y Fu

nctio

n D

eplo

ymen

t Mod

el fo

cuse

s on

de

finin

g us

er re

quire

men

ts a

nd p

lann

ing

softw

are

proj

ects

–Th

e So

ftwar

e En

gine

erin

g In

stitu

te’s

Cap

abilit

y M

atur

ity M

odel

(C

MM

) pro

vide

s a

gene

ric p

ath

to p

roce

ss im

prov

emen

t for

so

ftwar

e de

velo

pmen

t–

Seve

ral g

roup

s ar

e w

orki

ng o

n pr

ojec

t man

agem

ent m

atur

ity

mod

els

http

://w

ww

.sts

c.hi

ll.af

.mil/

cros

stal

k/19

99/m

ay/p

utm

an.a

sp

OPR

E 63

6498

SW

Pro

ject

Man

agem

ent

Mat

urity

Model

s1.

Ad-

Hoc

: The

pro

ject

man

agem

ent p

roce

ss is

des

crib

ed a

s di

sorg

aniz

ed,

and

occa

sion

ally

eve

n ch

aotic

. The

org

aniz

atio

n ha

s no

t def

ined

sys

tem

s an

d pr

oces

ses,

and

pro

ject

suc

cess

dep

ends

on

indi

vidu

al e

ffort.

Ther

e ar

e ch

roni

c co

st a

nd s

ched

ule

prob

lem

s.2.

Abb

revi

ated

: The

re a

re s

ome

proj

ect m

anag

emen

t pro

cess

es a

nd

syst

ems

in p

lace

to tr

ack

cost

, sch

edul

e, a

nd s

cope

. Pro

ject

suc

cess

is

larg

ely

unpr

edic

tabl

e an

d co

st a

nd s

ched

ule

prob

lem

s ar

e co

mm

on.

3. O

rgan

ized

: The

re a

re s

tand

ardi

zed,

doc

umen

ted

proj

ect m

anag

emen

t pr

oces

ses

and

syst

ems

that

are

inte

grat

ed in

to th

e re

st o

f the

or

gani

zatio

n. P

roje

ct s

ucce

ss is

mor

e pr

edic

tabl

e, a

nd c

ost a

nd s

ched

ule

perfo

rman

ce is

impr

oved

.4.

Man

aged

: Man

agem

ent c

olle

cts

and

uses

det

aile

d m

easu

res

of th

e ef

fect

iven

ess

of p

roje

ct m

anag

emen

t. Pr

ojec

t suc

cess

is m

ore

unifo

rm,

and

cost

and

sch

edul

e pe

rform

ance

con

form

s to

pla

n.5.

Ada

ptiv

e: F

eedb

ack

from

the

proj

ect m

anag

emen

t pro

cess

and

from

pi

lotin

g in

nova

tive

idea

s an

d te

chno

logi

es e

nabl

es c

ontin

uous

im

prov

emen

t. Pr

ojec

t suc

cess

is th

e no

rm, a

nd c

ost a

nd s

ched

ule

perfo

rman

ce is

con

tinuo

usly

impr

ovin

g.