How to graph, analyze and compare sets of repair data

How to Graph, Analyze and Compare Sets of 

Repair DataWayne Nelson

©2011 ASQ & Presentation Wayne NelsonPresented live on Jun 09th, 2011

http://reliabilitycalendar.org/The_Reliability_Calendar/Webinars_‐_English/Webinars_‐_English.html

ASQ Reliability Division English Webinar SeriesOne of the monthly webinars 

on topics of interest to reliability engineers.

To view recorded webinar (available to ASQ Reliability Division members only) visit asq.org/reliability

To sign up for the free and available to anyone live webinars visit reliabilitycalendar.org and select English Webinars to find links to register for upcoming events

http://reliabilitycalendar.org/The_Reliability_Calendar/Webinars_‐_English/Webinars_‐_English.html

1

Tutorial for RAMS 2011. Copyright (C) 2011 Wayne Nelson.

HOW TO GRAPH, ANALYZE, AND COMPARE SETS OF REPAIR DATA

Wayne Nelson, consultant, Schenectady, NY, WNconsult@aol.com

PURPOSE: To survey new nonparametric models, analyses, and informative plots for recurrent events data with associated values (costs, time in hospital, running hours, and other quantities). Previous theory (often parametric, e.g., NHPP) handles just counts of recurrent events. 7.`6 '10

2

OVERVIEW

• RECURRENCE DATA AND INFORMATION SOUGHT • NONPARAMETRIC POPULATION MODEL • MCF -- MEAN CUMULATIVE (INTENSITY) FUNCTION • RECURRENCE RATE FOR COUNT DATA • MCF ESTIMATE AND CONFIDENCE LIMITS ● MCF ESTIMATE FOR COST • COMPARISON OF DATA SETS • ASSUMPTIONS • SOFTWARE • EXTENSIONS • NEEDED WORK • CONCLUDING REMARKS ● MORE APPLICATIONS

3

TYPICAL EXACT AGE DATA Automatic Transmission Repair Data (+ obs'd miles)

CAR M I L E A G E . CAR M I L E A G E

We use age (or usage) of each unit rather than calendar date. Multiple censoring times are typical.

4

Display of Automatic Transmission Repair Data

0 5 10 15 20 25 30 CAR +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ THOUS. MILES 024 | X | 026 X | 027 X X | 029 |X | 031 | | 032 | | 034 | X | 035 | | 098 | | 107 | X | 108 | | 109 | | 110 | | 111 | | 112 | | 113 | | 114 | | 115 | | 116 | | 117 | | 118 | | 119 | | 120 | | 121 X | 122 | | 123 | | 124 | | 125 | | 126 | | 129 | | 130 | | 131 | | 132 | X | 133 | X | +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ THOUS. MILES 0 5 10 15 20 25 30

5

Information Sought • Average number of repairs per transmission

at 24,000 test miles ( x 5.5 = 132,000 customer miles = design life).

• How do automatic and manual transmissions

compare? • The behavior of the repair rate (increasing or

decreasing?).

6

UNIT MODEL: Each unit has an uncensored Cumulative History Function for the cumulative number of events.

4 - Cum. No.

3 -

2 -

1 -

0 -----+x-x-+-x--+----+----+----+ THOUSAND MILES

7

Cumulative History Function for "cost", a new advance:

400 - Cum.$ Cost

300 -

200 -

100 -

0 -----+----+----+----+----+----+----+----+ 0 1000 2000 3000 4000

A G E I N D A Y S

Costs (or other values) may be negative, a new advance, e.g., scrap value and bank account withdrawal.

8

NONPARAMETRIC POPULATION MODEL consists of all uncensored cumulative history functions. No process assumed.

The population Mean Cumulative Function (MCF) denoted M(t) (usually Λ(t) for counts) contains most of the sought information.

9

Recurrence Rate for the number of recurrences per population unit m(t) ≡ dM(t)/dt is often of interest. m(t) is the mean number of transmission repairs per car per 1000 miles at (mile)age t. λ(t) is the notation for NHPPs.

Some wrongly call m(t) the “failure rate” and confuse it with the hazard rate of a life distribution.

Using multivariate distributions of times to and between events is complicated and less informative.

For cost, m(t) is the mean cost rate (average $ per month per population unit).

10

PLOT OF NONPARAMETRIC MCF ESTIMATE M*(t) & CONF. LIMITS Transmissions (Nelson 1988, 1995, 2003)

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.90

1.0

M

C F

0 5 10 15 20 25 30

1 0 0 0 M i l e s Decreasing repair rate. Here M*(24,000) = 0.31 repairs/car.

11

Calculate the MCF Estimate M*(t)

(1) (2) (3) (4)

(1) (2) (3) (4) Mileage No. r mean MCF Mileage No. r mean MCF

obs'd no. 1/r obs'd no. 1/r 28 34 1/34=0.03 0.03 20425+ 21 48 34 1/34=0.03 0.06 20890+ 20 375 34 0.03 0.09 20997+ 19 530 34 0.03 0.12 21133+ 18 1388 34 0.03 0.15 21144+ 17 1440 34 0.03 0.18 21237+ 16 5094 34 0.03 0.21 21401+ 15 7068 34 0.03 0.24 21585+ 14 8250 34 1/34=0.03 0.27 21691+ 13 13809+ 33 21762+ 12 14235+ 32 21876+ 11 14281+ 31 21888+ 10 17844+ 30 21974+ 9 17955+ 29 22149+ 8 18228+ 28 22486+ 7 18676+ 27 22637+ 6 19175+ 26 22854+ 5 19250 26 1/26=0.04 0.31 23520+ 4 19321+ 25 24177+ 3 19403+ 24 25660+ 2 19507+ 23 26744+ 1 19607+ 22 29834+ 0

12

Bladder Tumor Treatment MCFs (Placebo and Thiotepa) Compare treatments; understand the course of the disease and when to schedule exams. SAS plots.

13

Cost Data and the MCF Estimate Days Cost$ No. r Mean Cost MCF for Rate m MCF

at risk Cost / r Cost = 1/r for No. 141 44.20 119 44.20 /119= 0.37 0.37 0.008 0.008 252 + 118 288 + 117 358 + 116 365 + 115 376 + 114 376 + 113 381 + 112 444 + 111 651 + 110 699 + 109 820 + 108 831 + 107 843 110.20 107 110.20 /107= 1.03 1.40 0.009 0.018 880 + 106 966 + 105 973 + 104 1057 + 103 1170 + 102 1200 + 101 1232 + 100 1269 130.20 100 130.20 /100= 1.30 2.70 0.010 0.028 1355 + 99 1381 150.40 99 150.40 /99 = 1.52 4.22 0.010 0.038 1471 113.40 99 113.40 /99 = 1.15 5.37 0.010 0.048 1567 151.90 99 151.90 /99 = 1.53 6.90 0.010 0.058 1642 191.20 99 191.20 /99 = 1.93 8.83 0.010 0.068

14

MCF for Cost for Fan Motor Repairs (Excel Plots)

MCF for Number of Fan Motor Repairs

0.00

0.05

0.10

0.15

0.20

0.25

0.30

0 1000 2000 3000 4000D  A  Y  S

MCF#

0

10

20

30

40

50

60

0 1000 2000 3000 4000 D A Y S

MCF$

15

MIX OF EVENTS Subway Car Traction Motors. Design Failures with and without Modes A, B, C: MAll(t) = M1(t) + ⋅⋅⋅ + MK(t). Modes need not be statistically independent. Excel plot.

0

10

20

30

40

50

0 12 24 36MONTHS

MCF%

All Design Modes

ABC

Design w/o ABC

16

COMPARISON OF MCFs OF DATA SETS

Do sample MCFs differ statistically significantly?

0.0

0.2

0.4

0.6

0.8

1.0

0 20 40 60 80 100A G E

MCF

(A)

0.0

0.2

0.4

0.6

0.8

1.0

0 20 40 60 80 100A G E

MCF

(B)

17

Manual Transmission Repair Data (+ obs'd miles) CAR _____M I L E A G E______ 025 27099+ 028 21999+ 030 11891 27583+ 097 19966+ 099 26146+ 100 3648 13957 23193+ 101 19823+ 102 2890 22707+ 103 2714 19275+ 104 19803+ 105 19630+ 106 22056+ 127 22940+ 128 3240 7690 18965+

18

Automatic Transmission Manual Transmission

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

MCF

0 5000 10000 15000 20000 25000 30000

Mileage

0.0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1.0

MCF

0 5000 10000 15000 20000 25000 30000

Mileage ReliaSoft RDA plots.

19

POINTWISE COMPARISON AT A SINGLE AGE t Var[M*1(t)−M*2(t)] = Var[M*1(t)] + Var[M*2(t)]

-0.6

-0.5

-0.4

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

0.4

MCF

0 5000 10000 15000 20000 25000 30000

Mileage Limits enclose 0 => no convincing difference. ReliaSoft plot.

(Automatic − Manual)

20

Herpes Episodes -- Comparison of Episodic and Suppressive Valtrex Treatments

Provided by Richard Cook with permission of GSK. S-Plus plot.

21

Amazon.com Orders MCF for Two Promotions , SAS plot

Promo. 1

Promo. 2

22

Babies Born to Statisticians (♦ Men, □ Women). Excel plot

0

0.2

0.4

0.6

0.8

1

1.2

1.4

1.6

0 10 20 30 40 50 60A G E (Y E A R S)

MCF

23

SOFTWARE for calculating and plotting the MCF estimate and limits and the difference of two sample MCFs for count and "cost" data:

• SAS Reliability Procedure.

• SAS JMP Package.

• Meeker's (1999) SPLIDA routines for S-PLUS.

• ReliaSoft RDA Utility (Repair Data Analysis). This has naïve (too short) confidence limits.

● SuperSmith Visual "Nelson Recurrent Event Plot."

• Nelson & Doganaksoy (1989) Fortran PC program.

● Minitab "Nonparametric Growth Curves".

24

ASSUMPTIONS FOR THE MCF ESTIMATE M*(t): 1> Simple random sample from the population. 2> Random (uninformative) censoring. 3> M(t) is finite. Clearly so for finite populations. Then the nonparametric estimator M*(t) is unbiased. ASSUMPTIONS FOR APPROX. CONF. LIMITS: 4> M*(t) is approximately normally distributed. 5> Variances and covariances in Var[M*(t)] are finite. 6> The population is infinite, at least 10× the sample size. NOT ASSUMED (usually false in practice) • Counting process such as NHPP, renewal, parametric, etc. • Independent increments, a common dubious simplifying

assumption for counts.

25

AVAILABLE EXTENSIONS • Continuous cumulative history functions, e.g., - cumulative energy output of a power plant, - cumulative up-time of locomotives (availability). • A mixture of types of events (e.g. failure modes). • Predictions of future numbers and costs of recurrences. • Estimates, plots, and conf. limits for interval age data. • Other sampling plans (stratified, cluster, etc.). • Left censoring and gaps in histories. • Multivariate event values (cost and downtime). • Regression models (Cox proportional hazards, etc.). • Informative censoring (frailties). • Parametric models, Rigdon & Basu (2000).

26

NEEDED WORK • A hypothesis test for independent increments of a NHPP. • Prediction limits for a future number or cost of recurrences. • Confidence limits and software for data with left censoring and gaps. • Theory and commercial software for comparison of entire MCFs. - Cook, R.J., Lawless, J.F., and Nadeau, C. (1996), "Robust Tests

for Treatment Comparisons Based on Recurrent Event Responses, Biometrics 52, 557-571.

• Methodology for terminated histories. • Better confidence limits for interval age data. • Efficient computation of confidence limits for large data sets.

Current computations are too intensive. • More regression models (Cox model is often poor) for count and

cost/value data. • Better parametric models (without independent increments).

27

CONCLUDING REMARKS

These new nonparametric methods, plots, and software for cost or other values of recurrent events are useful for many applications. Extensions of the methods and corresponding software are needed to handle more complicated applications.

28

REFERENCES Cook, R.J. and Lawless, J.F. (2007), The Statistical Analysis

of Recurrent Events, Springer, New York. Nelson, Wayne (1988), "Graphical Analysis of System

Repair Data," J. of Quality Technology 20, 24-35. Nelson, Wayne (1995), "Confidence Limits for Recurrence

Data -- Applied to Cost or Number of Product Repairs," Technometrics 37, 147-157.

Nelson, Wayne B. (2003), Recurrent Events Data Analysis for Product Repairs, Disease Recurrences, and Other Applications, SIAM, Philadelphia, ASA/SIAM. www.siam.org/books/sa10/.

Rigdon, S.E., and Basu, A.P. (2000), Statistical Methods for the Reliability of Repairable Systems, Wiley, New York.

29

MORE APPLICATIONS Bladder Tumor Treatment MCFs (Placebo & Thiotepa, SAS plots)

30

MCF difference (Placebo−Thiotepa) & 95% limits, SAS plot.

31

Naval Turbines MCF Censoring Ages 1211 2 1 2 1 1 11 1 1 1 11 11 ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ - - 1 20- 1 - 21 - 3 - 4 - 2 11 15- 31 - 1121 - 5 MCF - 12 2 - 42 10- 221 21 - 9 - 1 234 - 21332 - 32 43 5- 23252 1 - 1F1 - F3 - 3H - L 0- K ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ ׀ 0 5 10 15 20 25 THOUSANDS OF HOURS

32

Defrost Control MCF (data grouped by month, SAS plot)

33

Defrost Control Log-Log Plot (Excel plot)

0.1

1

10

100

1000

1 10 100 1000MONTHS

MCF%

2 5 20 50 200

34

Blood Analyzer Burn-in -- Sample MCF

20

MCF

35

Cumulative Hours in the Workforce

800 - Cum. Hours Worked 600 - 400 - 200 -

0 +----+----+----+----+----+----+ 0 10 20 30 W E E K S

36

Unemployment Contributions and Payments

Cum. $ Contributed 200 - 0 - -200 - -400 -

0 +----+----+----+----+----+----+ 0 10 20 30 W E E K S

37

NEGATIVE VALUES OF EVENTS (BANK ACCT.)

3000 - $ in Acct 2000 - 1000 - 0 -----+----+----+----+----+----+

0 200 400 600 800 1000 1200 D A Y S

38

Doses of a Concomitant Medication under Two Treatments Chris Barker (2009), "Exploratory method for summarizing concommitant

medication data – the mean cumulative function," Pharmaceutical Statistics.

39

Difference of the Two MCFs

40

Repairs of Large Power Transformers (Left Censored) Correct MCF

0

50

100

150

200

250

0 5 10 15 20 25 30 35 40 45YEARS

MCF%

Essentially constant repair rate (4.8% per year) over all vintages.