IEEE TRANSACTIONS ON RELIABILITY, VOL. R-25, NO. 1, APRIL 1976 EDITORIALS Don't Use Hypothesis Tests The theory and application of hypothesis tests is included in the statistics part of quality-control and reliability courses. Engineers and budding applied-statisticians learn the nice clean theory and figure that there must be many useful applications of it-otherwise their time would not have been wasted on it. Unfortunately, their time was wasted. The main application of hypothesis tests in quality-control is accept/reject tests for incoming inspection, inter-process inspection, final inspection, etc. Reliability and development engineers are taught to use hypothesis tests to decide on the better of two methods and to decide whether reliability goals have been met. The main difficulty with an hypothesis test is that it is a point estimate-it gives no idea of the uncertainty involved. An Operating Characteristic is used to show the dis- criminating ability of the test, but it doesn't give the uncertainty in a straightforward way. Rarely will a decision be made solely on the basis of the hypothesis test. There will be many important factors in the decision which were not part of the hypothesis. Decision theory in general suffers from this same disease. The purpose of running a test is to be smarter than you were before the VVh at Do the Data Tell Youu? test-not to make a decision. Eventually a decision will be made by the now smarter person-but lots of 'smarts' are An argument against the routine use of Bayes techniques combined into that decision, not merely the results of an for arriving at a degree-of-belief after an experiment (viz, the hypothesis test. (That's why there are material review boards 'posterior') is that you don't get a chance to see what the data and why some inspectors are wrongly encouraged not to be tell you by themselves. The likelihood function in Bayes for- strict on 'unimportant' matters.) mula is just that: what the data tell you by themselves. It is straightforward to use the test information to make Why not make a formal approach to interval estimation by some kind of interval estimate about the parameter of interest using the likelihood function? If the relative-likelihood func- (eg, fraction defective of the lot). That way, you will be tion is used (actual-likelihood divided by maximum-likelihood), smarter after the test, not merely more arbitrary. anyone can see the relative-likelihood of getting the data for a -R.A.E. particular parameter value (or values of a set of parameters). If, for example, the scale (oa) and shape (3) parameters of a Weibull distribution are unknown, one could draw contour plots (for fixed relative-likelihoods) in the (ct, i) plane. Using relative-likelihood means that many factors will 'cancel out' of the likelihood expression, leaving it much easier to gen- erate and use. The meaning of relative-likelihood will be much easier for engineers to grasp than is s-confidence. The likelihood statements are just 'what if' statements. Probably the theory for this is already well developed in the dusty pages of statistics journals (no doubt by Fisher himself); if so let's get it into the hands of engineers to use. If not-here's a bunch of ready-made thesis topics for budding applied-statisticians. -R.A.E.

What Do the Data Tell You?

Download PDF Report

View
216
Download
0

Embed Size (px)

Citation preview

IEEE TRANSACTIONS ON RELIABILITY, VOL. R-25, NO. 1, APRIL 1976

EDITORIALS

Don't Use Hypothesis TestsThe theory and application of hypothesis tests is included

in the statistics part of quality-control and reliability courses.Engineers and budding applied-statisticians learn the nice cleantheory and figure that there must be many useful applicationsof it-otherwise their time would not have been wasted on it.Unfortunately, their time was wasted.

The main application of hypothesis tests in quality-controlis accept/reject tests for incoming inspection, inter-processinspection, final inspection, etc. Reliability and developmentengineers are taught to use hypothesis tests to decide on thebetter of two methods and to decide whether reliability goalshave been met. The main difficulty with an hypothesis test isthat it is a point estimate-it gives no idea of the uncertaintyinvolved. An Operating Characteristic is used to show the dis-criminating ability of the test, but it doesn't give the uncer-tainty in a straightforward way.

Rarely will a decision be made solely on the basis of thehypothesis test. There will be many important factors in thedecision which were not part of the hypothesis. Decisiontheory in general suffers from this same disease. The purposeof running a test is to be smarter than you were before the VVh at Do the Data Tell Youu?test-not to make a decision. Eventually a decision will bemade by the now smarter person-but lots of 'smarts' are An argument against the routine use of Bayes techniquescombined into that decision, not merely the results of an for arriving at a degree-of-belief after an experiment (viz, thehypothesis test. (That's why there are material review boards 'posterior') is that you don't get a chance to see what the dataand why some inspectors are wrongly encouraged not to be tell you by themselves. The likelihood function in Bayes for-strict on 'unimportant' matters.) mula is just that: what the data tell you by themselves.

It is straightforward to use the test information to make Why not make a formal approach to interval estimation bysome kind of interval estimate about the parameter of interest using the likelihood function? If the relative-likelihood func-(eg, fraction defective of the lot). That way, you will be tion is used (actual-likelihood divided by maximum-likelihood),smarter after the test, not merely more arbitrary. anyone can see the relative-likelihood of getting the data for a

-R.A.E. particular parameter value (or values of a set of parameters).If, for example, the scale (oa) and shape (3) parameters of aWeibull distribution are unknown, one could draw contourplots (for fixed relative-likelihoods) in the (ct, i) plane.

Using relative-likelihood means that many factors will 'cancelout' of the likelihood expression, leaving it much easier to gen-erate and use. The meaning of relative-likelihood will be mucheasier for engineers to grasp than is s-confidence. The likeli-hood statements are just 'what if' statements. Probably thetheory for this is already well developed in the dusty pages ofstatistics journals (no doubt by Fisher himself); if so let's getit into the hands of engineers to use. If not-here's a bunch ofready-made thesis topics for budding applied-statisticians.

-R.A.E.