Section 10.2: Errors in Hypothesis Testing

• Test Procedure – the method we use to determine whether H0 should be rejected.

• Type 1 Error: the error of rejecting H0 when H0 is true

• Type 2 Error: the error of failing to reject H0 when H0 is false

Example

• The U.S. Department of Transportation reported that during a recent period, 77% of all domestic passenger flights arrived on time (meaning within 15 minutes of the scheduled arrival). Suppose that an airline with a poor on-time record decides to offer its employees a bonus if, in an upcoming month, the airline’s proportion of on-time flights exceeds the overall industry rate of 0.77. Let π be the true proportion of the airline’s flights that are on time during the month of interest.

• A random sample of flights might be selected and used as a basis for choosing between

H0: π = .77 and Ha: π > .77

In this context, a Type I error (rejecting a true H0) results in the airline rewarding its employees when in fact their true proportion of on-time flights did not exceed .77. A Type II error (not rejecting a false H0) results in the airline employees not receiving a reward that in fact they deserved.

• The probability of a Type I error is denoted by α and is called the level of significance of the test. Thus, a test with α = .01 is said to have a level of significance of .01 or to be a level .01 test.

• The probability of a Type II error is denoted by β.

Example

• Women with ovarian cancer are usually not diagnosed until the disease is in an advanced stage, when it is most difficult to treat. A new blood test has been developed that appears to be able to identify ovarian cancer at its earliest stages. In a report issued by the National Cancer Institute and the Food and Drug Administration the following information from a preliminary evaluation of the blood test was given:

• The test was given to 50 women known to have ovarian cancer, and it correctly identified all of them as having cancer.

• The test was given to 66 women known not to have ovarian cancer, and it correctly identified 63 of these 66 as being cancer free.

• We can think of using this blood test to choose between two hypotheses:

H0: woman has ovarian cancer

Ha: woman does not have ovarian cancer

• In this situation, believing that a woman with ovarian cancer is cancer free would be a Type I error – rejecting the hypothesis of ovarian cancer when it is, in fact, true.

• Believing that a woman who is actually cancer free does have ovarian cancer is a Type II error – not rejecting the null hypothesis when it is, in fact, false.

• We can estimate the error of probabilities.

• The probability of a Type I error α is approximately 0/50 = 0.

• The probability of a Type II error β is approximately 3/66 = .046

• After assessing the consequences of Type I and Type II errors, identify the largest α that is tolerable for the problem. Then employ a test procedure that uses this maximum acceptable value – rather than anything smaller – as the level of significance (because using a smaller α increases β). In other words, don’t make α smaller than it needs to be.

Example

• The Associated Press reported that the Environmental Protection Agency had warned 819 communities that their tap water contained too much lead. Drinking water is considered unsafe if the mean concentration of lead is 15 ppb (parts per billion) or greater. The EPA requires the cited communities to take corrective actions and to monitor lead levels.

• With μ denoting the mean concentration of lead, a cited community could test

H0: μ = 15 versus Ha: μ < 15The null hypothesis states that the mean lead

concentration is excessive by EPA standards.The alternative hypothesis states that the mean

lead concentration is at an acceptable level and that the water system meets EPA standards for lead.

• In this context, a Type I error leads to the conclusion that a water source meets EPA standards for lead when, in fact, it does not.

• Possible consequences of this type of error include health risks associated with excessive lead consumption (ex. Increased blood pressure, hearing loss, and, in severe cases, anemia and kidney damage)

• A Type II error is to conclude that the water does not meet EPA standards for lead when, in fact, it actually does.

• Possible consequences of a Type II error include elimination of a community water source. Because a Type I error might result in potentially serious public health risks, a small value of α such as .01 could be selected. This could however, increase the risk of a Type II error.