Section 9.1(re-visited)Section 9.1(re-visited)

Making Sense of Statistical Making Sense of Statistical SignificanceSignificance

Inference as DecisionInference as Decision

Warm-upWarm-up

The one-sample t statistic for testing HThe one-sample t statistic for testing H00: : μμ = 0 = 0

and and HHaa: : μμ > 0 from a sample of n = 15 > 0 from a sample of n = 15

observations has the value t = 1.82observations has the value t = 1.82 What are the degrees of freedom for this statistic?What are the degrees of freedom for this statistic? Between what two values does the P-value of the test Between what two values does the P-value of the test

fall?fall? Is the value t = 1.82 significant at the 5% level? Is it Is the value t = 1.82 significant at the 5% level? Is it

significant at the 1% level?significant at the 1% level?

Practical Applications Practical Applications In practice, statistical tests are used for In practice, statistical tests are used for

marketing, research, and the marketing, research, and the pharmaceutical industry.pharmaceutical industry.

The decisions we make as statisticians The decisions we make as statisticians must have must have practical significancepractical significance. This . This means that it must be worthwhile to use means that it must be worthwhile to use the information we find significant.the information we find significant.

Points to Keep in MindPoints to Keep in Mind

If you are going to make a decision based If you are going to make a decision based on a statistical test, choose on a statistical test, choose αα in advance. in advance.

When choosing When choosing αα, ask these questions:, ask these questions: Does HDoes H00 represent an assumption that people represent an assumption that people

have believed for years? If so, then strong have believed for years? If so, then strong evidence (small evidence (small αα) is needed to persuade ) is needed to persuade them.them.

What are the consequences of rejecting HWhat are the consequences of rejecting H00? ?

Costly changes will require strong evidence.Costly changes will require strong evidence.

Statistical Significance Statistical Significance vs.vs.

Practical SignificancePractical Significance Statistical significance is based on the Statistical significance is based on the

hypothesis test.hypothesis test. A large sample size will almost always A large sample size will almost always

show that small deviations are significant.show that small deviations are significant. Why?Why?

Practical significance means the data isn’t Practical significance means the data isn’t convincing enough to make a change.convincing enough to make a change.

Example of statistical significance Example of statistical significance that is not practicalthat is not practical

Suppose we are testing a new antibacterial Suppose we are testing a new antibacterial cream, “Formulation NS” on a small cut made on cream, “Formulation NS” on a small cut made on the inner forearm. We know from previous the inner forearm. We know from previous research that with n medication, the mean research that with n medication, the mean healing time (defined as the time for the scab to healing time (defined as the time for the scab to fall off) is 7.6 days, with a standard deviation of fall off) is 7.6 days, with a standard deviation of 1.4 days. The claim we want to test here is that 1.4 days. The claim we want to test here is that Formulation NS speeds healing. We will use a Formulation NS speeds healing. We will use a 5% significance level.5% significance level.

We cut 25 volunteer college students and apply We cut 25 volunteer college students and apply Formulation NS to the wound. The mean Formulation NS to the wound. The mean healing time for these subjects is x-bar = 7.1 healing time for these subjects is x-bar = 7.1 days. We will assume that days. We will assume that σσ = 1.4 days. = 1.4 days.

SolutionSolution

We find that the data is We find that the data is statistically statistically significantsignificant..

However, it does not appear that the effect However, it does not appear that the effect is all that great. Is it is all that great. Is it practical practical to use this to use this treatment if it only reduces the amount of treatment if it only reduces the amount of time you have a scab by about a day?time you have a scab by about a day?

CautionsCautions

Watch out for badly designed surveys or Watch out for badly designed surveys or experiments!experiments!

Statistical inference cannot correct for Statistical inference cannot correct for basic flaws in design.basic flaws in design.

Always plot the data (if it’s given to you) Always plot the data (if it’s given to you) and look for outliers or other deviations and look for outliers or other deviations from a consistent pattern.from a consistent pattern.

PROCEED WITH…..

Type I and Type II ErrorsType I and Type II Errors

Sometimes our decision (reject or fail to Sometimes our decision (reject or fail to reject Hreject H00) will be wrong. ) will be wrong.

We could reject HWe could reject H00 when we shouldn’t when we shouldn’t

have.have.

We could fail to reject HWe could fail to reject H00 when we should when we should

have.have.

Type I and Type II ErrorsType I and Type II Errors

HH00 is True is True HH00 is False is False

Reject HReject H00 Type I ErrorType I Error

Fail to Reject HFail to Reject H00 Type II ErrorType II Error

In words…In words…

Type I Error: Reject HType I Error: Reject H00 when H when H00 is actually is actually

true.true.

Type II Error: Fail to reject HType II Error: Fail to reject H00 when H when H00 is is

actually false.actually false.

Why do we care about errors?Why do we care about errors?

If a potato chip factory rejects bags of If a potato chip factory rejects bags of chips that statistically fail to meet a salt chips that statistically fail to meet a salt value, they lose money if the batch is value, they lose money if the batch is really ok.really ok.

On the other hand, if they fail to reject a On the other hand, if they fail to reject a batch that has too much salt, they will batch that has too much salt, they will have unhappy customers.have unhappy customers.

Probability of Type I errorProbability of Type I error

The probability of a Type I error occurring The probability of a Type I error occurring is equal to alpha.is equal to alpha.

Find Probability of Type I errorFind Probability of Type I error

The mean salt content of a certain type of The mean salt content of a certain type of potato chips is supposed to be 2.0mg. The potato chips is supposed to be 2.0mg. The salt content of these chips varies normally salt content of these chips varies normally with standard deviation with standard deviation σσ = 0.1mg. From = 0.1mg. From each batch produced, an inspector takes a each batch produced, an inspector takes a sample of 50 chips and measures the salt sample of 50 chips and measures the salt content of each chip. The inspector rejects content of each chip. The inspector rejects the entire batch if the sample mean salt the entire batch if the sample mean salt content is significantly different from 2mg at content is significantly different from 2mg at the 5% significance level.the 5% significance level.

I’ve Got the Power!I’ve Got the Power! Power is good!Power is good! Power is the probability that a fixed Power is the probability that a fixed αα level level

significance test will reject Hsignificance test will reject H00 if H if Haa is true. is true. Power of a test = 1 – P(Type II Error)Power of a test = 1 – P(Type II Error) Power can increase by having a larger n. More and Power can increase by having a larger n. More and

more often, statisticians are looking at the power of a more often, statisticians are looking at the power of a study along with confidence intervals and significance study along with confidence intervals and significance teststests

Try this out for sizeTry this out for sizeHave HIVHave HIV Do not Do not

have HIVhave HIV

Test is Test is HIV +HIV +

2626 3838

Test is Test is HIV-HIV-

11 12351235

What is the alternative hypothesis?What is the alternative hypothesis? What is a Type I error? What is a Type II error?What is a Type I error? What is a Type II error? What is the probability of a Type I error? Type 2 What is the probability of a Type I error? Type 2

error? Power? error? Power?

H0: A person has HIV.

Error ProbabilitiesError ProbabilitiesThe potato-chip producer wonders whether the significance test of The potato-chip producer wonders whether the significance test of HH 00 : : p p = =

0.08 versus 0.08 versus HH aa : : p p > 0.08 based on a random sample of 500 potatoes has > 0.08 based on a random sample of 500 potatoes has

enough power to detect a shipment with, say, 11% blemished potatoes. enough power to detect a shipment with, say, 11% blemished potatoes. In this case, a particular Type II error is to fail to reject In this case, a particular Type II error is to fail to reject HH 00 : : p p = 0.08 when = 0.08 when

p p = 0.11.= 0.11.

Sign

ificance T

ests: The

Basics

Sign

ificance T

ests: The

Basics

Earlier, we decided to reject H0 at α = 0.05 if our sample yielded a sample proportion to the right of the green line.

What if p = 0.11?

Since we reject H0 at α= 0.05 if our sample yields a proportion > 0.0999, we’d correctly reject the shipment about 75% of the time.

The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - β.

The power of a test against any alternative is 1 minus the probability of a Type II error for that alternative; that is, power = 1 - β.

Power and Type II Error

( ˆ p 0.0999)

HomeworkHomeworkChapter 9Chapter 9

# 19 - 23# 19 - 23