Power of a testPower of a test
The powerpower of a test (against a specific alternative value)• Is the probability that the test
will reject the null hypothesis when the alternative is true
• In practice, we carry out the test in hope of showing that the null hypothesis is false, so high power is important
H0 True
H0
FalseReject
Fail to reject
Type I Correct
Correct
Type II
Power
Suppose H0 is true – what if we decide to
fail to reject it?
Suppose H0 is false – what if we decide to
reject it?
Suppose H0 is true – what if we decide to reject
it?
Suppose H0 is false – what if
we decide to fail to reject it?
We correctly We correctly reject a false Hreject a false H00!!
A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.
What are the hypotheses?H0: p = .7
Ha: p < .7
A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.
Find p and p.p = .7p = .0458
A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. A random sample of 100 people are vaccinated and then exposed to the flu. Is this claim too high? Use = .05.
What is the probability of committing a Type I error? = .05
.7
H0: p = .7Ha: p < .7 = .05For what values of the sample proportion would you reject the null hypothesis?
Invnorm(.05,.7,.0458) =.625
= .05
p?
So if we get p-
hat=.625 or less, we
would reject H0.
H0: p = .7Ha: p < .7
We reject H0 and decide that p<.7.
Suppose that pa is 0.6.
What is the probability of committing a Type II error?
Where did this
number come from?
I selected a number that
was less than .7
.7
.6
= .05Reject
What is a type II error?
= ?
How can we find this area?What is the standard deviation of this curve?
Normalcdf(.625,∞,.6,.0458) =.293
failing to reject H0 when the alternative is true
What is the power of the test?
Power = 1 - .293= .707
.7
.6
= .05
=.293
What is the definition of
power?The probability that the test correctly rejects H0
Power = ?
Power - the probability
that the test correctly
rejects H0, if p = .6, is .707
Is power a conditional probability?
Suppose we select .55 as the alternative proportion (p).
a)What is the probability of the type II error?
b) What is the power of the test?
= normalcdf(.625,∞, .55,.0458) = .051
.7
.6
.55
= .05
Power = 1 - .051= .949
What happened to the power of the
test when the difference |p0 – pa| is
increased?
Suppose we select .65 as the alternative proportion (p).
a) What is the probability of the type II error?
b) What is the power of the test?
Power = 1 - .707= .293
.7
.6
.65
= .05
= normalcdf(.625,∞, .65,.0458) = .707
What happened to the power
when the difference |p0-pa|
is decreased?
Power
Suppose that we change alpha to 10%.
Using pa = .6, what would happen to the probability of a type II error and the power of the test?
.7
.6
= .05 = .1
Power
The probability of the type II error () decrease and power increased, BUT BUT the probability of a type I error also also increasedincreased.
= .1836
Power = .8164
What happens to What happens to , , , & , & power when the sample size power when the sample size
is increased?is increased?Reject H0Fail to Reject H0
p0
pa
P(type II) decreases
when n increases
Power
Power increases when n
increases
p0
pa
Reject H0Fail to Reject H0
Power = 1 -
Recap:What affects the power of a test?
As |p0 – pa| increases, power increases
As increases, power increases
As n increases, power increases
Facts:Facts:• The researcher is free to determine the
value of .• The experimenter cannot control , since it
is dependent on the alternate value.• The ideal situation is to have as small as
possible and power close to 1. (Power > .8)• As increases, powerpower increases. (But also
the chance of a type I error has increased!)• Best way to increase power, without
increasing , is to increase the sample sizesample size
A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high?
Identify the decision:
a) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is not.
Type I Error
A new flu vaccine claims to prevent a certain type of flu in 70% of the people who are vaccinated. In a test, vaccinated people were exposed to the flu. Is this claim too high?
Identify the decision:
b) You decide that the proportion of vaccinated people who do not get the flu is less than 70% when it really is.
Correct –
Power!!
Recommended
