Dr. Kari Lock MorganDepartment of StatisticsPenn State University

Teaching the Common Core: Making Inferences and Justifying Conclusions

ASA Webinar

2/25/15

Use data from a sample survey to estimate a population mean or proportion; develop a margin of

error through the use of simulation methods for random sampling

Use data from a randomized experiment to compare two

treatments; use simulation to decide if differences between

parameters are significant

What proportion of online adults use Facebook?

http://www.pewinternet.org/2015/01/09/social-media-update-2014/ (January 9th, 2015)

Key question: How far might the true p lie from this estimate?

Sample proportion:

Population proportion:

p = ???

Margin of Error

statistic ± margin of error

Estimates should come with a corresponding margin of error:

Margin of error: how far the true value might be from the sample statistic?

Key point: To see how far the truth might be from the statistic, we see how far the statistic might be from the truth.

Simulate lots of random samples!

What would happen if we could take lots of different samples (each of size n = 1597) from the population of US online adults?

In order to do this, we would need to know the true p… for now, just use our best guess

Simulate many random samples!

www.lock5stat.com/statkey Free easy to use online (or offline as chrome app)

Distance from parameter to statistic gives distance from statistic to parameter

p

Rare for statistics to be further than this from parameter

So rare for parameter to be further than this from statistic

Margin of error depends on variability of the

statistic

margin of errormargin of error

Standard Error

The standard error of a statistic, SE, is the standard deviation of the

sample statistic

The standard error measures how much the statistic varies from sample to sample

(or how far we expect statistics to fall from the true parameter)

The larger the SE, the larger the margin of error

p

SE = 0.15

SE = 0.05

Rare for statistics to be further than this from parameter

SE = 0.15SE = 0.05

95% of statistics will be within 2SE of the true parameter value

truth

95% of statistics

2 SE2 SE

We often use 2 standard errors as the margin of error

Interval Estimate

statistic ± 2 SE

A common interval estimate is

(This is a 95% confidence interval, and will capture the true parameter for 95% of all samples generated.)

What proportion of online adults use Facebook?

statistic ± 2 SE 0.71 ± 2(0.011) (0.688, 0.732)

We are 95% confident that between 68.8% and 73.2% of US adults who are online use Facebook.

SE = 0.011Margin of

error ≈ 2%

Interval Estimation

Population(???)

Population(???)

statistic ± ME

Sample

Best Guess at Population

Sample

Sample

Sample

SampleSampleSample

. . .

Distribution of the statistic

Calculate statistic for each sample

Standard Error (SE): standard deviation of the statistic

Margin of Error (ME)(95% CI: ME = 2×SE)

GOAL:

Use data from a sample survey to estimate a population mean or proportion; develop a margin of

error through the use of simulation methods for random sampling

Use data from a randomized experiment to compare two

treatments; use simulation to decide if differences between

parameters are significant

✔

Does consuming beer attract mosquitoes?

Experiment: 25 volunteers drank a liter of beer,18 volunteers drank a liter of waterRandomly assigned!Mosquitoes were caught in traps as they approached the volunteers.1

1 Lefvre, T., et. al., “Beer Consumption Increases Human Attractiveness to Malaria Mosquitoes, ” PLoS ONE, 2010; 5(3): e9546.

Beer and Mosquitoes

Beer and Mosquitoes Data

Beer mean = 23.6

Water mean = 19.22

Does drinking beer actually attract mosquitoes, or is the difference just due to random chance?

Beer mean – Water mean = 4.38

Number of Mosquitoes

Beer Water 27 21 20 22 21 15 26 12 27 21 31 16 24 19 19 15 23 24 24 19 28 23 19 13 24 22 29 20 20 24 17 18 31 20 20 22 25 28 21 27 21 18 20

Simulate random chance!Number of Mosquitoes

Beer Water 27 21 20 22 21 15 26 12 27 21 31 16 24 19 19 15 23 24 24 19 28 23 19 13 24 22 29 20 20 24 17 18 31 20 20 22 25 28 21 27 21 18 20

Find out how extreme these results would be, if there were no difference between beer and water.

What kinds of results would we see, just by random chance?

Number of Mosquitoes

Beverage 27 21 20 22 21 15 26 12 27 21 31 16 24 19 19 15 23 24 24 19 28 23 19 13 24 22 29 20 20 24 17 18 31 20 20 22 25 28 21 27 21 18 20

Beer Water

Find out how extreme these results would be, if there were no difference between beer and water.

What kinds of results would we see, just by random chance?

Number of Mosquitoes

Beverage 20 22 21 15 26 12 27 21 31 16 24 19 19 15 23 24 24 19 28 23 19 13 24 22 29 20 20 24 17 18 31 20 20 22 25 28 21 27 21 18 20

27 21

2127241923243113182425211812191828221927202322

2026311923152212242920272917252028

Simulate random chance!

Calculate statistic (difference in means)Repeat thousands of times!www.lock5stat.com/statkey

StatKey

P-value

Proportion as extreme as observed statistic

observed statistic

Distribution of Statistic Assuming No Difference

If there were no difference between beer and water regarding mosquito attraction, we would only get results this extreme 1 out of 1000 times

• If there were no difference between beer and water regarding mosquito attraction, we would only get results this extreme 1 out of 1000 times

• This is very unlikely to happen just by chance, so there probably is a difference! We have evidence that beer attracts mosquitoes.

• (Because this would be unlikely to happen just by random chance, the difference is statistically significant).

Making a Conclusion

Hormone Replacement TherapyUntil 2002, hormone replacement therapy (HRT) was

commonly prescribed to post-menopausal women. This changed in 2002, when the results of a large clinical trial were published

8506 women were randomized to take HRT, 8102 were randomized to placebo. 166 HRT and 124 placebo women developed invasive breast cancer

Does hormone replacement therapy cause increased risk of breast cancer?

How unlikely would this be, just by random chance, if there were no difference between HRT and placebo regarding invasive breast cancer?

HRT and Invasive Breast Cancer

If there were no difference between HRT and placebo regarding invasive breast cancer, we would only see results this extreme 2 out of 100 times.

We have evidence that HRT increases risk of invasive breast cancer.

Hormone Replacement TherapySame trial, different variable of interest.

8506 women were randomized to take HRT, 8102 were randomized to placebo. 502 HRT and 458 placebo women developed any kind of cancer.

Does hormone replacement therapy cause increased risk of cancer in general?

How unlikely would this be, just by random chance, if there were no difference between HRT and placebo regarding cancer?

HRT and All Cancer

If there were no difference between HRT and placebo regarding cancer, we would see results this extreme about 24 out of 100 times, or about a quarter of the time.

We do not have evidence that HRT increases risk of cancer in general.

Use data from a sample survey to estimate a population mean or proportion; develop a margin of

error through the use of simulation methods for random sampling

Use data from a randomized experiment to compare two

treatments; use simulation to decide if differences between

parameters are significant

✔

✔

Want more?

2 ½ day workshops on teaching statistics in the common core (pending NSF funding)

When and where? (dates tentative)

2015 (New York): June 29-July 1 or July 1-3

2016 (Philadelphia): June 20-22 or June 22-24

2017 (Boston): June 19-21 or June 21-23

Interested? Email me at klm47@psu.edu

Thanks for listening!