Decision Making with Data Section 8.4. Evaluate data collection procedures Sample size Random...

Decision Making with Data

Section 8.4

Evaluate data collection procedures

• Sample size

• Random assignment

• Validity– Did the test measure what it was supposed to

measure

• Reliability– Would the test provide the same results over

and over

Famous example….

In 1938 the magazine “Literary Digest” polled 10 million people to predict the winner of the 1938 Presidential election (Franklin Roosevelt vs. Alf Landon). The names of the 10 million people were taken from phone books and club membership lists. There surveys were mailed and the people had to return them by mail.

The magazine’s published results: Landon 57% vs. Roosevelt 42%

Election Day Results: Roosevelt (57%) beat Landon (43%).

How could the magazine’s prediction been so far off???

Interpreting GraphsAsk yourself these questions…

• Conclusions– What conclusions can I draw from the graph?

– Do the conclusions that I read seem reasonable?

• Construction of graph– Are the scales and units clear or are they misleading?

– Would another graph be more appropriate?

• Reliability/validity– Do I have questions about how the data were obtained that

could affect the accuracy of the data?

Making Conclusions from data

• Beware of vague or undocumented statements of comparison– “More people are killed in an average airplane accident than an average

car accident. Thus, driving is safer than flying.”

• Beware of percents in claims– 50% of females at Hooters prefer their wings

• Beware of conclusions based on correlation that claim causation– A strong correlation exists in the US between distance between

telephone poles and live births. Does that mean one causes the other?

Misleading conclusions from graphs

• Unlike units

• Scales on axes

• Comparing one-dimensional stats in a 2-D or 3-D picture

The US uses more oil than any other country…

Japan USA

5.6 million

20 million

Barrels per day

Connecting data analysis and problem solving

• Understand the question/problem– Is it qualitative or quantitative?

• Develop a plan– What data is needed? How to collect it?– Sample (randomness and size)

• Implement the plan– Does data collection technique match data needed?– What kind of data? What kind of plot and/or statistics?

• Analyze results– Analyze the statistics; draw conclusions only from what data tells you– Remember: correlation ≠ causation