2.1 Observe or Experiment? Should we use an observational study or an experiment? What’s the...

Statistics Unit 2Gathering Data

2.1 Observe or Experiment?

Should we use an observational study or an experiment?What’s the difference?

In an observational study, we record data or survey results WITHOUT INTERFERING with the subjects responsesIn an experiment, the researcher IMPOSES TREATMENTS on the subjects, and records data for each group.

Which is best?A well-controlled experiment is the ONLY

WAY to establish CAUSE AND EFFECTAn observational study with a well-chosen

sample group can give us quick results with little expense

Often, the results of an observational study will cause interest in designing an experiment to follow up.

Observational StudyAdvantages1. Quick results2. Not too expensive3. Few ethical

restraints4. Can cover a wide

variety of factors at once

Disadvantages1. Can only show

CORRELATION, NOT CAUSATION

2. Sometimes, it’s difficult to observe without affecting the data

ExperimentAdvantages1. Can establish

CAUSE and EFFECT

2. Can eliminate many lurking variables

Disadvantages1. Ethical restrictions,

especially on human testing

2. Financial restrictions

3. Time constraints

Either study can be done well or done poorly

Sample sizePoor questions Bad designLurking variablesOverreaching populationOverreaching inference

2.2 What makes a good observational study?

Types of Bias to AvoidSampling Bias: When the sample does not reflect the

population of interest. There are several sampling errors that might cause this: sample size is too small (called undercoverage), the sample is not random, the sample is not chosen from the population of interest.

Response Bias: When the members of the sample group are giving unusable or unmeaningful responses. This could be for a number of reasons: confusing questions, leading questions, unprotected anonymity

Nonresponse Bias: When significant numbers of members of the sample group fail to respond to the survey. Phone-in and reply-by-mail are highly likely to suffer from this type of bias.

So What Size Sample is Needed?Recall the formula for Margin of Error: MOE

=

Suppose we want our results to have a margin of error of only +/- 3%. What sample size will insure this?

Suppose we would be satisfied with a MOE of 7.5%. What sample size would be needed?

Are these results statistically significant?A randomly chosen group of 250 dentists

were polled about whether they would recommend brand A or brand B as the first choice for toothpaste. 120 chose brand A, 110 chose brand B, and the rest had no opinion.

What is the margin of error for this study?

Is the preference for brand A statistically significant? How can we tell?

2.3 Types of Observational Studies

Some Basic TypesSample Survey: Collects data now, about present. Most

opinion and political surveys are this type.

Retrospective Study: Collects data now, but about the past. We get the data immediately, but must trust people’s memories of past experiences

Case-Control Study: many medical studies are this type. It avoids the ethical restrictions on experimenting on dangerous medical concerns.

Prospective Study: set up now, but must wait to collect data in the future. Very reliable data, but the waiting is lengthy and not always practical.

Cautions for Observational StudiesBeware the LURKING VARIABLE

Beware the CONFOUNDING VARIABLE

Be cautious in making claims about CAUSE

CORRELATION does not imply CAUSATION!!!

2.4 What makes a good experiment?

4 Components of a good ExperimentReplication: Does the experiment have enough

subjects? Remember, we can use MOE to measure this

Randomization: Were the subjects divided into groups by a random method?

Control/Comparison Group: One group should receive no treatment, or perhaps the current standard of care. Not all groups should receive the new treatment being tested.

Blinding: Subjects should be unaware of which treatment group they are a part of. This is critical because of the placebo effect.

What’s wrong here?1. PharmaB wants to test their new

medication for migraines. Their 2700 subjects are randomly placed into three groups. Group A receives a does of 50 mg. Group B receives a dose of 100 mg. Group C receives a dose of 200 mg. Data is collected from each group.

What’s wrong here?2. Science Sam wants to know if vitamin C

will help students improve their math scores. His 2000 subjects choose to be in Group A, which receives the vitamin, or in Group B, which receives the placebo. Sam collects the data from both groups.

What’s wrong here?3. Nervous Nelly wants to test a new organic

form of Xanax. She randomly assigns her 21 subjects to three groups. Group A receives a placebo. Group B receives their regular Xanax. Group C receives the new treatment. Nelly records the data from all three groups.

Do these results really matter?Are the results statistically significant?

Use MOE

Are the results practically significant? Harder to compute, but also important. For example, if the new medication will end a migraine 30 minutes sooner, but costs 5 times as much as the current treatment, would consumers by likely to buy it?

Testing Statistical Significance1. At St. Luke's hospital in 1998, 674 women

were diagnosed with breast cancer. Five years later, 88% of the Caucasian women and 83% of the African American women were still alive. A researcher concludes that being Caucasian causes women with breast cancer to have an increased chance of surviving five years. Use MOE to determine whether this conclusion is statistically significant.

2.5 Types of ExperimentsMultifactor: Sometimes we want to test

more than one factor. Especially in medical situations, there is often more than one contributing factor. This design is also used to determine whether prescriptions affect each other, and how. They do require a larger group of subjects, and are usually limited to two or three factors at the most.

2.5 Types of ExperimentsMatched Pair: Matched Pair design simply

means that each subject is part of a pair. It could be “married couple” or “plant from the same field”, or any logical reason to pair the subjects and track their data together.

Crossover Design: is a type of matched pair in which each subject is part of the treatment group, and at a different time, also part of the control group. This is a very strong model, as it eliminates nearly all lurking variables.

2.5 Types of ExperimentsBlock Design: This design is similar to

stratified random sampling for observational studies. By blocking the subjects first by gender, for example, the researcher insures that each group will contain both genders. Blocking can be done by gender, age, economic status, ethnicity, or any variable of interest.