Statistics – Ch 4.2A Notes Name: ________________________ Experiments

Observational Study vs Experiment

The goal of an ____________________________can be to describe some group or situation, compare groups, or examine the relationship between variables

The goal of an __________________ is to determine whether a treatment causes a change in

the response. Experiments are the only way to establish a cause-and-effect relationship.

____________________________ occurs when two variables are associated in such a way that their effects on a response variable cannot be distinguished from each other.

Confounding Examples: Well-designed experiments take steps to prevent confounding.

Example Problem: The dentists in a dental clinic would like to determine if there is a difference between the number of new cavities in people who eat an apple a day and in people who eat less than one apple a week. They are going to conduct a study with 50 people in each group.

Fifty clinic patients who report that they routinely eat an apple a day and 50 clinic patients who report they eat less than one apple a week will be identified. The dentists will examine the patients and their records to determine the number of new cavities the patients have had over the past two years. They will then compare the number of new cavities in the two groups.

a. Why is this an observational study and not an experiment?

b. Explain the concept of confounding in the context of this study. Include an example of a possible confounding variable.

c. If the mean number of new cavities for those who ate an apple a day was statistically significantly smaller than the mean number of new cavities for those who ate less than one apple a week, could one conclude that the lower number of new cavities can be contributed to eating an apple a day? Explain.

We plant tomatoes in a garden that’s half-shaded. We test fertilizer by putting it on the plants in the sun and apply none to the shaded pants. Months later the fertilized plants bear more and better tomatoes.

Maybe it’s the fertilizer, maybe it’s the sun, maybe we need both. We’re unable to conclude that the fertilizer works because any effect of the fertilizer is confounded with any effect of extra sunshine.

An observational study observes individuals and measures variables of interest but does not attempt to

influence the ________________________________.

An experiment deliberately imposes some _________________ on individuals to measure their responses.

Exam Tip: If you are asked to identify a possible confounding variable in a given setting, you are expected to explain how the variable you choose (1) is associated with the explanatory variable and (2) how it affects the response.

Key

responsetreatment

observationalstudy

experiment

onfounding

subjects are notrandomlyassigned totreatments

Twovariables areconfounding if theireffectonthenumberofnewcavitiescannotbedistinguishedfromoneanotherConfoundingwouldoccurifpatientswhoeatanappleadaydifferfromthosewhoeatlessthanoneappleaweek InHisexampledietorgenerallevelofhealthmaybeconfoundingvariablesforexample itispossiblethatpeoplewhoeatanappleadayaremorenutritionconsciousandhaveamorehealthydietingeneralthanthosewhoeatlessthanoneappleaweekandthismayexplaintheobserveddifferenceindentalheat

Nobecause it isnotanexperimentcauseandeffectrelationships cannotbedrawnfromobservationalstudies

Statistics – Ch 4.2A Notes Name: ________________________ Experiments

Practice Problem: Does eating dinner with their families improve student’s academic performance? According to an ABC news article, “Teenagers who eat with their families at least 5 times a week are more likely to get better grades in school.” This finding was based on a sample survey conducted by researchers at Columbia University.

1. Was this an observational study or an experiment? Justify

2. What are the explanatory and response variables?

3. Explain clearly why such a study cannot establish a cause-and-effect relationship. Suggest a variable that may be confounded with whether families eat dinner together.

The Language of Experiments

• Sometimes, the explanatory variables in an experiment are called ____________________.

• In such an experiment, each treatment is formed by combining a specific value (often called a

_______________) of each of the factors.

Practice Problem: What are the effects of repeated exposure to an advertising message? The answer may depend on both the length of the ad and how often it is repeated. An experiment investigated this question using 120 undergraduate students who volunteered to participate. All subjects viewed a 40-minute television program that included ads for a digital camera. Some subjects saw a 30-second commercial; others, a 90-second version. The same commercial was shown either 1, 3, or 5 times during the program. After viewing, all the subjects answered questions about their recall of the ad, their attitude toward the camera, and their intention to purchase it.

For the advertising study, identify the experimental units or subjects, explanatory variable and response variables, and the treatments.

A specific condition applied to the individuals in an experiment is called a __________________________.

If an experiment has several explanatory variables, a treatment is a combination of specific values of these

variables.

The experimental _____________ are the smallest collection of individuals to which treatments are

applied. When the units are human beings, they often are called ___________________.

Anobservationalstudybecausestudentswerenotassigned to atreatmentTheywerenotassignedtoeat aparticularnumberofmealswiththeirfamily

Explanatory ofmealsperweekeatenwiththeirfamilyResponseGPA

onlyexperimentscanestablishacauseandeffectrelationshipTherearemanypossible confoundingvariablesStudentswhohaveparttimejobsmaynothaveasmuchtimetostudyParttimejobsmaybeinfluencingtheirgradesandhowoftentheyeatwiththeirfamily

treatment

unitsSubjects

factors

level

TreatmentsExperimentalunits120undergradstudents

30see 1time 90see ltimeExpvariableDlengthofcommercialGlevels sosee3times 90see stimes

2Numberofrepetitions 3levels goseegtimes gosee 5timesResponsevariablei

measureofsubjectsrecalloftheaddHeiraltitudeaboutthecameraandintentiontopurpose

Statistics – Ch 4.2A Notes Name: ________________________ Experiments

How to Experiment Poorly

A high school regularly offers a review course to prepare students for the SAT. This year, budget cuts will allow the school to offer only an online version of the course. Suppose the group of students who take the online course earn an average increase of 45 points in their math scores from a pre-test to the actual SAT test. Can we conclude that the online course is effective?

Subjects (students) Treatment (online course) outcome (Increase in Math score) No: Maybe the students in the online course improved their math scores because of what they were learning in their school math classes, not because of the online course. This confounding prevents us from concluding that the online course is effective. How to Experiment Well

________________________ alone isn’t enough, if the treatments are given to groups that differ greatly, bias will result.

• If we allow students to select online or classroom instruction, more self-motivated students are likely to sign up for the online course.

• Personal choice will bias our results

The solution to the problem of bias is _____________________________.

We can _________________________ for the effects of some variables by keeping them the same for both groups.

Principles of Experimental Design

In an experiment, random assignment means that experimental units are assigned to

________________________ using a chance process.

The basic principles for designing experiments are as follows:

1. _________________. Use a design that compares two or more treatments.

2. _________________ Use chance to assign experimental units to treatments. Doing so helps create roughly equivalent groups of experimental units by balancing the effects of other variables among the treatment groups.

3. _________________. Keep other variables that might affect the response the same for all groups.

4. _________________. Use enough experimental units in each group so that any differences in the effects of the treatments can be distinguished from chance differences between the groups.

Comparison

random assignment

treatments

Control

ComparisonRandomAssign

ControlReplication

Statistics – Ch 4.2A Notes Name: ________________________ Experiments

Practice Problem: Random Assignment

This year, the high school has enough budget money to compare the online SAT course with the classroom SAT course. Fifty students have agreed to participate in an experiment comparing the two instructional methods.

How would you randomly assign 25 students to each of the two methods?

1. Using 50 identical slips of paper

2. Using technology

3. Using Table D

How are the four principles used in this experiment?

Comparison: Experimenters used a design that compared SAT results from the two different treatments (online course and classroom course)

Random Assignment: Was used to assign students to treatments. This ensured that treatment groups were roughly equivalent to begin with. Other variables (Math courses or amount of study time available) were evenly spread among the two groups.

Control: Experimenters must keep some variables the same for both groups. All students must receive the same pre-test, SAT test, length of test, timing, content and instructor.

Replication: there were only 25 subjects per treatment group. This number may need to be larger to ensure that the difference in SAT scores was due to the online class and not to chance variation in the random assignment.

Why is Control Important?

1. To prevent __________________________

• If the SAT students have different teachers we won’t know the difference in average improvement was due to the teacher or course type.

2. To reduce variability in the _______________________________

• If students selected the number of class session to attend, some will choose to attend fewer and some more.

• Their choices will increase the variation in response variable (improvement)

Placea11nameson aslip theymustbethesamesizeandputthemin a hatShakeupthenamesanddraw25 one at a time Sendthese 25tothefirsttreatmentandtheremainingtothesecondtreatment

assigneachstudentanumberfrom a so Generate 25randomnumbers onacalculatorusingRandintorso as andignorerepeatsThese25studentsareassignedtothefirsttreatmentandHeremainingasstudentsareassignedtothesecond

assigneachstudentatwodigitnumberfromoltosouselinelol intableDlookingattwodigitsat atimeskippingrepeatsandnumbersgreaterthanso

Confounding

response variable

Statistics – Ch 4.2A Notes Name: ________________________ Experiments

Create a picture of the SAT Experiment below:

treatment 1 RecordSATonlineclass 0 scoresn 25 oVolunteers compare

A soRandomAssignment TTreatment2 RecordSATclassroom

n 25Scores