Experimental Design

Experimental Experimental DesignDesign

Definitions:1) Observational study -

observe outcomes without imposing any treatment

2) Experiment - actively impose some treatment in order to observe the response

I’ve developed a new rabbit food, Hippity Hop.

Rabbit Food

Makes fur soft & shiny!

Increases energy!100% of

daily vitamins & essential

oils!

Can I just make these claims?

What must I do to make these claims?

Who (what) should I test this on?

What do I test?

NO

Do an experiment

Rabbits

The type of food

3) Population – the entire group of individuals I am interested in

4) Sample – a part of the population we can examine closely.

5) Variable of interest – what you measure

6) Treatment – a specific experimental condition applied to the subjects or units

I plan to test my new rabbit food.

What is my sample?

What is my treatment?

What is the variable of interest?

The rabbits I choose for my experiment

Type of food

How well they grow

Hippity HopI’ll use my pet rabbit, Lucky!

Since Lucky’s coat is shinier & he has more energy, then Hippity Hop is a better rabbit food!

7) Control group – a group that is used to compare the factor against; can be a placebo or the “old” or current item8) Placebo – a “dummy” treatment that can have no physical effect

Old Food Hippity Hop

Now I’ll use Lucky & my friend’s rabbit, Flash. Lucky gets Hippity Hop food & Flash gets the old

rabbit food.

WOW! Lucky is bigger & shinier so Hippity Hop is better!

WOW! Lucky is bigger & shinier so Hippity Hop is better!

Old Food Hippity Hop

The first five rabbits that I catch will get

Hippity Hop food and the remaining

five will get the old food.The Hippity Hop rabbits

have scored higher so it’s the better food!

Old Food Hippity Hop

Number the rabbits from 1 – 10.

Place the numbers in a hat.

1

2

3

45

6

78

9 10

The first five numbers pulled from the hat will be the rabbits that get

Hippity Hop food.

I evaluated the rabbits & found that the rabbits eating Hippity Hop are better than the old food!

The remaining rabbits get the old food.

39857

9) blinding - method used so that units do not know which treatment they are getting

10) double blind - neither the units nor the evaluator know which treatment a subject received

Rabbit Food

Hippity Hop Rabbit Food makes fur soft and shiny, &

increases energy for

ALL types of rabbits!Can I

make this

claim?

Principles of Experimental Principles of Experimental DesignDesign

• Control methods - reduces extraneous variables on the response; reduces variation in replication

• Replication - the ability to repeat trials or an experiment on many subjects to quantify the natural variation in the experiment

• Randomization – the use of chance to assign subjects to treatments

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!

The ONLY way to show cause & effect is with a well-designed, well-controlled experiment!!The ONLY way to show

cause & effect is with a well-designed, well-controlled experiment!!!

Example 1: A farm-product manufacturer wants to determine if the yield of a crop is different when the soil is treated with three different types of fertilizers. Fifteen similar plots of land are planted with the same type of seed but are fertilized differently. At the end of the growing season, the mean yield from the sample plots is compared.

Sample?

Treatment?

Variable of interest?

Plots of land used for experiment

3 Types of fertilizer

Yield of crop

Example 2: An article from USA Today reports the number of victims of violent crimes per 1000 people. 51 victims have never been married, 42 are divorced or separated, 13 are married, and 8 are widowed.Is this an experiment? Why or why not?

No, no treatment was imposed on people.

Example 3: Four new word-processing programs are to be compared by measuring the speed with which standard tasks can be completed. One hundred volunteers are randomly assigned to one of the four programs and their speeds are measured.

Is this an experiment? Why or why not?What is the treatment?

Variable of interest?

Yes, a treatment is imposed.

Four different word-processing programs

speed

Randomization reduces bias (so it reduces error) by spreading any uncontrolled confounding variables evenly throughout the treatment groups.

Variability is controlled by sample size. Larger samples produce statistics with less variability.

Is there another way to reduce

variability?

High bias & low variability

Low bias & low variability

Low bias & high variability

High bias & high variability