5.2 Day 2: Designing Experiments

Block Design

In general, men and women may react differently to different medications or be able to lift different amounts of weight or on average do a different number of push ups.

When comparing the number of push ups that a group of fitness students can do at the end of a training camp, it would make more sense to separate men and women into separate comparison groups.

This type of separation is called blocking.

Block Design

A block is a group of experimental units or subjects that are known before the experiment to be similar in some way that is expected to systematically affect the response of the treatments.

In a block design, the random assignment of units to treatments is carried out separately within each block.

Blocks are another form of control.

Ex: Comparing Cancer Therapies

The progress of a type of cancer differs in women and men.

A clinical experiment will compare three different treatments.

Men and women will first be separated into blocks.

Then each block will be randomly assigned to the three different treatments.

It’s important that you understand that blocking is not always due to gender. Subjects could be blocked

based on whether or not they exercise. Blocking is used to reduce variability. Blocking

has nothing to do with random assignment-for example, one

does not randomly assign subjects to gender!

Outline of block design for cancer therapies experiment

Importance of Blocking

Blocks allow us to draw separate conclusions about each block, for example, about men and women in the cancer study.

Blocking also allows more precise overall conclusions because the systematic differences between men and women can be removed when we study the overall effects of the three therapies.

Blocking vs. Randomization

Blocking is used to control for the variables you know about that might influence the response.

Randomization is used to control for the variables you do not know about.

Use the mantra: control what you can, block what you can’t control, and randomize the rest.

Matched Pairs Design

Completely randomized designs are the simplest statistical designs that clearly demonstrate the principals of CRR.

However, completely randomized designs are often inferior to more elaborate statistical designs.

Using a matched pairs design, where subjects are matched in various ways can produce more precise results.

Matched Pairs Design

The subjects are matched in pairs and only two treatments are compared

For example, an experiment to compare two advertisements for the same product might use pairs of subjects with the same age, sex, and income.

It is not always easy to match subjects. One common variation of the matched pairs

design imposes both treatments on the same subjects, so that each subject serves as his or her own control.

Matched pairs are an

example of block design.

Ex: Cell Phones and Driving

In this experiment, the effects of driving while talking on a cell phone are to be observed.

There are two treatments: driving in a simulator and driving in a simulator while talking on a hands-free cell phone.

The response variable is the time the driver takes to apply the brake when the car in front brakes suddenly.

40 students subjects are assigned at random, 20 students to each treatment

Since subjects differ in driving skill and reaction times, experimenters used a matched pairs design in which all subjects drove both with and without using the cell phone.

They compared each individual’s reaction time with and without using the cell phone.

The reason that subjects are separated into two groups, those who drive first without a cell phone and those who drive first with a cell phone is to reduce the possibility that talking on a cell phone would be confounded with driving a simulator for the first time.

The proper procedure would require that all subjects first

be trained in using the simulator, that the order in which a subject drives with and without the phone be random, and that the two

drives be on separate days to reduce carryover effects.

Is the placebo effect pseudoscience?

Fourteen healthy men were given a saltwater injection that caused pain to their jaws. They were then injected with a placebo and told it was a pain killer. Researchers monitored their brain activity during the process. Each man’s brain released more natural painkilling endorphins after the placebos were administered.

Double Blind Experiment

In a double-blind experiment, neither the subjects nor the people who have contact with them know which treatment a subject received.

In the case of a medical study, neither the doctor nor the patient would know whether or not the patient was taking a placebo. This helps eliminate unconscious bias in the way the patient is treated.

Lack of Realism

Lack of realism is the most serious potential weakness of experiments. Ex: A study compares two television

advertisements by showing TV programs to student subjects. The students know it’s “just an experiment.” We can’t be sure that the results apply to everyday television viewers. Many behavioral science experiments use as subjects students who know they are subjects in an experiment. That’s not a realistic setting.

Ex: The Third Brake Light

Do high centered brake lights, which have been required on all cars sold in the U.S. since 1986, really reduce collisions?

When randomized comparative experiments were conducted prior to 1986, collisions were reduced by as much as 50%.

After 1986, requiring the third light only led to a 5% drop.

What happened?

When the experiment was first conducted, most cars

did not have the third brake light, so it caught the eye of following drivers. Now that all cars have them, they no

longer capture attention.

Ex: Placebo cigarettes?

A study of the effects of marijuana recruited young men who used marijuana.

Some were randomly assigned to smoke marijuana cigarettes, while others were given placebo cigarettes.

This failed: the control group recognized that their cigarettes were phony and complained loudly.

It may be quite common for blindness to fail because the subjects can tell which treatment they are receiving.