The Lady Tasting Coffee: A Case Study in Experimental Design

The History of ExperimentationExperimentation characterizes modern science.

Galileo (1564-1642) reportedly dropped balls of various masses from the Leaning Tower of Pisa.

Assuming the story of Galileo’s Pisa experiment is true: How many balls did he drop? How many times did he repeat the comparison? What were his independent and dependent variables? How did he measure the time to impact? We don’t know the answers to these questions…

Take Home Message: Experimental design was haphazard prior to the 1920’s.

Ronald Aylmer Fisher• Considered by some scientists to be the father of

modern statistics .• Poor eyesight; did a lot of math in his head without

paper or pencil.• In 1919, he began working as a statistician at the

Rothamsted Agricultural Experiment Station in the United Kingdom.

• Published many papers and wrote several books on experimental design and evolution.

Ronald Aylmer Fisher1890-1962

http://www-groups.dcs.st-and.ac.uk/~history/Mathematicians/Fisher.html

At Rothamsted, Fisher recognized problems with some of the agricultural experiments

Fisher’s Solution:Replicate and randomize to spread variation evenly among treatments.

Source of Picture: http://www.ipm.iastate.edu/ipm/icm/files/images/uneven-corn-VS6.jpg

Same field, same treatment, but plant performance is uneven...

Thick GrowthThin

Growth

Lessons Learned at RothamstedExperiments at Rothamsted prior to Fisher

generally involved two fields (containing hundreds of plants), each receiving a treatment.Example: two levels of nitrogen (N) fertilizer

Problem: So much variability exists within a field itself that it is difficult or impossible to tease out the effect the treatment.

Field withHigh N

Field withLow N

Fisher’s Solution at Rothamsted– Old Problematic Design: One large field receiving high

nitrogen (N), one large field receiving low nitrogen (N). (Today this design is sometimes called “pseudoreplication” if

the experimenter attempts to say that the sample size is the number of plants.)

– New Improved Design: Many small plots, randomly receiving high N or low N; plots can also be blocked to help tease out the variation due to location and local conditions.

Hurlbert, S. H. (1984). Pseudoreplication and the design of ecological field experiments. Ecological monographs 54(2): 187-211.

Examples of Correct & Incorrect Ways to Randomize Treatments

Correct Ways:

• Use a random number table.

• Pick treatments from a hat.

• Flip a coin.

Incorrect Ways:

• Haphazardly decide which experimental units should receive which treatments. (Problem: too tempting for experimenter to bias.)

• Use a net to grab the goldfish in an ecology study. (Problem: might pick just the easiest to catch, sickly animals.)

• Alternate treatments (every other one). (Problem: that’s systematic, not random; who knows what other factors vary in the same systematic way.)

• Assign people to drug study on the basis of their last name. (Problem: could be related to a person’s ancestry.)

Fisher, Randomization, Replication & Blocking• No replication (or pseudoreplication) (Rothamsted, pre-Fisher):

• Replicated with complete randomization:

• Replicated, randomized and blocked design:

Field withHigh N

Field withLow N

Field broken up into smaller plots

Plots are blocked by location or other condition; treatments are applied randomly to plots within blocks.

Field broken up into smaller plots & plots are grouped.

Treatments are applied to plots rather than to an entire field; this improves replication & interspersion of treatments.

Dashed rectangle is a block

Another of Fisher’s Contributions to Statistics:The Analysis of Variance (ANOVA)Allows scientists to mathematically partition variation among different sources (treatments, blocks, plots, for example).Some of Fisher’s contributions to the field of statistics grew out of his experience with spatial agricultural experiments at Rothamsted.

From: Sokal, Robert R., & F.James Rohlf, Biometry: The Principles andPractice of Statistics in Biological Research, San Francisco: W.H. Freeman.

At Rothamsted, Fisher saw firsthand that the purpose of good experimental design is not to eliminate variation entirely, but rather to try to ensure that extraneous variation is spread evenly among treatments. In the case of ANOVA, the experimental design can enable the variation to be partitioned mathematically during analysis.

Variation in growth of plants can be partitioned into different sources of variation:1. Variation in soil moisture, texture, etc. within a plot.2. Variation between treatments (high N and low N).3. Variation in soil moisture, texture, sunlight, etc., among blocks.

Why do these two plants differ in growth? Is it because of block, treatment, or extraneous variation within plots?

The Design of Experiments (1935)

One of the first chapters of this textbook written by Fisher is the essay, “Mathematics of a Lady Tasting Tea.”

A lady tasting teaCan she tell whether the milk was added before or after the tea?

Afternoon tea during study abroad experience by University of Pittsburgh at Bradford students at the University of Sussex in Brighton, Great Britain. Copyright © Janelle Elmquist. Used with permission.

So, you think statistics is boring . . . Statisticians and the history of statistics are far from boring.Other interesting trivia on Fisher:

-Charming but had a terrible temper (and a big ego)-Smoked a pipe & argued professionally in the 1950’s that smoking did not cause cancer-Supported eugenics

Picture taken from:

Parascandola, M. (2004). "Two approaches to etiology: the debate over smoking and lung cancer in the 1950s." Endeavour 28(2): 81-86.

Take Home Messages

• The 1920’s was a rich time for the development of concepts of modern experimental design.

• Fisher was one of a number of statisticians who greatly affected the development of modern statistics.

• Fisher’s experience at Rothamsted Agricultural Experiment Station influenced his vision of experimental design and helped him develop the concept of ANOVA .

• Fisher’s essay on a lady tasting tea eloquently outlines some important issues in experimental design.

To learn more, read the biographies of statisticians as you learn their techniques

The Student’s t-testStudent is the pseudonym of William Sealy Gosset, a contemporary of Fisher who worked for Guiness, the Irish brewery.

Other techniquesMany statistical techniques are named after interesting historical people:Bayes, Bernoulli, Cochran, Cox, Kolmogorov, Mann, Pearson, Smirnov, Tukey, Whitney, Wilcoxon to name just a few

You are more likely to remember specific statistical techniques if you know about the people who created them. Don’t be afraid to look at the original works published by these famous statisticians.

Examples of statistical techniques or tests named after people important in the history of statistics. Names below include: Cochran, Cox, Friedman, Gosset, Kolmogorov, Kruskal, Mann, Smirnov, Spearman, Wallace, Whitney, & Wilcoxon.

Recommended Reading

• Salsburg, D. 2002. The Lady Tasting Tea: How Statistics Revolutionized Science in the Twentieth Century. Henry Holt and Company, NY.

• Stigler, S. M. 1999. Statistics on the Table: The History of Statistical Concepts and Methods. Harvard University Press, Cambridge, MA.