Class 03. The Lady Tasting Tea

R A Fisher1890-1962

Rothamstead Experimental StationCirca 1917

R A Fisher1890-1962Dr B. Muriel Bristol

An Algologist

Rothamstead Experimental StationCirca 1917

The Experiment

• Ten pairs of cups (20 cups)• Each pair has one MT the other TM• The assignment of MT or TM to cups?– is done randomly

• Double Blind– Neither Dr B Muriel Bristol nor Sir R A Fisher know

the assignment pattern used.

The use of so-called double-blind, sequential lineups in criminal cases minimizes mistaken eyewitness identifications, according to a report released today by the American Judicature Society.

“Double-blind” lineups must be conducted by an officer without knowledge of the suspect's identity.

http://www.texastribune.org/texas-dept-criminal-justice/innocence-project-of-texas/report-police-lineup-protocol-can-be-improved/

Most Police Lineup’s are not double blind…but they should be.

Double-Blind Placebo controlled is the “gold standard” for medical experiments.

• In the 1950s, surgery for angina was internal mammary artery ligation.

• In a 1955 experiment, half the patents were cut and sewed up again.– IDENTICAL OUTCOMES (immediate relief lasting 3 months…identical ekgs)

• A 1993 experiment involving arthroscopic knee surgery for arthritic knees.– A was everything. B was everything but cartilage removal. C was sham

surgery.– All three got pretty much identical results.

From the book, Predictably Irrational

The Placebo Effect.

The Placebo Effect.

If she guesses, how many of the ten will she get correct?

Our experiment will be randomized, double-blind, but there is no need for a placebo.

How many will she have to get correct in order to convince you

she’s not guessing?

H0: The Null Hypothesis

• She is guessingIndependent trials with probability P=1/2 on each

Ha: The alternative Hypothesis

• She’s skillfulP > 1/2

A one-sided alternative.

EMBS 9.1

Examples of Null Hypotheses

• He is innocent• The drug and placebo are equally effective• There is no such thing as a “hot hand” in sports.• The mean heights of Men and Woman are equal.• Smoking is unrelated to cancer.• There is no relationship between NFL outcomes

and Presidential election outcomes.

Classical Statistics• Develop and state H0 and Ha.• Specify the level of significance

– α=0.05 is most common• Identify the test statistic, design and run the experiment, calculate

the test statistic– 10 paired cups, double blind– Test statistic is number of correct

• Calculate the p-value: the probability of observing a test statistic as “extreme” as the one calculated if H0 is true.– P(#correct ≥ 8 given she’s guessing) = .055– P(#correct ≥ 9 given she’s guessing) = .011– P(10 correct given she’s guessing) = .001

• Reject H0 if p-value is ≤ α

EMBS p 367

Volunteer?

Who can tell Coke from Pepsi?

The language of classical statistics• If the p-value comes out smaller than α = 0.05– The result is statistically significant at the 0.05 level– We reject the null hypothesis.– There is only a small probability this result happened by

chance.– There is strong evidence to conclude that Ha is true.

(EMBS p 39)

Language you should not use.• If the p-value comes out smaller than α = 0.05– There is a greater than 0.95 probability Ha is true.

The language of classical statistics• If the p-value comes out smaller than α = 0.05– The result is statistically significant at the 0.05 level– We reject the null hypothesis.– There is only a small probability this result happened by

chance.– There is strong evidence to conclude that Ha is true.

(EMBS p 39)

Language you should not use.• If the p-value comes out smaller than α = 0.05– There is a greater than 0.95 probability Ha is true.

Classical Statistics Assumes H0 is true and makes probability statements about

test statistics. Classical statistics does NOT make probability statements about

either H0 or Ha.

Two Types of “errors”

• Type I. Rejecting H0 when it is true.– (getting fooled by a guesser)

• Type II. Failing to reject H0 when Ha is true.– (someone with skill not being about to convince you she’s skillful).

• We focus on Type I because we can control it (by our selection of α)

• We give less concern to Type II because it is difficult to measure (how skillful is skillful?)

• The only way to make BOTH less likely, is to increase n.

EMBS 9.2

Monday: The Normal Distribution(a lecture, notes handed out, assignment due on Wed)

Case: Wunderdog Sports Picks