Embed Size (px)
Citation preview
The scientific methodF. Guesdon
MED610 DDPMarch 2013
Is square A darker than B?
“Checker shadow illusion”, first described by Adelson, 1995
The “scientific method” modelo Describes “best practice” method for scientific
discovery
o Developed from observation of succesful scientists
History10th Centuryo Ibn al-Haytham (Alhacen): Pioneer in experimental
optics and psychology, use of scientific method.
13th and 14th Centurieso Bacon (Collection of facts, induction)o Occam (Parsimony)
17th centuryo Descartes: Deductive method, predictionso Galileo: Experimental approach
History - Modern20th centuryo Statistical criteriao Popper: Falsification o Kuhn: non-rational aspects
Our learning aimso Reflect on what makes research scientifically
sound
o Understand what the “Scientific Method” is
o Ask if “Scientific Method” really accounts for all scientists need to do
Session plan
Problem-solving strategies
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Thinking about probabilitieso 1% of women have breast cancer (p = 0.01)
o If a woman has breast cancer, the probability of a mammogram detecting it is p = 0.8
o If a woman has no breast cancer, the probability of the test being positive is p = 0.1
Estimate the probability that a woman whose mammogram came up positive actually has cancer
From Gigerenzer, in The evolution of the mind, Dellarosa Cummins and Allen, Eds, 1988, chapter 1
Probabilistically correct answero For every 1,000 women tested, 10 will have breast
cancer and 990 won’t
o Of the 10 women with breast cancer, 8 will be diagnosed correctly by the mammogram
o Of the 990 other women, 99 will have falsely positive mammograms
o For every 1,000 women tested, 99 + 8 = 107 mammograms will be positive
o The probability that a positive mammogram indicates a true breast cancer is 8 / 107 = 7.47 %
There are two different ways of thinking
One way thoughts come to mind:
Is happy
This way of thinking uses perception and intution
Is angry
This way of thinking uses perception and intution
Another way thoughts come to mind
o Probability of having breast cancer is pc = 0.01 so of 1,000 women, we can expect 1,000 x 0.01 = 10 cases
o And therefore 1,000-10 = 990 healthy women
o The test has a rate of detection of pd = 0.8, so it should pick up 0.8 x 10 = 8 cases from the sample of 1,000
o The test has a false positive rate of p+ = 0.1, so 990 x 0.1 = 99 healthy women will also have a positive result
o So, there will be in total 99 + 8 = 107 positive results.
o If I am one of those, the probablility I have cancer is p= 8/107 ≈ 0.075
Type 1 thinking = “natural”o Automatic / intutive / effortless
o Uses perception, common sense, training (skills)
o Jumps to conclusion
o “Heuristic”
Limitations of type 1 thinking
o Perception (sensory) biases
o We tend to misjudge numerical information
o We tend to confuse the most typical with the most probable
o We seek solutions that conform with how we perceive a problem rather than how it is objectively (framing, economy of thought)
Type 2 thinking: organisedo Based on conscious processing
o Rational, analytical
o “Unatural”, difficult
o Technically accurate
o Slow or unconclusive when dealing with complex problems (social / economy etc.)
Common sense relies mostly on type 1 thinking
o Provides fast, practical answers
o Good for practical problems (hunting, farming, buying and selling, stay safe etc.)
o Easy to commmunicate or convince – “Feels right”
o Influenced by and produces common knowledge
Contemporary common knowledge
o In a given situation, people from different cultures are likley to react differently
o In a given situation, people will react differently depending on their personality traits
o Women have better verbal skills and more empathy than men
Selecting candidates: Common sense approach
o Candidates for PhD position selected by interview
o Staff believe they select the best candidates
o But they can only judge the performance of students they took in
o So how do staff know that they select correctly?
o We think common sense works because it “seems” to work
Science developped by mistrusting common sense and organising
knowledge
Hypothesiso Focused
o Generates specific predictions
o Designed to be tested rigorously
o Will be rejected as soon as it fails a single test
Theory• Broad scope
• Accommodates alternative hypotheses
• Designed to be inclusive: incorporates as many facts and explanations as possible in a unified framework
• Will be abandonned if cannot generate good hypotheses, or when a better theory is built
Problem-solving strategies:Common sense v. rational thinking
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Aristotle’s description of the motion of falling bodies
The speed of falling objects is proportional to their weights.
H
H
L
L
What happens if a light object (L) is tied to a heavy object (H)?
1. The falling speed of the tied objects should be intermediate between those that they would have individually.
2. When tied, the two objects (H+L) form a single object heavier than H, so should fall faster than H alone.
Aristotle’s description can lead to contradictory predictions
• Limited predictive value
• Can lead to alternative contradictory predictions
• Does not explain what it tries to describe
Identifying the problem1 – Galileo noted the logical inconsistency in Aristotle’s
description
2 - Observed that falling objects appear to start slowly and then accelerate
3 – Looked for supporting evidence: dents in a cushion
4 – Seeked to measure how speed increased with time and describe the relation in a manner fully consistent with measurements
How Galileo may have generated his hypothesis
He uses the most simple mathematical description of accelerated motion:
The speed (V) increases in direct proportion to time (T) since the object was dropped:
V = T
How to test this? The equation
V = T
leads to a prediction about distance fallen with time:
The distance (D) increases in
proportion to time squared (T2):
D = T2
T = 1D = α
T = 2D = 4 x α
T = 3D = 9 x α
The rolling ball experiments (1603)o Problem: Free fall is too fast
o Solution: Study balls rolling down an inclined beam
Gallileo assumed that this motion followed the same law as free fall
D1
D2 D3
Simulated Galileo data
The data does not fit perfectly the prediction
Does that means the hypothesis is wrong?
DPrediction Data
T T2 T T2
1 1 1 0.93 0.87
4 2 4 2.15 4.62
9 3 9 3.03 9.18
16 4 16 3.93 15.4
Replicating the experiment
The data is never perfectly reproducible either
Does that mean the experiment is not reliable?
DPredicted
T2
Experimental T2
♯1 ♯2 … ♯99
1 1 0.87 1.15 … 1.21
4 4 4.62 4.46 … 3.76
9 9 9.18 8.75 … 8.84
16 16 15.4 15.2 … 14.9
o The most important step when interpreting data is ask if the data is good enough to mean anything.
oMany experiments do not give “yes” or “no” answers, just “maybe” answers
Simulated Galileo data The differences are not significant, so the
data supports the prediction that
D = T2 DPredicted
T2
Experimental T2
♯1 ♯2 … ♯99
1 1 0.87 1.15 … 1.21
4 4 4.62 4.46 … 3.76
9 9 9.18 8.75 … 8.84
16 16 15.4 15.2 … 14.9
Value judgments: Interpreting data
o A researcher must interpret their data - decide what it means.
o Interpretation is informed by controls (standardisation), replication and statistical analysis
o But not fully objective, depends on assumptions
o The interpretation can be contested by other scientists (peers)
o The most important step when interpreting data is ask if the data is good enough to mean anything.
oMany experiments do not give “yes” or “no” answers, just “maybe” answers
The Scientific Method1. Observe phenomena2. Develop a hypothesis (inductive thinking)3. Derive predictions from the hypothesis
(deductive thinking).4. Test one prediction (experiment)5. Interpret the results: are they consistent with
the prediction?• If yes, the model passes the test; test another
prediction• If no, the hypothesis is proven wrong (falsified);
alter or discard hypothesis
Inductive reasoningo Imagines possible causes or mechanisms to
explain the data
o Based on recognition of patterns or trends
o Can be intuitive, subjective
o Error-prone: risks confusing correlation with causality
o Essential to make good hypotheses
Standard model of Scientific Method
Hypothesis
Data
PredictionTest
Induction
Deduc
tion
Hypotheses are at the core of the scientific method
o A hypothesis is an attempt at explaining
o Testing a hypothesis is testing our understanding
o understanding means being able to make predictions!
o This distinguishes investigative science from descriptive science (mapping, cataloguing, sequencing)
Testing hypotheses: Falsification
o Experiments must be designed so as to reveal if the hypothesis is wrong
o Experiments set up to confirm hypothesis are not informative
Karl Popper (1902-1994)
Testing to faslsify…
How would you test the following hypothesis?
“All cards that have a vowel on one side have an even number
on the other side”
4U
Testing the hypothesis
o You have a sample of 4 cards:
Which card(s) do you need to turn over to test the
hypothesis?
Write your choice(s) on a piece of paper
A J 2 7
“All cards that have a vowel on one side have an even number on the other side”
A J 2 7
Prediction:
Available cards:
Would turning card A test the hypothesis?
o What might we find if we turn over card A?1. An even number2. An odd number
o If it is and odd number, we will have learned that the hypothesis is false
Apply this reasoning to all available cards
A J 2 7
“All cards that have a vowel on one side have an even number
on the other side”
Correct choices: Cards A and 7
o If you find an odd number on the other side of A, you will know that the hypothesis is wrong
o If you find a vowel on the other side of 7, you will know that the hypothesis is wrong
Were our initial choices wrong?Why?
o Card 2 is not informative: whether there is a vowel or consonant on the other side will tell you nothing about the hypothesis - but many people choose it
o Most people choose card A but very few people choose card 7 - this shows a natural bias towards seeking confirmation, but ignores half the evidence available
Problem-solving strategies:Common sense v. rational thinking
Case study 1: The law of falling bodies
2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Pre-1984 view of peptic ulcero Erosions of the lining of the stomach or duodenumo Believed to be caused by overproduction of
stomach acidso Thought to result from lifestyle factors (stress or
excess absorption of spicy food)o Treatments were: avoiding lifestyle factors,
neutralising stomach acidity or preventing acid secretion by severing nerves
o Alleviated symptoms, did not cure the disease
Observations of bacteriao In the 1970s, fiber optic endoscopes made
possible stomach biopsies from live patientso Until then, most samples of peptic ulcer tissues
had been obtained post mortemo In the 1970s, researchers began to report
association of gram-bacillus in 80% of patients with gastric ulcers.
Problems with the new observations
o Medical texbooksk asserted that bacteria cannot live in the stomach
o The bacteria could be grown in vitro after isolation from the biopsies, preventing detailed characterisation
o They were assumed to be Pseudomonas, common contaminants of endoscopes
o All this suggested the bacteria seen in ulcer samples were not genuine hosts of the stomach
Flaws in accepted knowledgeo Warren noticed that the presence of bacteria in
his biopsies strongly correlated gastritis – suggesting an immune reaction against the bacteria
o Also, the large numbers of bacteria, their homogeneous distribution and their localisation at the top of the cell layer were inconsistent with accidental contamination
How could Warren’s hypothesis be tested?
How would you test / prove the role of bacteria in causing peptic ulcers?
Koch’s postulates1. The microbe must be found in the bodies of the
patients or diseased animals2. The microbe must be isolated from the
patients/ animals and grown outside the body3. The innoculation of the microbe grown in pure
culture should produce the disease in an experimental host
4. The same microbe shoud be re-isolated from the experimental subject after the disease develops
Read following sections of hand-out:
o The pilot study (p.2)o Isolating the bacteria (p. 3)o The data (pp. 3-4 – Ignore Fig. 3)oPresenting their results (p.5)
Think about questions 4, 1 and 5 (pp.6-7)
Pilot study (pp. 3-4): Design Marshall and Warren recruited 100 patients or
healthy volunteers undergoing endoscopy. Each participant had to complete a detailed survey on:
o Their symptomso Their lifestyle histories:
Exposure to animals Travels Dental hygiene Diet (Kentucky Fried Chicken?)
Why these questions?
Aim of the pilot study (1982)o Are there bacteria in normal stomachs?o Does the presence of bacteria correlate with
type and severity of pathology?o Can the bacteria be cultured?
Koch’s postulates1. The microbe must be found in the bodies of the
patients or diseased animals2. The microbe must be isolated from the
patients/ animals and grown outside the body3. The innoculation of the microbe grown in pure
culture should produce the disease in an experimental host
4. The same microbe shoud be re-isolated from the experimental subject after the disease develops
Testing postulate 2 (Isolation)o Attempts to grow the bacteria in vitro from 30
different biopsies failed repeatedly
o Until an accident happened: Due to an emergency, technical staff once left the petri dishes unattended for 5 days, and were then able to see bacteria. The growth in vitro was too slow for normal 2-days cultures.
Testing postulate 3 (Innoculation)
o In spite of repeated attempts, the bacteria grown in vitro did not induce the disease in model animals
o In desperation, Marshall subjected himself to a self-experimentation and injested 30 ml of aliquid culture of H. pylori
o Seven days later, he became ill
Postulate 4 – re-isolation
A silver stain of H. pylori on gastric mucus-secreting epithelial cells of Dr Marshall’s stomach biopsy taken 8 days after he drank a culture of H. pylori.
Problem-solving strategies:Common sense v. rational thinking
Case study 1: The law of falling bodies
Case study 2: The bacterial origin of peptic ulcer
How useful is the “scientific method” model?
Comparing the two case studies
o IF Galileo’s study is taken as perfect example of the scientific method, does the study of the causes of pptic ulcer devaites from it?
Is the “Scientific Method” a good model?
o Describes the rational element of scientific research
o Differentiates science from other disciplines
o Does not account for subjective or cultural aspects: How do scientists decide what to study? Who decides what to study? Qualitative (exploratory) research Role of chance discoveries
Pitfalls of the scientific methodComplex phenomena cannot always be understood – predicted - through simple hypotheses
Examples: o Complex interactions between multiple factorso Phenomena involving non-linear responses to
small changeso Common chronic diseases, weather, climate
“Science is what scientists do, and there are as many scientific methods as there are individual scientists.“
P.W. Bridgman
(Nobel Prize in Physics 1923)
Alternatives to the scientific method
1 - Exploratory research (mapping, systematic fact finding) coupled with pattern-finding approaches
• El niño
• Genome-wide associations with diseases, identification of disease markers)
2 – Modelling
Any questions?
