Sponsored by the Clinical and Translational Science Instituteand the Department of Population Health Division of Biostatistics
Concepts on the Way from Data to Decisions
Prakash Laud, PhD
Professor of Biostatistics
04/18/23 CTSI Biostatistics 2
Speaker Disclosure
In accordance with the ACCME policy on speaker disclosure, the speaker and planners who are in a position to control the educational activity of this program were asked to disclose all relevant financial relationships with any commercial interest to the audience. The speaker and program planners have no relationships to disclose.
04/18/23 CTSI Biostatistics 3
Outline
• Example
• Some design issues
• Hypotheses formulation
• Study-to-study variation
• Testing hypotheses
• Confidence interval
• Planning a study
04/18/23 CTSI Biostatistics 4
Data Scenario
• Outcome: Drop in systolic BP
• Does Drug A reduce systolic BP?
• 36 patients treated with Drug A
• 36 untreated patients
04/18/23 CTSI Biostatistics 5
Drop in BP, Control and Treated Groups
4 6 9 1 6 6 12 17
4 -3 0 -9 3 16 5 12
1 19 4 11 12 11 9 13
6 6 3 -2 6 11 19 17
5 5 5 7 12 15 14 10
5 1 -2 9 16 7 8 7
15 18 6 -1 9 15 9 19
8 10 4 1 14 5 16 4
1 15 5 9 10 12 17 4
04/18/23 CTSI Biostatistics 7
Design Issues
• More precise outcome definition needed• Patient recruitment• Inclusion of control group• Assignment to treatment arm• Assignment in matched pairs?• Balance via randomization• Randomization allows quantification of
decision uncertainties
04/18/23 CTSI Biostatistics 8
Decision Problem Formulation
• Clear statements of opposing possibilities
• Drug A reduces systolic BP
• Drug A does not reduce systolic BP
• Drug A reduces systolic BP more than does placebo
• Drug A does not reduce systolic BP more than does placebo
04/18/23 CTSI Biostatistics 9
Hypotheses and Burden of Proof
• Research hypothesis carries burden of proof
• Researcher as prosecutor
• Research hypothesis like accused is guilty
• Opposite working assumption is the Null Hypothesis (accused presumed innocent)
• Researcher’s goal is to establish evidence against the Null Hypothesis
04/18/23 CTSI Biostatistics 10
Need for Statistical Reasoning
• Generalizing: from patients in study to a much larger target population of patients
• Repeatability: what might happen if a different group of patients from the target population were in the study
• Active, deliberate randomization achieves conceptual equivalence of patient groups in repeated studies
04/18/23 CTSI Biostatistics 11
Two Kinds of Variability
• Patient-to-patient variation in response Range, spread, standard deviation of
individual levels of drop in BP
• Study-to-study variation in mean difference between control and treated groups Relevant to repeatability of studies
• The second has a mathematical relation to the first
04/18/23 CTSI Biostatistics 12
Back to DataControl: mean=5.2 sd=5.8 Treated: mean=11.1
sd=4.8
Difference in means=5.9
04/18/23 CTSI Biostatistics 13
SD and SE• Patient-to-patient variability in response
captured by SD (standard deviation)• Study-to-study variability in any study statistic
captured by SE (standard error)• Formula relating SD to SE depends on study
design and statistic chosen
2 21 2
1 2
SD SDSE
N N
04/18/23 CTSI Biostatistics 14
The Null Hypothesis
• Drug A is no better than placebo in reducing systolic BP
• Mean reduction in treated target population equals mean reduction with placebo given to all in target population
0 : 0treated placeboH
04/18/23 CTSI Biostatistics 15
Study Outcome
• Mean(treated) - Mean(placebo)=5.9
• Does this mean we have disproved H0?
• Remember H0 is about target populations, not just patients in study
• SD(treated)=4.6 SD(placebo)=5.8
• SE(mean difference)=1.24
04/18/23 CTSI Biostatistics 16
Simulation under H0
• Statistic used for testing is standardized by dividing study mean difference by SE
• Website : http://lstat.kuleuven.be/java/
04/18/23 CTSI Biostatistics 17
Decision Rule
• Reject H0 if mean difference is larger than a critical value, not just larger than 0
1 : 0treated placeboH
04/18/23 CTSI Biostatistics 18
Type I Error
• Rejecting H0 when it is true
• α = Probability ( Reject H0 | H0 true )
• α = 0.05 for BP study with decision rule: Reject H0 if mean difference > 2.04
• Mean difference in study = 5.9
• Reject H0 and conclude that Drug A reduces BP more than does the placebo
04/18/23 CTSI Biostatistics 19
More on Type I Error
• Having rejected the null hypothesis, we could have made a Type I error.
• But the decision rule controlled the probability of Type I error.
• This gives us confidence in our decision
• Decisions can be made at any set α level
• To allow others to choose their own α, we report what is called the p-value
04/18/23 CTSI Biostatistics 20
Some More on Type I Error
• With mean difference = 5.9, p-value is very small, 0.000001
• If mean difference = 1.8, p-value is 0.073
• If p-value is less than set α, reject H0
• Smaller the p-value, stronger the evidence against the null
• P-value is NOT the probability that the null hypothesis is true
04/18/23 CTSI Biostatistics 22
Back to Original Data
• Mean difference = 5.9, p-value=0.000001
• Strong evidence against H0
• Strong evidence that Drug A is better than placebo
• How much better? Achieving 5.9 mmHg higher drop than placebo?
• What about study-to-study variation?
04/18/23 CTSI Biostatistics 23
Quantifying Improvement Confidence Interval
• Mean difference ± 1.96 × SE (mean diff)
• 5.9 ± 1.96 × 1.24, i.e., 5.9 ± 2.43
• 3.47 to 8.33
• This is a 95% Confidence Interval
• This interval was made using a procedure that has a 95% probability of capturing the true improvement
04/18/23 CTSI Biostatistics 25
Two Types of Error
Null
Hypothesis
True
Null
Hypothesis
False
Rejected
Null
Hypothesis
Type I Error Correct Decision
Did not reject
Null
Hypothesis
Correct
Decision
Type II Error
04/18/23 CTSI Biostatistics 26
Type II Error and Power
• In planning a study, setting only a level for probability of Type I Error, like α = 0.05, only protects against this error
• We also want a small probability of accepting H0 if it is false
• Equivalently, we want a large probability of rejecting H0 if it is false
• This is called Power; it depends on effect size
04/18/23 CTSI Biostatistics 29
Terminology
• Treatment arm• Control arm• Randomization• Target population• Study sample• Null hypothesis• Alternative /Research
hypothesis• Standard deviation
(patient-to-patient variation)
• Standard error (study-to-study variation in statistic)
• Type I error• α = probability of Type I
error• p-value• Type II error• Power• Effect size• Sample size• Confidence interval
04/18/23 CTSI Biostatistics 30
Resources
• The Clinical and Translation Science Institute (CTSI) supports education, collaboration, and research in clinical and translational science: www.ctsi.mcw.edu
• The Biostatistics Consulting Service provides comprehensive statistical support www.mcw.edu/biostatistics.htm
•
04/18/23 CTSI Biostatistics 31
Free drop-in consulting
• MCW/Froedtert/CHW: 1 – 3 PM – Monday, Wednesday, Friday @ CTSI Administrative
offices (LL772A)– Tuesday, Thursday 1 – 3 PM @ Health Research
Center, H2400
• VA: 1st and 3rd Monday, 8:30-11:30 am– VA Medical Center, Room 6119
• Marquette: 2nd and 4th Monday, 8:30-11:30 am– Olin Engineering Room 338D