Sponsored by the Clinical and Translational Science Instituteand the Department of Population Health Division of Biostatistics

Concepts on the Way from Data to Decisions

Prakash Laud, PhD

Professor of Biostatistics

04/18/23 CTSI Biostatistics 2

Speaker Disclosure

In accordance with the ACCME policy on speaker disclosure, the speaker and planners who are in a position to control the educational activity of this program were asked to disclose all relevant financial relationships with any commercial interest to the audience. The speaker and program planners have no relationships to disclose.

04/18/23 CTSI Biostatistics 3

Outline

• Example

• Some design issues

• Hypotheses formulation

• Study-to-study variation

• Testing hypotheses

• Confidence interval

• Planning a study

04/18/23 CTSI Biostatistics 4

Data Scenario

• Outcome: Drop in systolic BP

• Does Drug A reduce systolic BP?

• 36 patients treated with Drug A

• 36 untreated patients

04/18/23 CTSI Biostatistics 5

Drop in BP, Control and Treated Groups

4 6 9 1 6 6 12 17

4 -3 0 -9 3 16 5 12

1 19 4 11 12 11 9 13

6 6 3 -2 6 11 19 17

5 5 5 7 12 15 14 10

5 1 -2 9 16 7 8 7

15 18 6 -1 9 15 9 19

8 10 4 1 14 5 16 4

1 15 5 9 10 12 17 4

04/18/23 CTSI Biostatistics 6

Data Description

Control: mean=5.2 sd=5.8 Treated: mean=11.1 sd=4.6

04/18/23 CTSI Biostatistics 7

Design Issues

• More precise outcome definition needed• Patient recruitment• Inclusion of control group• Assignment to treatment arm• Assignment in matched pairs?• Balance via randomization• Randomization allows quantification of

decision uncertainties

04/18/23 CTSI Biostatistics 8

Decision Problem Formulation

• Clear statements of opposing possibilities

• Drug A reduces systolic BP

• Drug A does not reduce systolic BP

• Drug A reduces systolic BP more than does placebo

• Drug A does not reduce systolic BP more than does placebo

04/18/23 CTSI Biostatistics 9

Hypotheses and Burden of Proof

• Research hypothesis carries burden of proof

• Researcher as prosecutor

• Research hypothesis like accused is guilty

• Opposite working assumption is the Null Hypothesis (accused presumed innocent)

• Researcher’s goal is to establish evidence against the Null Hypothesis

04/18/23 CTSI Biostatistics 10

Need for Statistical Reasoning

• Generalizing: from patients in study to a much larger target population of patients

• Repeatability: what might happen if a different group of patients from the target population were in the study

• Active, deliberate randomization achieves conceptual equivalence of patient groups in repeated studies

04/18/23 CTSI Biostatistics 11

Two Kinds of Variability

• Patient-to-patient variation in response Range, spread, standard deviation of

individual levels of drop in BP

• Study-to-study variation in mean difference between control and treated groups Relevant to repeatability of studies

• The second has a mathematical relation to the first

04/18/23 CTSI Biostatistics 12

Back to DataControl: mean=5.2 sd=5.8 Treated: mean=11.1

sd=4.8

Difference in means=5.9

04/18/23 CTSI Biostatistics 13

SD and SE• Patient-to-patient variability in response

captured by SD (standard deviation)• Study-to-study variability in any study statistic

captured by SE (standard error)• Formula relating SD to SE depends on study

design and statistic chosen

2 21 2

1 2

SD SDSE

N N

04/18/23 CTSI Biostatistics 14

The Null Hypothesis

• Drug A is no better than placebo in reducing systolic BP

• Mean reduction in treated target population equals mean reduction with placebo given to all in target population

0 : 0treated placeboH

04/18/23 CTSI Biostatistics 15

Study Outcome

• Mean(treated) - Mean(placebo)=5.9

• Does this mean we have disproved H0?

• Remember H0 is about target populations, not just patients in study

• SD(treated)=4.6 SD(placebo)=5.8

• SE(mean difference)=1.24

04/18/23 CTSI Biostatistics 16

Simulation under H0

• Statistic used for testing is standardized by dividing study mean difference by SE

• Website : http://lstat.kuleuven.be/java/

04/18/23 CTSI Biostatistics 17

Decision Rule

• Reject H0 if mean difference is larger than a critical value, not just larger than 0

1 : 0treated placeboH

04/18/23 CTSI Biostatistics 18

Type I Error

• Rejecting H0 when it is true

• α = Probability ( Reject H0 | H0 true )

• α = 0.05 for BP study with decision rule: Reject H0 if mean difference > 2.04

• Mean difference in study = 5.9

• Reject H0 and conclude that Drug A reduces BP more than does the placebo

04/18/23 CTSI Biostatistics 19

More on Type I Error

• Having rejected the null hypothesis, we could have made a Type I error.

• But the decision rule controlled the probability of Type I error.

• This gives us confidence in our decision

• Decisions can be made at any set α level

• To allow others to choose their own α, we report what is called the p-value

04/18/23 CTSI Biostatistics 20

Some More on Type I Error

• With mean difference = 5.9, p-value is very small, 0.000001

• If mean difference = 1.8, p-value is 0.073

• If p-value is less than set α, reject H0

• Smaller the p-value, stronger the evidence against the null

• P-value is NOT the probability that the null hypothesis is true

04/18/23 CTSI Biostatistics 21

P-value

04/18/23 CTSI Biostatistics 22

Back to Original Data

• Mean difference = 5.9, p-value=0.000001

• Strong evidence against H0

• Strong evidence that Drug A is better than placebo

• How much better? Achieving 5.9 mmHg higher drop than placebo?

• What about study-to-study variation?

04/18/23 CTSI Biostatistics 23

Quantifying Improvement Confidence Interval

• Mean difference ± 1.96 × SE (mean diff)

• 5.9 ± 1.96 × 1.24, i.e., 5.9 ± 2.43

• 3.47 to 8.33

• This is a 95% Confidence Interval

• This interval was made using a procedure that has a 95% probability of capturing the true improvement

04/18/23 CTSI Biostatistics 24

Simulation of CI’s

• Website : http://lstat.kuleuven.be/java/

04/18/23 CTSI Biostatistics 25

Two Types of Error

Null

Hypothesis

True

Null

Hypothesis

False

Rejected

Null

Hypothesis

Type I Error Correct Decision

Did not reject

Null

Hypothesis

Correct

Decision

Type II Error

04/18/23 CTSI Biostatistics 26

Type II Error and Power

• In planning a study, setting only a level for probability of Type I Error, like α = 0.05, only protects against this error

• We also want a small probability of accepting H0 if it is false

• Equivalently, we want a large probability of rejecting H0 if it is false

• This is called Power; it depends on effect size

04/18/23 CTSI Biostatistics 27

Effect Size and Power

Need to specify a specific scenario under H1

04/18/23 CTSI Biostatistics 28

Power and Sample Size

04/18/23 CTSI Biostatistics 29

Terminology

• Treatment arm• Control arm• Randomization• Target population• Study sample• Null hypothesis• Alternative /Research

hypothesis• Standard deviation

(patient-to-patient variation)

• Standard error (study-to-study variation in statistic)

• Type I error• α = probability of Type I

error• p-value• Type II error• Power• Effect size• Sample size• Confidence interval

04/18/23 CTSI Biostatistics 30

Resources

• The Clinical and Translation Science Institute (CTSI) supports education, collaboration, and research in clinical and translational science: www.ctsi.mcw.edu

• The Biostatistics Consulting Service provides comprehensive statistical support www.mcw.edu/biostatistics.htm

•

04/18/23 CTSI Biostatistics 31

Free drop-in consulting

• MCW/Froedtert/CHW: 1 – 3 PM – Monday, Wednesday, Friday @ CTSI Administrative

offices (LL772A)– Tuesday, Thursday 1 – 3 PM @ Health Research

Center, H2400

• VA: 1st and 3rd Monday, 8:30-11:30 am– VA Medical Center, Room 6119

• Marquette: 2nd and 4th Monday, 8:30-11:30 am– Olin Engineering Room 338D