1

STA 101: Properly Setting up and Designing a Clinical Research Study Including Power Analysis for Proper Patient Numbers

Lecturer: Dr. Daisy DaiDepartment of Medical

Research

2

Who are biostatisticians? Ashley Sherman

Phone: 816-701-1347 aksherman@cmh.edu

Daisy Dai

Phone: 816-701-5233 Email: hdai@cmh.edu

Consultation Experimental design

and sampling plan Collaboration in

presentation and publication of studies

Education Research

3

Statistical Courses SPSS 201: Using SPSS to

perform statistical tests I (Sep 23rd)

SPSS 202: Using SPSS to perform statistical tests II

SPSS 204: Using SPSS to manage data

SPSS 203: Summarize data with tables and graphs

STA 101: Properly Setting up and Designing a Clinical Research Study Including Power Analysis for Proper Patient Numbers (July 16th)

STA 102: Commonly Used

Statistical Tests in Medical Research - Part I (Aug. 20th)

STA 103: Commonly Used Statistical Nonparametric Tests in Medical Research - Part II (Nov. 5th)

4

Statistics on Scope Daisy’s statistics website is located

in “Research” tab under scope main page.

Link: http://www.childrensmercy.org/content/view.aspx?id=9740

The most useful categories are “SPSS”, “Useful links” and “Course”.

5

Why do we need sample size/power calculation in medical research?

Grant application/IRB study protocol

Peer reviewed journal publication Journal review

6

Medical Research Clinical Trials

Intervention or therapeutic Preventative

Retrospective Studies

7

Statistics Descriptive Statistics

Methods to organize and summarize information

Mean, median, max, min, frequency and proportions, etc. that summarize sample demographics

Inferential Statistics Methods to draw conclusions about a

population based on information obtained from a sample of the population

8

Population

Sample

Descriptive Statistics

Inferential Statistics

Sampling Plan

Conclusion

9

Information Collections1. Historical Data

Pro: Convenient; Save a lot of work Con: Outdated; Different Objectives and Designs;

Unknown Detailed Information

2. Census Pro: reliable, accurate and comprehensive (e.g.

Population census) Con: Time consuming; requiring more resources; difficult

to investigate all subjects in the population

3. Sampling Pro: Efficient; Less risky; exploratory; informative Caveats: Selection bias; misinterpretation; design flaw

10

Misconducts in Sampling A clinical foundation used the average weight of a sample of

professional football players to make an inference about the average weight of all adult males.

A local newspaper estimated the median income of California residents by sampling the incomes of Beverly Hills residents.

Before the presidential election in 1936, the Literary Digest magazine conducted an opinion poll and predicted that Alfred Landon, the Republican candidate, would win the election. However, Franklin Roosevelt, the democratic candidate, won by the greatest landslide in the history of presidential elections!

11

Why do we need sampling plan? Warrant Research Ethics.

Too many participants could put more subjects under risk.

Improve Research Efficiency. A un-planned study with too many participants may take

longer to finish and require more resources but miss the early opportunity to publish interesting findings.

Deliver Reliable Information. A study without sufficient subjects may lose evidence to

demonstrate potential effects, which could waste resources or generate misleading information to readers.

12

Protocol – Surgical resection for patients with gastric cancer “Sample size calculation were based on a

pre-study survey of 26 surgeons, which indicated that the baseline 5-year survival rate of D1 surgery was expected to be 20%, and an improvement in survival to 34% (14% chance) with D2 resection would be a realistic expectation. Thus 400 patients (200 in each arm) were to be randomized, providing 90% power to detect such a difference with p-value<0.05. ” [1]

[1] Cushieri et al. (1999) Patient survival after D1 and D2 resections for gastric cancer: long-term

results of the MRC randomized surgical trial. Surgical Co-operative Group.

13

Protocol – Steroid or cyclosporine for oral lichen planus “It is anticipated that in patients taking

topical steroids, the response rate at 1 month will be approximately 60%. It is anticipated that this may be raised to as much as 80% in those receiving cyclosporine. With two-sided test size 5%, power 80%, then the corresponding number of patients required is approximately 200.” [2]

[2] Poon et al. (2006) A randomized controlled trial to compare steroid with cyclosporine for the topical

treatment of oral lichen planus

14

Three Steps to Calculate Sample Size

Step 1: Establish study design and study objectives.

Step 2: Select the outcome variables.

Step 3: Collect information and determine sample size.

15

Key Elements in Sample Size Calculation

The level of statistical significance.

The anticipated clinical difference between treatment groups.

The chance of detecting the anticipated clinical difference.

16

Statistical Testing Procedures1. Null Hypothesis

- Ho: Mean_Treatment=Mean_Control

2. Alternative Hypothesis

- Ha: Mean_Treatment ≠ Mean_Control (Two-sided Test)- Ha: Mean_Treatment > Mean_Control (One-sided Test)- Ha: Mean_Treatment < Mean_Control (One-sided Test)

3. Calculate statistics

4. Make Inference

- If P-value > 0.05, then Ho holds- If P-value < 0.05, then Ha holds

17

Two Types of Decision Errors

Type I error ( ) The probability of claiming a

significant difference between two treatments that are actually in parity.

Usually = 0.05

Type II error (1- ) The probability of failing to

differentiate two treatments. Ideally, 1- 0.2.

 

Action:Support Ho

Action:Support Ha

Fact:Ho is true

Type I error

Fact:Ha is true

Type II Error

18

Effect Size ( ) The standardized difference

between means of two treatments:

CT

19

Software Commercial software: nQuery Advisor

7.0 Product Website:

http://www.statsol.ie/index.php?pageID=2 User Guide

http://www.statsol.ie/documents/nQ70_version2_manual.pdf

Free software: PS 3.0 http://biostat.mc.vanderbilt.edu/twiki/bin/vie

w/Main/PowerSampleSize

20

Compare means in two groups

Control

Test

21

Case Study: Asthma Control Test An asthma control Test has been conducted to

develop a patient-based tool for identifying patients with poorly controlled asthma.

Mean of total ACT score for the poorly controlled group (Control) is 15 and mean of total ACT score for the well controlled group (Test) is 21. Assume the standard deviation of total ACT score is 4.

The effect size between Control and Test is 5.1

4

1521

CT

22

23

nQuery Advisor

24

Compare Means

25

Compare Means

26

0.85

0.98

9 15

4 6 8 10 12 14 16

Sample Size Per Group

0.5

0.6

0.7

0.8

0.9

1.0

Pow

er

27

Compare proportions in two groups

Control

Test

28

Case Study: Asthma Control Test A researcher is interested to compare allergic

asthmatic patients versus non-allergic asthmatic patients in response to an antihistamine treatment.

After treatments, patients will evaluate their asthma status as 0-very bad, 1-bad, 2-good and 3-very good.

This researcher needs to find out the sample size and power of a study that hypothesizes 80% of allergic cohort versus 60% of non-allergic cohort will be in good or very good status.

29

nQuery Advisor - Proportion

30

Compare Proportions

31

Compare Proportions

32

Agreement Test (Kappa Score)

33

Case Study: Helmet Cure Children with flat head

syndrome will wear helmet to keep their head in shape. The diagnosis and severity of flat head vary by physicians.

A study is planned to compare the rating consistency among physicians.

Assume that 50% of reviewed cases will be diagnosed as flat head syndrome. The null hypothesis assumes only 0.4 (slight) degree of agreement between two physicians. The alternative hypothesis assumes 0.7 (strong) degree of agreement.

34

nQuery Advisor

35

Assess agreement

36

Assess Agreement

37By Julius Sim and Chris Wright

38

Sample Size Calculation for Non-parametric tests

39

What is non-parametric test? Tests that are

distribution free. Compare medians

rather than mean.

Wilcoxon Signed Rank Test

Wilcoxon Rank Sum Test

Kruskall Wallis Test We will cover these

tests in details with more examples in STA103.

40

Case Study: SeroxateneA studied was conducted to evaluate whether a new anti-depressant, Seroxatene has a benefit of pain relief. Patients (n=28) with MRI-confirmed disk herniation and symptomatic leg pain were enrolled and randomly assigned to receive Seroxatene or a placebo for 8 weeks. At the end of the study, patients were asked to provide a overall rating of their pain, relative to baseline.

Deterioration

No Change

Improvement

Marked Moderate Slight Sight Moderate Marked

-3 -2 -1 0 1 2 3

41

Pain Relieving Scores------- Seroxatene Group -------

ID Score ID Score

2 0 16 -1

3 2 17 2

5 3 20 -3

6 3 21 3

8 -2 22 3

10 1 24 0

12 3 26 2

14 3 27 -1

------- Placebo Group -------

ID Score ID Score

1 3 15 0

4 -1 18 -1

7 2 19 -3

9 3 23 -2

11 -2 25 1

13 1 28 0

42

Histograms of Pain Scores

-3 -2 -1 0 1 2 3

PainScore

0.0

0.5

1.0

1.5

2.0

2.5

Fre

qu

en

cy

Mean = 0.08Std. Dev. = 1.975N = 12

Group: Placebo

-3 -2 -1 0 1 2 3

PainScore

0

1

2

3

4

5

6

Fre

qu

ency

Mean = 1.12Std. Dev. = 2.029N = 16

Group: Seroxatene

43

Sample Size Calculation for Nonparametric Tests

Although the non-parametric tests do not reply on distribution, the corresponding sample size calculation is based on distribution.

A general rule of thumb is to compute the sample size required for a t test and add 15%.

44

Practicalities More than one primary outcome

Internal pilot studies

More than two groups

45

Rules of Thumb The level of significance needs to be

determined beforehand. One can balance the testing sensitivity and

resources by appropriately choose sample size and power.

Feel free to consult statisticians if you have questions. Here we discussed some principles in sample size calculation. More sophisticated methods are available for experimenters.

46

Summary Review research ethics. Avoid research misconducts. Raise awareness in statistical sampling

and design. Learn basic sample size and power

calculation for means, proportions and agreement.

47

Thank You For more information, visit my

websitehttp://www.childrensmercy.org/content/view.aspx?id=9740Or go to Scope -> Research ->

Statistics