Amsterdam Rehabilitation Research Center | Reade Testing significance - categorical data Martin van der Esch, PhD

Amsterdam Rehabilitation Research Center | Reade

Testing significance - categorical data

Martin van der Esch, PhD


Relative Risk and Odds Ratios


RD and RR only in prospective study

Otherwise: Odds Ratio (OR)

OR ≈ RR

large differences can occur with small groups or low prevalence (‘rare diseases)

Odds Ratio vs Relative risk


Estimation •9

5% conficence interval

Hypothesis testing•C

hi squared + continuity correction•F

isher’s exact test

Odds Ratio vs Relative Risk


Analysis of categorical data


2x2 frequency table

Categorical variables

Recovery + Recovery - Total

Intervention a b a+b

Control c d c+d

Total a+c b+d a+b+c+d


2 independent groups


For example: A new intervention for Multiple Sclerosis Patients

Is the experimental intervention better than the control intervention?



Intervention a b a+b

Control c d c+d

Total a+c b+d a+b+c+d


Chance recovery intervention group: 18 / 27 = 0.67Chance recovery control group: 6 / 23 = 0.26

•Relative Risk (RR): 0.67 / 0.26 = 2.6

•Risk Difference (RD): 67% - 26% = 41 %



Intervention 18 9 27

Controle 6 17 23

Total 24 26 50


RD = 41% RR = 2.6

Statistically significant?



Hypothesis testing•C

hi-squared (p-value)Estimation

•95% confindence interval

•RD

•RR



1) Set neutral value: H0 : no treatment effect = no difference = 0. And alternative : H1: difference

2) Compute expected values based on H0

3) Compute X2

4) Compute p-value

5) Accept or reject H0

Hypothesis testing: Chi-squared (X2)


1) H0 : RR = 1 or RD = 0

2) Compute expected values if H0 is true.


Intervention 18 9 27

Controls 6 17 23

Total 24 26 50



2)

•By hand:- Cell A:(chance recovery+ ) * (chance intervention ) * N = (24/50) * (27/50) * 50 = 13

•Or…


Intervention ? ? 27

Controls ? ? 33

Total 34 26 60



2)



3) Chi-squared

•X2 = Ʃ ((O – E)2 / E )

O = observed valuesE = expected values

And again : ‘signal / noise’



O E O-E (O-E)2 / E

Intervention * recovery+ 18 13 5 1.92

Intervention * recovery- 9 14 -5 1.79

Controls* recovery + 6 11 -5 2.27

Controls * recovery - 17 12 5 2.08

Total 50 50 0 X2 = 8.06

X2 = Ʃ ((O – E)2 / E )

3)



3) X2 = Ʃ ((O – E)2 / E ) = 8.06

4) DF (2x2 table) = 1

Table for p-value: p < 0.005

5) Reject H0 in favour of H1




Fisher’s Exact Test and continuity (Yates)

Continuity correction•F

or small sample size: X2 = ((|O-E|-½)2/E)

Fisher exact test•F

ew samples, and cell(s) with frequency < 5 •D

o not use X2, but Fisher’s exact test instead




95% CI of RD

se (p1-p2) = 0.13; RD = 0.41

95% BI = 0.41 ± 1.96 x 0.13 = [0.16 - 0.66]

We can assume for 95% that the real RD is between 16-66%

Estimation


95% CI of RR

se[ln(RR)] = 0.38 ; RR = 2.6

95% CI =ln(2.6) ± 1.96 x 0.38 = [0.22 – 1.69] Back transformation: [1.24 – 5.43]

We can assume with 95% confidence that the real RR is between 1.24 en 5.43

Estimation



Hypothesis testing: •R

D en RR: p < 0.01

Estimation: •9

5% CI RD: [0.16 – 0.66] •9

5% CI RR: [1.24 – 5.43]

•Conclusion: reject H0 in favour of H1


2 paired groups


‘Cross over trial’

For example: difference between medication A and medication B.

2 paired groups

Medication B

TotalNo improvement

Improvement

Medication A No improvement

184 54 238

Improvement 14 63 77Total 198 117 315


Hypothesis testing •M

cNemar test

Estimation •9

5% CI for RD

2 paired groups


1) Set neutral value: H0 : no treatment effect = no difference = 0. And alternative : H1: difference

2) Compute X2

3) Compute p-value

4) Reject or accept H0

Hypothesis testing: McNemar test



‘Cross over trial’

Medication B

TotalNo improvement

Improvement


184 54 238



1) H0 : RD = 0

2) X2 = (b-c)2 / (b+c)

X2= (54 – 14)2 / (54 + 14) = 23.53


Medication B

TotalNo improvement

Improvement


184 54 238



1) H0 : RD = 0

2) X2 = (b-c)2 / (b+c) F

X2= (54 – 14)2 / (54 + 14) = 23.53

3) With 1 DF p << 0.001

4) Reject H0 in favour of H1




In SPSS always with continuity correction


95% CI of RD

RD = 0.13 ; se(p1-p2) = 0.026 95% CI = 0.13 ± 1.96 x 0.026 =[0.079 – 0.18]

We can assume with 95% confidence that the real RD is between 16-66%

Estimation


>2 groups


Larger tables

Comparison of 3 different diets

> 2 groups

Lost weight?

yes no Total

Diet 1 15 28 43

Diet 2 52 46 98

Diet 3 67 74 141

Total 134 148 282


Same approach:

Difference between observed and expected values

Chi-squared test with more DF

In r x c table: •D

F = (r – 1) x (c – 1)

In ordered categories: Chi-squared test for trend

(e.g. The three diets have an incremental amount of daily calories)

> 2 groups

Amsterdam Rehabilitation Research Center | Reade36

Variable

Measurement level? nominal

ordinal

numerical

Observations? independent (unpaired)

dependent (paired)

Groups? 2 groups

>2 groups

Which test?parametric (ASSUMPTIONS!)

non-parametric


Statistical tests

Continuous Categorical Parametric Non-parametric

1 group, 1 observation

t-test Signed rank test Testing 1 measurement


Independent

t-test

Mann-Whitney U Χ2 –test (with continuity correction

Fisher exact test

2 paired groups Paired t-test Wilcoxon McNemar

>2 groups ANOVA Kruskal-Wallis X2 – test for trend if applicable

Documents

Amsterdam Rehabilitation Research Center | Reade Testing significance - categorical data Martin van der Esch, PhD