BS704 Class 8Analysis of Variance

HW Set #7Chapter 7Problems 5, 14, 19 and 28R Problem Set 7 (on Blackboard)Due November 2

Please complete Quiz 9 Before Nov 2

An RCT to Assess the Efficacy of a New Drug for Asthma in Children Background characteristics

Age Sex Years since diagnosis of asthma

Outcomes Self-reported improvement in symptoms FEV1

Did the randomization work?

Yes No

0%0%

1. Yes2. No

Characteristic Placebo New Drug pAge, years 10 (2.4) 9.9 (2.1) .76 % Male 54% 43% .04Yrs since Dx 3.4 (1.9) 3.1 (2.1) .34

What are hypotheses to compare ages?

H0:m1=

m2 vs H

1:m...

0% 0%0%0%

Characteristic Placebo New Drug pAge, years 10 (2.4) 9.9 (2.1) .76 % Male 54% 43% .04Yrs since Dx 3.4 (1.9) 3.1 (2.1) .34

1. H0:m1=m2 vs H1:m1≠m2

2. H0:p1=p2 vs H1:p1≠p2

3. H0:m=10 vs H1:m≠104. H0:md=0 vs H1:md≠0

What test would be used to compare % improvement between groups?

Test fo

r equali

ty of ..

.

Test fo

r equali

ty of p

...

Test fo

r mea

n diffe

rence

No clue

0% 0%0%0%

1. Test for equality of means2. Test for equality of

proportions3. Test for mean difference4. No clue

What test would be used to compare FEV1 between groups?

Test fo

r equali

ty of ..

.

Test fo

r equali

ty of p

...

Test fo

r mea

n diffe

rence

No clue

0% 0%0%0%

1. Test for equality of means2. Test for equality of

proportions3. Test for mean difference4. No clue

Objectives Understand the procedure for testing

the equality of k > 2 means Perform the test by hand and using R Appropriately interpret results

Hypothesis Testing Procedures1. Set up null and research

hypotheses, select a2. Select test statistic3. Set up decision rule4. Compute test statistic5. Draw conclusion & summarize

significance (p-value)

Hypothesis Testing for More than 2 Means - Analysis of Variance Continuous outcome k Independent Samples, k > 2

H0: m1=m2=m3 … =mk

H1: Means are not all equalTest Statistic

(Find critical value in Table 4)

k)/(N)XΣΣ(X1)/(k)XX(Σn

F 2j

2jj

=

Test Statistic - F Statistic Comparison of two estimates of variability in

data Between treatment variation, is based on the

assumption that H0 is true (i.e., population means are equal)

Within treatment, Residual or Error variation, is independent of H0 (i.e., we do not assume that the population means are equal and we treat each sample separately)

F Statistic

k)/(N)XΣΣ(X1)/(k)XX(Σn

F 2j

2jj

=

Difference BETWEEN each group mean and overall mean

Difference between each observation and its group mean (WITHIN group variation - ERROR)

F Statistic

F = MSB/MSE

MS = Mean Square

What values of F that indicate H0 is likely true?

Decision Rule

Reject H0 if F > Critical Value of F with df1=k-1 and df2=N-k from Table 4

k= # comparison groupsN=Total sample size

ANOVA TableSource of Sums of MeanVariation Squares df Squares F

BetweenTreatments k-1 SSB/k-1

MSB/MSE

Error N-k SSE/N-k

Total N-1

)X - X( n Σ = SSB j2

j

)X - X( Σ Σ = SSE j2

)X -X( Σ Σ = SST 2

ExampleIs there a significant difference in mean weight loss among 4 different diet programs? (Data are pounds lost over 8 weeks)

Low-Cal Low-Fat Low-Carb Control8 2 3 29 4 5 26 3 4 -17 5 2 03 1 3 3

ExampleSummary Statistics on Weight Loss by

Treatment

Low-Cal Low-Fat Low-CarbControl

n 5 5 5 5Mean 6.6 3.0 3.4 1.2

Overall Mean = 3.6

Is there a statistically significant difference in weight loss programs?

Yes No ??

0% 0%0%

1. Yes2. No3. ??

Example1. H0: m1=m2=m3=m4

H1: Means are not all equal a=0.05

2. Test statistic

k)/(N)XΣΣ(X1)/(k)XX(Σn

F 2j

2jj

=

Example3. Decision rule

df1=k-1=4-1=3df2=N-k=20-4=16

Reject H0 if F > 3.24

Example

)X - X( n Σ = SSB j2

j

=5(6.6-3.6)2+5(3.0-3.6)2+5(3.4-3.6)2+5(1.2-3.6)2

= 75.8

Example)X - X( Σ Σ = SSE j

2

Low-Cal (X-6.6) (X-6.6)2

8 1.4 2.09 2.4 5.86 -0.6 0.47 0.4 0.23 -3.6 13.0

Total 0 21.4

Example)X - X( Σ Σ = SSE j

2

Low-Fat (X-3.0) (X-3.0)2

2 -1.0 1.04 1.0 1.03 0 05 2.0 4.01 -2.0 4.0

Total 0 10.0

Example)X - X( Σ Σ = SSE j

2

Low-Carb (X-3.4) (X-3.4)2

3 -0.4 0.25 1.6 2.64 0.6 0.42 -1.4 2.03 -0.4 0.2

Total 0 5.4

Example)X - X( Σ Σ = SSE j

2

Control (X-1.2) (X-1.2)2

2 0.8 0.62 0.8 0.6-1 -2.2 4.80 -1.2 1.43 1.8 3.2

Total 0 10.6

Example

)X - X( Σ Σ = SSE j2

=21.4 + 10.0 + 5.4 + 10.6 = 47.4

ExampleSource of Sums of MeanVariation Squares df Squares F

Between 75.8 3 25.3 8.43Treatments

Error 47.4 16 3.0

Total 123.2 19

Example4. Compute test statistic

F=8.43

5. Conclusion. Reject H0 because 8.43 > 3.24. We have statistically significant evidence at a=0.05 to show that there is a difference in mean weight loss among 4 different diet programs.

ANOVA Using R.csv data file

ExampleAn investigator wishes to compare the average time to relief of headache pain under three distinct medications, A, B and C.

Fifteen patients who suffer from chronic headaches are randomly selected for the investigation.

The outcome is time to pain relief, in minutes.

One Way ANOVARCT to Compare 3 Medications for

Chronic Pain N=15Randomize

A BC

Outcome: Time to Pain Relief, minutes

One Way ANOVA (cont’d)Data

Drug A Drug B Drug C30 25 1535 20 2040 30 2525 20 2035 30 20

Mean 33.0 25.0 20.0

One Way ANOVA (cont’d)1. Hypotheses

H0: m1 = m2 = m3

H1: means not all equal a=0.05

2. Test Statistic F

One Way ANOVA (cont’d)3. Decision Rule

K-1=3-1=2, N-k=15-3=12Reject H0 if F > 3.89

4. Compute Sums of Squares

One Way ANOVA (cont’d)

)X - X( n Σ = SSB j2

j

26.0 = 3

20.0) + 25.0 + (33.0=X..

= 5((33-26.0)2 + (25-26.0)2 + (20-26.0)2) = 430

)X - X( Σ Σ = SSE j2

One Way ANOVA (cont’d)Drug A

X (X-33) (X-33)2

30 -3 935 -2 440 7 4925 -8 6435 -2 4

0 130

One Way ANOVA (cont’d)Drug B

X (X-25) (X-25)2

25 0 120 -5 2530 5 2520 -5 2530 5 25

0 100

One Way ANOVA (cont’d)Drug C

X (X-20) (X-20)2

15 -5 2520 0 025 5 2520 0 020 0 0

0 50

One Way ANOVA (cont’d)

= 130+100+50 = 280

Source SS df MS FBetween 430.0 2 215 9.21Error 280.0 12 23.3Total 710.0 14

)X - X( Σ Σ = SSE j2

One Way ANOVA (cont’d)

Reject H0 since 9.21 > 3.89 – Means are not all equal.

Paper – Testosterone Replacement Study design?

RCT Number of comparison groups?

-placebo, no exercise-testosterone, no exercise-placebo and exercise -testosterone and exercise

Primary outcomes? Change in muscle strength, body weight, muscle

volume, lean body mass (continuous)

Paper – Testosterone Replacement Objective is to compare mean change in

muscle strength, body weight, muscle volume, lean body mass (One at a time) across four treatment groups

Figure 1 – generalizability?

Paper – Testosterone Replacement Table 1 – what tests were used?

Table 2 – what tests were used?

Practice Problem – Complete the ANOVA TableH0: m1=m2=m3=m4=m5

H1: means not all equal a=0.05

Source SS df MS FBetweenWithin 50 2.5Total 225

Practice Problem – Complete the ANOVA TableH0: m1=m2=m3=m4=m5H1: means not all equal a=0.05

Source SS df MS FBetween 100 4 25

10Within 125 50 2.5Total 225Reject H0 if F > F0.05(4,50)=2.56

ANOVA When the sample sizes are equal, the

design is said to be balanced Balanced designs give greatest power

and are more robust to violations of the normality assumption

Extensions Multiple Comparison Procedures –

Used to test for specific differences in means after rejecting equality of all means

Higher-Order ANOVA - Tests for differences in means as a function of several factors

Extensions Repeated Measures ANOVA - Tests for

differences in means when there are multiple measurements in the same participants (e.g., measures taken serially in time)

Multiple Comparisons Procedures (MCPs)If we reject H0 in an ANOVA – we

conclude that the k means are not all equal. Which means are different?

Pairwise comparisons H0: mi=mj

General contrasts H0: (mi+mj)/2=mk

MCPs (continued) With k treatments there are k(k-1)/2

possible pairwise comparisons The overall Type I error rate can be as

large as a{k(k-1)/2}!

There are a number of different MCPs – they differ in terms of treatment of Type I error rate

MCPs (continued)Error rate per comparison (ER_PC) = P(Type I error) on any one test or

comparison (usually ER_PC is 0.05).

Error rate per experiment (ER_PE) =the number of Type I errors we

expect to make in any experiment under H0 (in 100 tests, we expect to make 5 Type I errors = #tests(a)).

MCPs (continued)Familywise error rate (FW_ER)=P(at least 1 Type I error) in experiment.

FW_ER =1 - (1-ai)c,

where ai is the ER_PC c=# contrasts in experiment.

MCPs (continued)Example. Suppose we test the equality of 5

treatment means using ANOVA and the null hypotheses is rejected at a = 0.05.

Suppose that it is of interest to perform all pairwise comparisons.

There are k(k-1)/2 = 5(5-1)/2 = 10 distinct pairwise comparisons.

MCPs (continued)Suppose we wish to conduct each comparison at a

5% level of significance.

NOTE: Only tests that are of substantive interest should be run and not all possible tests.

ER_PC = 0.05. ER_PE = 10(0.05) = 0.5.FW_ER = 1 - (1 - 0.05)10 = 0.401.

Scheffe MCP Conservative procedure that controls

familywise error rate regardless of the number of contrasts

Handles both pairwise and general contrasts Other MCPs include the Tukey procedure,

Duncan procedure (multiple range test), Fisher's Least Significant Difference, the Newman-Keuls test, and Dunnett's test (used to compare a control to several active treatments).

Scheffe MCPFor pairwise tests

H0: mi = mj H1: mi ≠ mj

Reject H0 if F > (k-1) Fa (k-1, N-k)

n1 +

n1 MSE

)X - X( = F

ji

2.j.i

One Way ANOVA-ScheffeIn Example we determined 3 drugs were significantly different with respect to mean time to pain relief

Drug A Drug B Drug CMean 33.0 25.0 20.0

Which drugs are different?

Scheffe Test – Drug A Vs. BH0: mA = mB H1: mA ≠ mB

Reject H0 if F > (k-1) Fa (k-1, N-k) (k-1) F 0.05 (2,12) = 2(3.89) = 7.78

n1 +

n1 MSE

)X - X( = F

BA

2BA

Scheffe Test – Drug A Vs. B

87.6

51

513.23

)0.250.33(

n1 +

n1 MSE

)X - X( = F2

BA

2BA =

=

Do not reject H0 since 6.87<7.78. No significant difference in mean times to pain relief for Drugs A and B.

Scheffe Test – Drug A Vs. CH0: mA = mC H1: mA ≠ mC

Reject H0 if F > (k-1) Fa (k-1, N-k) (k-1) F 0.05 (2,12) = 2(3.89) = 7.78

n1 +

n1 MSE

)X - X( = F

CA

2CA

Scheffe Test – Drug A Vs. C

13.18

51

513.23

)0.200.33(

n1 +

n1 MSE

)X - X( = F2

CA

2CA =

=

Reject H0 since 18.13>7.78. Significant difference in mean times to pain relief for Drugs A and C.

Scheffe Test – Drug B Vs. CH0: mB = mC H1: mB ≠ mC

Reject H0 if F > (k-1) Fa (k-1, N-k) (k-1) F 0.05 (2,12) = 2(3.89) = 7.78

n1 +

n1 MSE

)X - X( = F

CB

2CB

Scheffe Test – Drug B Vs. C

68.2

51

513.23

)0.200.25(

n1 +

n1 MSE

)X - X( = F2

CB

2CB =

=

Do not reject H0 since 2.68<7.78. No significant difference in mean times to pain relief for Drugs B and C.

Overall conclusion??

Tukey Test in RANOVA Pairwise Tests using Tukey MCP

Only significant result is A vs C