If you are qualified, Fill out make-up exam form by 5pm this Friday, attach necessary documents.

Check your calendar…

3/23/12 Lecture 20 1

Interpreting the ANOVA results

Chapter 9

3/23/12 2 Lecture 20

Assumptions (prior to ANOVA)

•  Two important assumptions for ANOVA 1.  Constant variance: The variances of the k

populations are the same. –  Check this with the ratio of the largest and smallest

standard deviations, the ratio must be < 2

2.  Each of the k populations follows a normal distribution.

–  Check this by looking at QQplots for each group

3/23/12 3 Lecture 19

•  ANOVA is a good example of a situation where often a nonsignificant test is actually useful. –  Suppose we are comparing a new drug to several standard

drugs already used –  Suppose also that the new drug is less expensive to produce –  In this case, mostly what we’d like to show is that the new

drug is at least effective as the other standard drugs used –  So in this situation, a non-significant ANOVA is a great result!

•  Remember: statistical significance ≠ practical significance

Quick sidenote

3/23/12 4 Lecture 19

9.3 Interpreting ANOVA results

•  If the results are significant in ANOVA, we’d like to know explicitly which means are different

•  Remark: If insignificant in ANOVA, we don’t have to try further steps…

•  Two benefits of ANOVA 1.  Single test with single chance (α) of type I error 2.  Better estimation of error among all groups

•  By comparing all the groups simultaneously, we get a better picture of the overall error among groups.

3/23/12 5 Lecture 20

How do we know which means are different?

•  We need some "Supplementary Analysis" to tell that. –  One way is to do a check visually, using the "effect plots"

•  Scatter plot of means •  Side-By-Side Boxplots

–  Another way is to perform multiple comparison of means

•  Tukey's method •  Dunnett's method •  many more: LSD, Scheffe, Bonferroni, FDR, etc.

Actually, We can compare the means pairwise and keep the two benefits of ANOVA, we do this by adjusting the T value we use to compare the two means.

3/23/12 6 Lecture 20

Visual Check - Boxplots

3/23/12 Lecture 20 7

Visual Check – Scatterplot of Means

3/23/12 Lecture 20 8

Multiple comparisons (General Concepts)

•  How far apart do two means need to be to be statistically significant? – This value can be calculated directly similar to

what we did with confidence intervals •  It functions like a t from a t-test

– Generally, the critical value (t for example) is modified to “correct” the inflated type I error rate to keep it at the desired α level (like 0.05)

•  So instead of an α error rate for each test, we get an “family” α error rate—one rate for the entire comparison

3/23/12 9 Lecture 20

Tukey’s Method •  Controls the type I error rate directly by modifying

the T value

or if ni = nj •  qα comes from the studentized range tables

–  Table IX on pages 577-578 –  Df of qα is (k, n – k) for single-factor ANOVA

•  If the difference in two means is greater than this critical value, we say those two means are statistically significantly different

)11(2 ji nnMSEqT += α

inMSEqT α=

3/23/12 10 Lecture 20

More simply

•  Calculate T, calculate the differences

•  If < T, means are not significantly different

•  If > T, means ARE significantly different •  Remark: Tukey’s method is conservative,

sometimes its conclusion will be inconsistent with that by using ANOVA test results.

If we took a less stringent alpha level we might see some of the significant differences.

ji xx −

ji xx −

ji xx −

3/23/12 11 Lecture 20

Example—generic

•  Let’s say we are comparing 4 means with equal sample sizes of ni = 5 for all i. With an MSE of 10.

•  Looking at Table IX, we have k = 4, and Error df = n – k = 16 –  qα = 4.05

•  So,

•  Any difference of means more than 5.73 apart would be significant different

73.551005.4 ===

inMSEqT α

3/23/12 12 Lecture 20

13

Example—generic

•  Suppose that you have four treatment groups and the treatment means are:

• TRT 1: 52 • TRT 2: 63 • TRT 3: 58 • TRT 4: 54

•  Which pairs are significantly different?

3/23/12 Lecture 20

Example (from Monday’s Class) •  For the cereal example, let’s use Tukey’s method using α = 0.01

•  The means are (arranged in descending order):

•  Note, group 3’s sample size, what effect will that have on the comparisons?

4.13 ,56.14 ,55.19 ,42.27 ,5

22

11

33

44

==

==

==

==

xnxnxnxn

3/23/12 14 Lecture 20

Another Method: Using SAS code

proc glm data=cereal alpha=0.01; class design; model cases = design; means design / tukey cldiff; run;

3/23/12 15 Lecture 20

Example (cont) using SAS •  Notice it stars the pairs

that are significantly different.

•  So the only pairs that are

significantly different are: 1 and 4 2 and 4

3/23/12 16 Lecture 20

Alternate SAS code

proc glm data=cereal alpha=0.01;

class design; model cases = design; means design / tukey lines;

run;

3/23/12 17 Lecture 20

Example (cont) using SAS

•  Same information as before, differences are:

1 and 4 2 and 4

•  Notice the nice “groupings” though

3/23/12 18 Lecture 20

Dunnett’s Multiple Comparison

3/23/12 19 Lecture 20

3/23/12 Lecture 20 20

Multiple Comparison – Dunnett’s Method

3/23/12 Lecture 20 21

Dunnett’s Method •  Functions like a Tukey, just uses a different T

•  tα comes from the Dunnett’s t table –  Table X on page 579 –  Only use when one of the groups is a control group

–  Only interested in comparing the “other” groups to the control group

•  Again we take pairwise differences,

)11(),1(Ci nn

MSEknktT +−−= α

Ci xx −

3/23/12 22 Lecture 20

Example (Cereal Design) using SAS

•  Dunnett’s SAS code, pretend design 1 was the regular design already used

proc glm data=cereal alpha=0.01; class design; model cases = design; means design / dunnett(“1”) cldiff; run;

Note: lines doesn’t work with Dunnett’s

3/23/12 23 Lecture 20

Example (cont) using SAS

Dunnett's t Tests for cases NOTE: This test controls the Type I experimentwise error for

comparisons of all treatments against a control. Alpha 0.01 Error Degrees of Freedom 15 Error Mean Square 10.54667 Critical Value of Dunnett's t 3.43026 Comparisons significant at the 0.01 level are indicated by ***. Difference design Between Simultaneous 99% Comparison Means Confidence Limits 4 - 1 12.600 5.554 19.646 *** 3 - 1 4.900 -2.573 12.373 2 - 1 -1.200 -8.246 5.846

•  So if group1 was the control, only group 4 is significantly different

3/23/12 24 Lecture 20

Summary about ANOVA and Multiple Comparison

•  ANOVA (Analysis of Variances) –  Check the assumptions (constant variance/normality) –  Be able to do most of ANOVA by hand or by SAS both

•  Lots of hand calculations •  Be able to read and interpret SAS output

–  For Hw, do it either way you like, but for the exam be prepared to do both!

•  Multiple Comparison methods (ONLY when ANOVA result is significant) –  are useful in other situations, but they all involve calculating a T

value and using it to compare pairs of means –  Tukey’s is approprirate if there’s no control group; try Dunnett’s

if there is any control(s)

3/23/12 25 Lecture 20

After Class

•  Hw#8, due by 5pm next Monday

•  Start review of Exam 2 (Ch.7, 8 and Monday’s complete notes) – Practice Test 2 – Hw5-8, Lab 3 and 4

3/23/12 Lecture 20 26