Chapter 13 Analysis of Variance (ANOVA) PSY295-001 Spring 2003

Chapter 13Analysis of Variance

(ANOVA)

PSY295-001PSY295-001

Spring 2003Spring 2003

Chapter 13 One-way Analysis of VarianChapter 13 One-way Analysis of Variancece

22

Major PointsMajor Points

An exampleAn example

RequirementsRequirements

The logicThe logic

CalculationsCalculations– Unequal sample sizesUnequal sample sizes

Cont.


33

Major Points--cont.Major Points--cont.

Multiple comparisonsMultiple comparisons– Fisher’s LSDFisher’s LSD– Bonferroni Bonferroni tt

Assumptions of analysis of varianceAssumptions of analysis of variance

Magnitude of effectMagnitude of effect– eta squaredeta squared– omega squaredomega squared

Review questionsReview questions


44

ExampleExample

Daily caffeine intake (DV)Daily caffeine intake (DV)

Year in school (IV)Year in school (IV)– four groupsfour groups– 6 pairs of comparison6 pairs of comparison


55

RequirementsRequirements

The DV must be quantitative (Interval or Ratio)

The IV must be between subjects (so each subject is in one and only one treatment group)

The IV must have 2 or more levels.


66

The F-ratioThe F-ratio

For the t-test we hadFor the t-test we had• t =t = mean differences between samplesmean differences between samples

standard error of the meanstandard error of the mean

for the F-ratio we use,for the F-ratio we use,• F =F = variance (differences) between meansvariance (differences) between means

variance (differences) from sampling errorvariance (differences) from sampling error


77

Another silly exampleAnother silly example

Alcohol consumption and driving Alcohol consumption and driving performance (number of trashed cones)performance (number of trashed cones)

3 levels of the IV: 1oz, 3oz, 5oz of ETOH3 levels of the IV: 1oz, 3oz, 5oz of ETOH

N=15, nN=15, n11=5, n=5, n22=5, n=5, n33=5=5

What is the null hypothesis?What is the null hypothesis?

What is the alternative?What is the alternative?


88

The dataThe data

1oz 3oz 5oz0 2 41 3 60 0 32 3 21 1 3

Mean # of cones .8 1.8 3.6


99

The stepsThe steps

Measure total (or overall) variabilityMeasure total (or overall) variability– Two componentsTwo components

between treatments variabilitybetween treatments variability

within treatments variabilitywithin treatments variability


1010

The steps The steps (continued)(continued)

Between treatments variability comprised Between treatments variability comprised of:of:– treatment effectstreatment effects– individual differencesindividual differences– experimental errorexperimental error

Within treatments variability comprised of:Within treatments variability comprised of:– individual differencesindividual differences– experimental errorexperimental error


1111

The steps The steps (continued)(continued)

Production of the F-ratioProduction of the F-ratio

F =F =Variance between treatment group meansVariance between treatment group means

Variance within treatment groupsVariance within treatment groups

OROR

F =F =Treatment Effects+Individual Differences+ErrorTreatment Effects+Individual Differences+Error

Individual Differences+ErrorIndividual Differences+Error


1212

Logic of the Analysis of Logic of the Analysis of VarianceVariance

Null hypothesis: Null hypothesis: HH00 Population means Population means

equalequal 11 = =

Alternative hypothesis: Alternative hypothesis: HH11

– Not all population means equal.Not all population means equal.

Cont.


1313

F-ratioF-ratio

Ratio approximately 1 if null trueRatio approximately 1 if null true– Ratio significantly larger than 1 if null falseRatio significantly larger than 1 if null false– ““approximately 1” can actually be as high as 2 approximately 1” can actually be as high as 2

or 3, but not much higheror 3, but not much higher


1414

Epinephrine and MemoryEpinephrine and Memory

Based on Introini-Collison & McGaugh (1986)Based on Introini-Collison & McGaugh (1986)– Trained mice to go Trained mice to go leftleft on on YY maze maze

– Injected with 0, .1, .3, or 1.0 mg/kg epinephrineInjected with 0, .1, .3, or 1.0 mg/kg epinephrine

– Next day trained to go Next day trained to go rightright in same in same YY maze maze

– dep. Var. = # trials to learn reversaldep. Var. = # trials to learn reversalMore trials indicates better retention of Day 1More trials indicates better retention of Day 1

Reflects epinephrine’s effect on memoryReflects epinephrine’s effect on memory


1515

Grand mean = 3.78


1616

CalculationsCalculations

Start with Sum of Squares (SS) Start with Sum of Squares (SS) – We need:We need:

SS SS totaltotal

SS SS withinwithin or sometimes called SS or sometimes called SS errorerror

SS SS betweenbetween or sometimes called SS or sometimes called SS groupsgroups

Compute degrees of freedom (Compute degrees of freedom (df df ))

Compute mean squares and Compute mean squares and FF

Cont.


1717

Calculations--cont.Calculations--cont.

889.83

556.132444.216

556.132)364.7(18

78.389.1...78.350.478.322.318

444.216

78.31...78.33)78.31(

)(

222

2..

222

2..

groupstotalerror

jgroups

total

SSSSSS

XXnSS

XXSS


1818

Degrees of Freedom (Degrees of Freedom (df df ))

Number of “observations” free to varyNumber of “observations” free to vary– dfdftotaltotal = = NN - 1 - 1

Variability of Variability of NN observations observations

– dfdfgroupsgroups = = kk - 1 - 1

variability of variability of gg means means

– dfdferrorerror = k = k ((nn - 1) - 1)

nn observations in each group = observations in each group = nn - 1 - 1 dfdf

times times kk groups groups


1919

Summary TableSummary Table


2020

ConclusionsConclusions

The The FF for groups is significant. for groups is significant.– We would obtain an We would obtain an FF of this size, when of this size, when HH00

true, less than one time out of 1000.true, less than one time out of 1000.– The difference in group means cannot be The difference in group means cannot be

explained by random error.explained by random error.– The number of trials to learn reversal depends The number of trials to learn reversal depends

on level of epinephrine.on level of epinephrine.

Cont.


2121

Conclusions--cont.Conclusions--cont.

The injection of epinephrine following The injection of epinephrine following learning appears to consolidate that learning appears to consolidate that learning.learning.

High doses may have negative effect.High doses may have negative effect.


2222

Unequal Sample SizesUnequal Sample Sizes

With one-way, no particular problemWith one-way, no particular problem– Multiply mean deviations by appropriate Multiply mean deviations by appropriate nnii as as

you goyou go– The problem is more complex with more The problem is more complex with more

complex designs, as shown in next chapter.complex designs, as shown in next chapter.

Example from Foa, Rothbaum, Riggs, & Example from Foa, Rothbaum, Riggs, & Murdock (1991)Murdock (1991)


2323

Post-Traumatic Stress Post-Traumatic Stress DisorderDisorder

Four treatment groups given psychotherapy– Stress Inoculation Therapy (SIT)

Standard techniques for handling stress

– Prolonged exposure (PE)Reviewed the event repeatedly in their mind

Cont.


2424

Post-Traumatic Stress Post-Traumatic Stress Disorder--cont.Disorder--cont.

– Supportive counseling (SC)Standard counseling

– Waiting List Control (WL)No treatment


2525

SIT PE SC WL3 18 24 12

13 6 14 3013 21 21 278 34 5 20

11 26 17 179 11 17 23

12 2 23 137 5 19 28

16 5 7 1215 26 27 1318 25128

10Mean 11.071 15.400 18.091 19.500St.Dev.

3.951 11.118 7.134 7.106

SIT = Stress Inoculation Therapy

PE = Prolonged Exposure

SC = Supportive Counseling

WL = Waiting List Control

Grand mean = 15.622


2626

Tentative ConclusionsTentative Conclusions

Fewer symptoms with SIT and PE than Fewer symptoms with SIT and PE than with other twowith other two

Also considerable variability within Also considerable variability within treatment groupstreatment groups

Is variability among means just a reflection Is variability among means just a reflection of variability of individuals?of variability of individuals?


2727

CalculationsCalculations

Almost the same as earlierAlmost the same as earlier– Note differencesNote differences

We multiply by We multiply by nnjj as we go along. as we go along.

MSMSerrorerror is now a is now a weighted averageweighted average. .

Cont.


2828

Calculations--cont.Calculations--cont.

889.83

8.5076.2786

8.507

62.15500.1910)62.15091.18(1162.15118.111062.15071.1114

6.2786

62.1513...62.1513)62.153(

)(

22

22

2..

222

2..

groupstotalerror

jjgroups

total

SSSSSS

XXnSS

XXSS


2929

Summary TableSummary Table

F.05(3,41) = 2.84


3030

ConclusionsConclusions

FF is significant at is significant at = .05 = .05

The population means are not all equal The population means are not all equal

Some therapies lead to greater Some therapies lead to greater improvement than others.improvement than others.– SIT appears to be most effective.SIT appears to be most effective.


3131

Multiple ComparisonsMultiple Comparisons

Significant Significant FF only shows that not all groups only shows that not all groups are equalare equal– We want to know what groups are different.We want to know what groups are different.

Such procedures are designed to control Such procedures are designed to control familywise error rate.familywise error rate.– Familywise error rate definedFamilywise error rate defined– Contrast with per comparison error rateContrast with per comparison error rate


3232

More on Error RatesMore on Error Rates

Most tests reduce significance level (Most tests reduce significance level () for ) for each each tt test. test.

The more tests we run the more likely we The more tests we run the more likely we are to make Type I error.are to make Type I error.– Good reason to hold down number of testsGood reason to hold down number of tests


3333

Fisher’s LSD ProcedureFisher’s LSD Procedure

Requires significant overall Requires significant overall F,F, or no tests or no tests

Run standard Run standard tt tests between pairs of tests between pairs of groups.groups.– Often we replace Often we replace s s 22

jj or pooled estimate with or pooled estimate with

MSMSerrorerror from overall analysis from overall analysis

It is really just a pooled error term, but with more It is really just a pooled error term, but with more degrees of freedom--pooled across all treatment degrees of freedom--pooled across all treatment groups.groups.


3434

Bonferroni Bonferroni tt Test Test

Run Run tt tests between pairs of groups, as tests between pairs of groups, as usualusual– Hold down number of Hold down number of tt tests tests– Reject if Reject if tt exceeds critical value in Bonferroni exceeds critical value in Bonferroni

tabletable

Works by using a more strict value of Works by using a more strict value of for for each comparison each comparison

Cont.


3535

Bonferroni Bonferroni tt--cont.--cont.

Critical value of Critical value of for each test set at .05/ for each test set at .05/cc, , where where cc = number of tests run = number of tests run– Assuming familywise Assuming familywise = .05 = .05– e. g. with 3 tests, each e. g. with 3 tests, each tt must be significant must be significant

at .05/3 = .0167 level.at .05/3 = .0167 level.

With computer printout, just make sure With computer printout, just make sure calculated probability < .05/calculated probability < .05/cc

Necessary table is in the bookNecessary table is in the book


3636

AssumptionsAssumptions

Assume:Assume:– Observations normally distributed within each Observations normally distributed within each

populationpopulation– Population variances are equalPopulation variances are equal

Homogeneity of variance or homoscedasticityHomogeneity of variance or homoscedasticity

– Observations are independentObservations are independent

Cont.


3737

Assumptions--cont.Assumptions--cont.

Analysis of variance is generally robust to Analysis of variance is generally robust to first twofirst two– A robust test is one that is not greatly affected A robust test is one that is not greatly affected

by violations of assumptions.by violations of assumptions.


3838

Magnitude of EffectMagnitude of Effect

Eta squared (Eta squared (22))– Easy to calculateEasy to calculate– Somewhat biased on the high sideSomewhat biased on the high side– FormulaFormula

See slide #33See slide #33

– Percent of variation in the data that can be Percent of variation in the data that can be attributed to treatment differencesattributed to treatment differences

Cont.


3939

Magnitude of Effect--cont.Magnitude of Effect--cont.

Omega squared (Omega squared (22))– Much less biased than Much less biased than 22

– Not as intuitiveNot as intuitive– We adjust both numerator and denominator We adjust both numerator and denominator

with MSwith MSerrorerror

– Formula on next slideFormula on next slide


4040

12.6.556.2786)6.55(38.507)1(

18.6.2786

8.507

2

2

errortotal

errorgroups

total

groups

MSSS

MSkSS

SS

SS

22 and and 22 for Foa, et al. for Foa, et al.

22 = .18: 18% of variability in = .18: 18% of variability in symptoms can be accounted for by symptoms can be accounted for by treatmenttreatment

22 = .12: This is a less biased = .12: This is a less biased estimate, and note that it is 33% estimate, and note that it is 33% smaller.smaller.


4141

Review QuestionsReview Questions

Why is it called the analysis of “variance” Why is it called the analysis of “variance” and not the analysis of “means?”and not the analysis of “means?”

What do we compare to create our test?What do we compare to create our test?

Why would large values of Why would large values of FF lead to lead to rejection of rejection of HH00??

What do we do differently with unequal What do we do differently with unequal n n ’s?’s?

Cont.


4242

Review Questions--cont.Review Questions--cont.

What is the per comparison error rate?What is the per comparison error rate?

Why would the familywise error rate Why would the familywise error rate generally be larger than .05 unless it is generally be larger than .05 unless it is controlled?controlled?

Most instructors hate Fisher’s LSD. Can Most instructors hate Fisher’s LSD. Can you guess why?you guess why?– Why should they not hate it?Why should they not hate it?

Cont.


4343

Review Questions--cont.Review Questions--cont.

How does the Bonferroni test work?How does the Bonferroni test work?

What assumptions does the analysis of What assumptions does the analysis of variance require?variance require?

What does “robust test” mean?What does “robust test” mean?

What do we mean by “magnitude of What do we mean by “magnitude of effect?”effect?”

What do you know if What do you know if 22 is .60? is .60?

Documents

Chapter 13 Analysis of Variance (ANOVA) PSY295-001 Spring 2003