January 7, 2009 - afternoon session
1
Multi-factor ANOVA and
Multiple Regression
January 5-9, 2008
Beth Ayers
January 7, 2009 - afternoon session
2
Thursday Session
• ANOVA‒ One-way ANOVA‒ Two-way ANOVA‒ ANCOVA‒ With-in subject‒ Between subject‒ Repeated measures‒ MANOVA‒ etc.
• Comparisons of different designs
January 7, 2009 - afternoon session
3
Some Terminology
• Between subjects design – each subject participates in one and only one group
• Within subjects design – the same group of subjects serves in more than one treatment‒ Subject is now a factor
• Mixed design – a study which has both between and within subject factors
• Repeated measures – general term for any study in which multiple measurements are measured on the same subject‒ Can be either multiple treatments or several
measurements over time
January 7, 2009 - afternoon session
4
With-in Subjects
• New methods are needed that do not make the assumption of no correlation (independence) of errors
• Since subjects are receiving more than one treatment in within-subjects designs, we expect outcomes to be correlated
January 7, 2009 - afternoon session
5
Why With-in Subjects Designs?
• We may want to study the change of an outcome over time
• Studying multiple outcomes for each subject allows each subject to be his or her own “control”
January 7, 2009 - afternoon session
6
Advantages
• All sources of variability between subjects is excluded from the experimental error
• Repeated measures economizes subjects, which is important when only a few subjects can be utilized for the experiment
• Increased power
January 7, 2009 - afternoon session
7
Disadvantages
• Interference/confounding‒ Order effect
‒ Connected to the position in the treatment order
‒ Carryover effect‒ Connected with the preceding treatment or
treatments
‒ Practice effect‒ Students get better with practice on preceding
treatment
• Various steps can be taken to minimize the dangers of interference
January 7, 2009 - afternoon session
8
Fixed vs. Random Factors
• Fixed factors – the levels are the same levels you would use if you repeated the experiment‒ Treatments are usually fixed factors
• Random factors – a different set of levels would be used if you repeated the experiment‒ Subjects are normally considered a random
factor
January 7, 2009 - afternoon session
9
Repeated Measures Analysis
• Repeated measures analysis is appropriate when one or more factors is a within-subjects factor
• Planned (main effect) contrasts are appropriate for both factors if there is no significant interaction
• Post-hoc comparisons can also be performed‒ Must take ® level into consideration if doing
post-hoc testing
January 7, 2009 - afternoon session
10
Assumptions of Repeated Measures
• Normal distribution of the outcome for each level of the with-in subjects factor
• Errors are assumed to be uncorrelated between subjects
• Within a subject, the multiple measurements are assumed to be correlated
• A technical condition called sphericity must be met‒ Population variances of repeated measures are
equal‒ Population correlations among all pairs of
measures are equal‒ Statistical packages can check this!
January 7, 2009 - afternoon session
11
Relation to Paired t-test
• If we have a treatment with two levels and each subject received both, a paired T-test gives the same results as a two-way ANOVA with subject and treatment as factors
January 7, 2009 - afternoon session
12
Keyboard Example
• Paired t-test results
• ANOVA results
January 7, 2009 - afternoon session
13
Example
• An experiment is conducted to compare energy requirements of three activities: running, walking, and biking
• 12 subjects are asked to run, walk, and bike a required distance and the number of kilocalories burned is measured
• The activities are done in a random order with recovery time in between
• Each subject does each activity once
January 7, 2009 - afternoon session
14
Example
• Why is random order used?
• Why can’t we used a paired t-test?
January 7, 2009 - afternoon session
15
Example
• Why is random order used?‒ Concerned about carry-over effect
• Why can’t we used a paired t-test?‒ There are three levels to the explanatory
variable
January 7, 2009 - afternoon session
16
Exploratory Analysis
January 7, 2009 - afternoon session
17
Exploratory Analysis
• Mean energy output for each activity
January 7, 2009 - afternoon session
18
Analysis
• Use Sphericity Assumed row, assuming that we’ve run the check and the assumption is met
January 7, 2009 - afternoon session
19
Contrasts
• Since there are k=3 levels of exercise, we can only do 2
• Level 1 = cycling, level 2 = walking, level 3 = running
• Can say that walking consumes more energy than cycling and that running consumes more than walking
January 7, 2009 - afternoon session
20
Comparisons
• Need to run comparisons to compare cycling to running
• The 1 vs. 3 shows us that there is a significant difference
January 7, 2009 - afternoon session
21
Mixed between/within ANOVA
• One factor is varied between subjects and the other is within subjects
• Need to check interaction
January 7, 2009 - afternoon session
22
Example
• Add gender to the previous within subjects exercise and energy consumption example
January 7, 2009 - afternoon session
23
Exploratory Analysis
January 7, 2009 - afternoon session
24
Exploratory Analysis
January 7, 2009 - afternoon session
25
Analysis
• Conclusions?
January 7, 2009 - afternoon session
26
Analysis
• Unfortunately SPSS doesn’t allow you to remove the interaction in repeated measures
• Options‒ Interpret main effects in presence of the non-
significant interaction‒ Use another statistical package
January 7, 2009 - afternoon session
27
Power
• A simple Google search for power repeated measures ANOVA turns up pages worth of online applets
• Pick one that you understand
January 7, 2009 - afternoon session
28
Name that Experimental Design
X1
X2
Level 1 Level 2
Level 1s1
s2
s3
s4
S5
s16
s17
s18
s19
s20
Level 2 s6
s7
s8
s9
s10
s21
s22
s23
s24
s25
Level 3s11
s12
s13
s14
s15
s26
s27
s28
s29
s30
X1
X2
Level 1 Level 2
Level 1s1
s2
s3
s4
s5
s1
s2
s3
s4
s5
Level 2 s1
s2
s3
s4
s5
s1
s2
s3
s4
s5
Level 3s1
s2
s3
s4
s5
s1
s2
s3
s4
s5
X1
X2
Level 1 Level 2
Level 1s1
s2
s3
s4
s5
s6
s7
s8
s9
s10
Level 2 s1
s2
s3
s4
s5
s6
s7
s8
s9
s10
Level 3s1
s2
s3
s4
s5
s6
s7
s8
s9
s10
321
January 7, 2009 - afternoon session
29
Notes on designs
• All three give interaction and main effects information, but vary in the number of subjects needed
• Two-factor repeated measures – provides good precision since all sources of variability between subjects is excluded
• Mixed design – reduce carryover effects since each subject is exposed to less treatments
• The mixed design is usually the design of choice when the researcher is studying learning and the process that influences the speed with which learning takes place
January 7, 2009 - afternoon session
30
MANOVA
• An extension of ANOVA where there is more than one dependent variable and the dependent variables can not be combined