Class Introduction and Overview; of ANOVA, … · Review of ANOVA, Regression, and Psychological Measurement ... Analysis of Variance ... • D light D rl =0 for

Class Introduction and Overview;Review of ANOVA, Regression, and

Psychological Measurement

Introduction to Structural Equation ModelingLecture #1 – January 11, 2012

ERSH 8750: Lecture 1

Today’s Class

• Introduction and overview of the course Syllabus information

• Review of prerequisites Analysis of Variance (ANOVA) Regression Psychological Measurement

• Introduction to Mplus Conducted in Room 228 of Aderhold Hall (3rd hour of today)

ERSH 8750: Lecture 1 2

A Note About The Room Change…


Introduction to SEM: From A‐Z…

Today’s Data Set• To introduce and motivate SEM, and to review some

prerequisites, we will make use of an example data set

• Data come from a sample of 200 (79 men and 121 women) participants in a study of the role of self esteem and motivation on achievement test scores

• Participant responded to three tests/surveys: 20‐item achievement test (each item was scored right/wrong) 5‐item motivation survey (each item used a 9‐point Likert scale for

responses; 1‐9 in integers) 5‐item self esteem survey (each item used a 9‐point Likert scale for

responses; 1‐9 in integers)

• The researchers were interested in the effects of motivation and self esteem on achievement


Data File Setup

• The data file (exampledata.xls) has the following variables: ID – identification number for each respondent AchievementScore – Total score for achievement items SelfEsteemScore – Total score for self esteem items MotivationScore – Total score for motivation items SelfEsteem1‐SelfEsteem5 – Self esteem item responses Motivation1‐Motivation5 – Motivation item responses Achievement1‐Achievement20 – Achievement item responses

• Note: In order to use this file with Mplus, you must: Save it as a comma‐delimited file (.csv) Remove the first row containing the variable names


ANALYSIS OF VARIANCE REVIEW


ANOVA Refresher

• As a refresher on ANOVA, lets imagine that we are interested in the factors that have an effect on work output (denoted by Y)

• We design a two‐factor study where work outputmay be affected by: Lighting of the workplace

High or Low

Temperature Cold or Warm

• This experimental design is known as a 2‐Way ANOVA


Low Lighting

High Lighting

Cold Temperature

Warm Temperature

ANOVA Model

• Here is the 2 x 2 Factorial design:

• The ANOVA model for a respondent’s work output is


ANOVA Model

• The ANOVA model allows us to test for the presence of:

The grand mean,

A main effect associated with Temperature (At)

A main effect associated with Lighting (Bl)

An interaction effect associated with Temperatureand Lighting (AB) tl


Linear Model Assumptions

• In the ANOVA and Regression (otherwise known as the general linear model or GLM), we assume that the error terms are Independent Normally distributed ∼ 0, → for one observation

• Therefore, is a parameter that must be estimated We will come to call this a residual variance


ANOVA with Dummy Coded Variables

• The ANOVA model can also be re‐written using two dummy‐coded variables Drt and Drl Becomes a linear model (i.e., regression model)

• Drt Drt =0 for respondents in cold temperature condition Drt =1 for respondents in warm temperature condition

• Dlight Drl =0 for respondents in low lighting condition Drl =1 for respondents in high lighting condition


ANOVA with Dummy Coded Variables

• The ANOVA model then becomes:


Drl = 0Low Lighting

Drl = 1High Lighting

Drt = 0 Cold Temperature

Drt = 1 Warm Temperature

ANOVA Effects Explained

• β0 is the mean for the cold and low light condition (reference group)• The intercept

• βt is the change of the mean when comparing cold to warm temperature for a business with low lights (Simple Main Effect)

• βl is the change of the mean when comparing low to high lights for a business with a cold temperature (Simple Main Effect)

• βt*l is additional mean change that is not explained by the shift in temperature and shift and lights, when both occur (2‐Way Interaction)

• Respondents from in the same condition have the same predicted value


ANOVA and our Example

• To further demonstrate the ANOVA model, we will now use our example data to determine if there are gender differences in achievement

• Because Mplus does not dummy‐code variables for us, we will use the following code for gender: 0 = male 1 = female

• Our gender ANOVA model would then be:


ANOVA Results• The results from Mplus (note: we will replicate this

analysis in lab)

• Here:

• Represents the

predicted score of achievement when all predictors are zero In this case, predicted value for GENDER = 0 (a male)

• Represents the change in the predicted achievement score for a 1‐

unit increase of the predictor Here, the predictor can either be 0 (a male) or 1 (a female) Gives the difference between achievement scores for the genders


Analysis Input File: sem12ersh8750_L01_M01.inp

More ANOVA Results

• Using the model parameters, we can find that the predicted achievement score for men was:

0 16.860

• The predicted achievement score for women was:1 16.860 2.454 14.406

• But…was the difference significant? The p‐value listed is not from your typical ANOVA test

It is from a Wald test (we will learn about this in week 3) Because we have only one level for our IV, we can tell that women are significantly different from men Something to think about: how would we know for IVs with more than one level?


Even More ANOVA Results

• The residual variance was estimated to be: This represents the amount of variance in achievement scores not explained by gender

• To determine what the overall model would be, we would have to fit a model without gender and compare residual variances – shown below:

• Gender explained:. .

.Or 10% of the variance of achievement


REVIEW OF REGRESSION


Regression Review

• Although the ANOVA model presented previously was phrased as a regression model, we will now discuss the finer points of regression analysis

• We will now build upon our model with gender, adding self esteem and motivation as additional dependent variables:

• This is what is often called an analysis of covariance model (ANCOVA) where the slope for self esteem and for motivation is equal across genders


Regression Results

• 3.623 Predicted value of

achievement for: Men (GENDER = 0) With self esteem = 0 With motivation = 0

• .561 (ns) Indicates that the intercept for women (achievement when self esteem and

motivation are zero) is .561 points lower Is now not significant

• .201 (ns) The slope for self esteem – for every one point increase in self esteem

score, achievement goes up .201 (holding gender and motivation constant)

• .320 (p = 0.002) The slope for motivation – for every one point increase in motivation score,

achievement goes up .320 (holding self esteem and gender constant)


More Regression Results

• The estimated residual variance was

• Therefore, adding self esteem and motivation to the model helped explain an additional . .

.or

13% of the variance in achievement score

• So…are you satisfied with this result? We should check the assumption of equal slopes within group


Investigating Equal Slopes

• To investigate the assumption of equal slopes, we need to test the interaction between: Gender and motivation Gender and self esteem

∗∗

• Conditional on gender, the regression lines are:For males (GENDER = 0):

For females (GENDER = 1):


Model Results

• .185 .432 Difference in slope for self esteem for females Not significant – meaning equal slopes seems appropriate for self esteem

• .252 .234 Difference in slope for motivation for females Not significant – meaning equal slopes seems appropriate for motivation


Not Done Yet…

• The ANCOVA model indicated: A significant relationship between motivation and achievement Equal slopes within gender But non‐significant relationships between gender and achievement and between self esteem and achievement P‐value was 0.058 (close to significance)

• Given our sample size of 200, it makes sense to remove one or more non‐significant effects Will increase power for remaining IVs

• Therefore, we will remove gender Self esteem was closer to significant (although choice is somewhat arbitrary)


Model without Gender

• The new model is then:

• Now, self esteem is significant… So we think we may be done, right?

• Or are we? Can you think of anything we may have to worry about with this analysis?


REVIEW OF MEASUREMENT (RELIABILITY)


Reliability

• The general linear model (regression/ANOVA combo) assumed that all variables were measured without error

• Surveys, questionnaires, and tests are all psychological instruments that are imprecise Adding up scores is a very naïve statistical model – so the scores all have measurement error

• Related to the measurement error of a test score is the reliability of a test The GLM assumes Reliability = 1 for all variables


Measurement Models

• Measurement models can be divided into two families of models based on response format alone: Continuous responses :: Confirmatory Factor Models

• Both of these families fall under a larger framework: Generalized Linear Latent and Mixed Models Provide measurement models for other types of responses

• Other relevant families we will be discussing: Structural Equation Models :: provides estimates of correlations amongst latent variables in measurement models

Path Analysis :: simultaneous regression amongst multiple observed variables

28ERSH 8750: Lecture 1

Differences Among Measurement Models

• Fundamental difference is in unit of analysis CTT :: unit of analysis is the entire test

Sum of items = latent trait estimate Positives: Can always be done; No need for advanced computing Negatives: Restrictive assumptions; limited generalizability

CFA (and, actually, item response theory IRT):: unit of analysis is the item

Model how item response relates to latent trait Different models for different types of item response formats Provides a framework for testing adequacy of measurement models

• Each family of models has a different name for the trait: CTT :: True Score (T) CFA :: Factor Score (F) IRT :: Ability (commonly); Theta (θ)


Classical Test Theory Basics

• In CTT, the test is the unit of analysis:

True score T: best estimate of “latent trait”, mean over infinite replications of the test

Error e: mean of zero, uncorrelated with T• Variance of test scores:• Goal is to quantify reliability :: proportion of test variance accounted for by true score variance:

• Items are assumed to be exchangeable (they all count the same) More items means higher reliability, regardless of type


More Classical Test Theory

• Error is a unitary construct in CTT Error variance has been quantified in various ways Goal is to reduce error variance as much as possible

Standardization of testing conditions (reduces confounds) Aggregation of additional items (errors should cancel out)

Items are exchangeable

• Followed by generalizability theory to decompose error e.g., rater variance, person variance, time variance…

• Because each of our variables in our final analysis is a scale, we must see how reliable each happens to be


Classical Test Theory• In CTT, the TEST is the unit of analysis: Ytotal = T + e

True score T: Best estimate of ‘latent trait’: Mean over infinite replications

Error e: Expected value (mean) of 0, expected to be uncorrelated with T e’s are supposed to wash out over repeated observations

So the expected value of T is Ytotal In terms of observed test score variance:

Observed variance = true variance + error variance

• Goal is to quantify reliability Reliability = true variance / (true variance + error variance) Reliability calculation is conducted on sums across items (so type of

item is not relevant), but will require assumptions about the items…

Ytotal

TrueScore

error

?

?


Conceptualizing Reliability• CLASSICAL TRUE SCORE MODEL: Ytotal = True Score + error• The expected value (predicted value) of a person’s test score is

their “true score” If a test is reliable, then a given person should get pretty much the same

score over repeated replications (except for random “error” processes) We can’t measure everybody a multiple times (cost? retest effects?) So, we can conceptualize reliability as something that pertains to a sample

of persons instead… by writing it in terms of variances

• Var(Y) = Var(T) + Var(e)= Var(T) + Var(e) + 2Cov(T, e)= Var(T) + Var(e)

• Reliability = Var(T) / Var(Y) Proportion of variance due to “true score” out of total variance


How Only Two Scores Give Us a Reliability Coefficient in CTT Y1 = T + e1 Y2 = T + e2

,, , , , , ,

• Same as: Reliability of Y = Var(True) / Var(Y)• We express unobservable true score variance in terms of the correlation between

the two total scores and the variance of the total scores (assumed to be the same across tests)

• We now have an index of how much of the observed variance is “true” (if we believe all the assumptions)

CTT assumptions to calculate reliability:• Same true score (T) observed at both times• e1 and e2 are uncorrelated with each other and T• e1 and e2 have same variance• Y1 and Y2 have same variance


Reliability Measured by Alpha• For quantitative items (items with a scale – although used on

categorical items), this is Cronbach’s Alpha… Or ‘Guttman‐Cronbach alpha’ (Guttman 1945 > Cronbach 1951) Another reduced form of alpha for binary items: KR 20

• Alpha is described in multiple ways: Is the mean of all possible split‐half correlations Is expected correlation with hypothetical alternative form of the

same length Is lower‐bound estimate of reliability under assumption that all items are

tau‐equivalent (more about that later) As an index of “internal consistency”

Although nothing about the index indicates consistency!


Where Alpha Comes From• The sum of the item variances is given by:

Var(I1) + Var(I2) + Var(I3)…. + Var(Ik) (just the item variances)

• The variance of the sum of the items is given by the sum of ALL the item variances and covariances: Var(I1 + I2 + I3) = Var(I1) + Var(I2) + Var(I3) …

+ 2Cov(I1,I2) + 2Cov(I1,I3) + 2Cov(I2,I3) … Where does the ‘2’ come from?

Covariance matrix is symmetric Sum the whole thing to get to thevariance of the sum of the items

I1 I2 I3I1 σ1

2 σ12 σ13

I2 σ21 σ22 σ23

I3 σ31 σ32 σ32


Cronbach’s Alpha

• Numerator reduces to just the covariance among items Sum of the item variances…

Var(X) + Var(Y) = Var(X) + Var(Y) just the item variances Variance of total Y (the sum of the items)…

Var(X+Y) = Var(X) + Var(Y) + 2Cov(X,Y) PLUS covariances So, if the items are related to each other, the variance of the total Y

item sum should be bigger than the sum of the item variances How much bigger depends on how much covariance among the items – the primary index of relationship

Covariance Version:k = # items


Kuder Richardson (KR) 20: Alpha for Binary Items

• KR20 is actually the more general form of alpha• From ‘Equation 20’ in 1937 paper:

• Numerator again reduces to covariance among items… Sum of the item variances (sum of pq) is just the item variances

• Variance of the sum of the items has the covariance in it, too

• So, if the items are related to each other, the variance of the total sum should be bigger than the sum of the item variances How much bigger depends on how much covariance among the items – the primary index of relationship

k = # itemsp = prop. passingq = prop. failing


Reliability for Motivation Scale

• To calculate the reliability for motivation (assumed to be quantitative or continuous variables) we need the covariance matrix We will discuss covariances in more detail next week For now, let’s just consider these to be indices of association that have a scale

• Luck for us, Mplus will estimate covariance matrices More on how, what, and why in Week #3…

• Reliability for self esteem and achievement (although not shown) can be calculated similarly


Variances and Covariances of Motivation Scale


Calculating Alpha for Motivation Scale

• The sum of the variances is:

• The sum of TWO times the covariances is: So, the total variance is 6.772 4.749 11.521

• The estimate of reliability for the motivation scale is:

• Therefore – the motivation scale is not perfectly reliable Violates the assumption of the GLM/Regression


MOTIVATING THE USE OF STRUCTURAL EQUATION MODELNG


Not 100% Reliable: What’s the Big Deal?

• When the GLM (regression/ANOVA/all other forms) are used with variables that are not measured perfectly, several deleterious things can happen: The effects may be biased

Depends on variable type (i.e., how continuous are your measures) The standard errors may be biased

Happens any time Therefore…the effect sizes are biased…

• Basically, any conclusions you make can be drawn into question when you have not‐reliable variables used The issue is with just how reliable is reliable enough


The Answer…Don’t Use Aggregates – USE SEM

• Structural Equation Modeling seeks to determine the relationship between Latent constructs only

Latent our example: Achievement, Motivation, and Self Esteem Latent and observed constructs

From our example: how does gender factor into the model? Complex relationships between latent constructs and

observed variables Does motivation mediate the relationship between self esteem and achievement?

Do either/both mediate the relationship between gender and achievement?

• SEM is a generalization of linear modeling using observed and latent (sometimes called random) variables I tend to think of SEM as a part of a bigger picture…you will see that

SEM people think everything is part of SEM


Path Diagram of Our Regression Example

• A common way of depicting models in SEM is with a path diagram :: a pictorial representation of the statistical model Observed variables: Squares Latent variables: Circles Direct effects: Arrows with one head Indirect effects: Arrows with two heads


Achievement

Error

Self Esteem

Motivation

Path Diagram of Regression Result

A More Accurate Path Diagram


A1

A2

A3

A4

A5

A6

A7

A8

A9

A10

A11

A12

A13

A14

A15

A16

A17

A18A19 A20

Achievement

Self Esteem

Motivation

SE1 SE2 SE3 SE4 SE5

M1 M2 M3 M4 M5

Continuous Latent

Variables

Continuous Observed Variables

Confirmatory Factor Analysis

CategoricalObservedVariables

Item Response

Theory (AKA Item Factor Analysis)

Structural Equation Modeling (AKA Generalized Linear Mixed

Model)

The (Really) Big Picture • Statistical distributions are what drive the process

Each distribution is described by a set of parameters Think of the normal distribution (mean and variance)

• Each of the lines represents model parameters The statistical distribution of the boxes and circles are described by the

model parameters

• Model parameters provide constraints to the statistical distribution parameters Reduce complexity of model Provide for meaningful inference

• A model is bound by distributions assumed and, hence, the number of possible parameters We will learn statistics and path models

Both are needed to be good at SEM


Up Next…

• Quick Mplus tutorial

• Head to Room 228 and share the Windows Machines


INTRODUCTION TO MPLUS


The Mplus Statistical Package

• Mplus provides a general latent variable modeling framework that allows for combinations of: Continuous or categorical latent variables Continuous, categorical, count, nominal or censored data

• Mplus is commercial software that is available on Windows machines in the Aderhold computer labs

• Mplus is also available for purchase: Available at http://www.statmodel.com From $195 to $350 (student)

50


Mplus Data File Input Format• Mplus input files must be ASCII text based (so not binary)

Text‐based file formats: *.txt, *.dat, *.csv Not‐text‐based file formats: *.xlsx, *.sas7bdat, *.sav

• The easiest way to get data files into Mplus is to use “free‐formatting” (some type of delimiter between columns) I prefer comma‐delimited files and will only use

those in this course Cannot start with variable names in first row of data

• Typically, I store data in Excel and save as a comma‐delimited file Save As…*.csv…

(then click OK to the first question)…(then click YES to the second) Ignore the warning (click NO) to re‐save when closing the Excel Workbook


Mplus Syntax Conventions• Most syntax must have a semi‐colon end each line (;)

Exceptions: TITLE section, comments, and continuing lines

• Comments are denoted with an exclamation point (!)

• Syntax is organized by sections; headings of sections end with colons (:) TITLE:, DATA:, VARIABLE:, DEFINE:, and MODEL: are what we use this week

• Syntax cannot exceed 90 characters per row ( ‽)

• Mplus input files are typically saved with the extension *.inp

• Mplus output files are typically saved with the extension *.out Both are ASCII text (i.e., you can open with text editors)

• The default location for the data file and output file are the folder containing the input file


Mplus TITLE Section

• The TITLE section contains the label of the analysis

• You can type whatever you want here…it will appear verbatim at the top of the output file

• You do not have to terminate this section with a semi‐colon

• This section is optional


Mplus DATA Section

• The DATA section is where data files are defined

• Define the name (and path if different from input file folder) by using the command:

FILE = mydata.csv

• POTENTIAL MISTAKES: Data must be numeric – if not errors happen First row of data should not contain variable names

• This section is NOT OPTIONAL


Mplus VARIABLE Section

• The Mplus VARIABLE section defines the names of the variables in the data file, variable types, and variables in your analysis

NAMES = provides variable names Names cannot be more than 8 characters Lists of variables can be created (i.e., X1‐X10 makes 10 variables) By default all variables listed in the NAMES section are assumed to be part of the analysis

USEVARIABLE = provides names of variables used in the analysis (optional)

IDVARIABLE = provides the ID variable name (optional)

• This section is NOT OPTIONAL


Mplus DEFINE Section

• The DEFINE section is where new variables are created

• To test our equal slopes hypothesis we created interaction variables by the following syntax:


Mplus MODEL Section

• The MODEL section is where you define the model

• The only models we ran this week were GLMs These models use the ON statement (ON = REGRESSION)

achieve ON selfest motivate

• And an empty model to figure out the covariance matrix of the MOTIVATION items This used the WITH statement (WITH = COVARIANCE)

motiv1‐motive5 WITH motive1‐motive5


CONCLUDING REMARKS


Wrapping Up

• Today we covered the structure of the course, a review of the prerequisites, and an introduction to Mplus

• We will not come to lab again during class (office hours are held in this room, though)

• Please note: Homework Assignment #1 is posted on the course website and is due next Wednesday


Documents

Class Introduction and Overview; of ANOVA, … · Review of ANOVA, Regression, and Psychological Measurement ... Analysis of Variance ... • D light D rl =0 for