General Latent Variable Modeling Approaches to Measurement Issues using Mplus Rich Jones [email protected] [email protected] Psychometrics

General Latent Variable Modeling Approaches to

Measurement Issues using Mplus

Rich Jones [email protected]

Psychometrics WorkshopFriday Harbor, San Juan Island, WA

August 24, 2005

mailto:[email protected]

Overview• Part 1

– IRT overview– DIF overview

• Part 2– IRT via Factor Analysis– Factor analysis and general latent variable models for

measurement issues using Mplus– Limitations of Mplus approach

• Part 3– Applied Example

• Part 4 (time permitting)– Bells and Whistles– Discussion

Part 1a

IRT overview

Semantics

• Multiple Fields, Conflicting Language– Educational Testing, Psychological Measurement,

Epidemiology & Biostatistics, Psychometrics & Structural Equation Modeling

• Characteristics of People– ability, trait, state, construct, factor level, item

response

• Characteristics of Items – difficulty, severity, threshold, location– discrimination, sensitivity, factor loading,

measurement slope

Key Ideas of IRT

• Persons have a certain ability or trait• Items have characteristics

– difficulty (how hard the item is)– discrimination (how well the item measures the ability)– (I won’t talk about guessing)

• Person ability, and item characteristics are estimated simultaneously and expressed on unified metric

• Interval-level measure of ability or trait• Used to be hard to do

Some Things You Can Do with IRT

1. Refine measures2. Identify ‘biased’ test items3. Adaptive testing4. Handle missing data at the item level5. Equate measures

Latent Ability / Trait

• Symbolized with i or i

• Assumed to be continuously, and often normally, distributed in the population

• The more of the trait a person has, the more likely they are to ...whatever...(endorse the symptom, get the answer right etc.)

• The latent trait is that unobservable, hypothetical construct presumed to be measured by the test (assumed to “cause” item responses)

Item Characteristic Curve

• The fundamental conceptual unit of IRT

• Relates item responses to ability presumed to cause them

• Represented with cumulative logistic or cumulative normal forms

Item Response Function

P(yij=1|i) = F[aj(i-bj)]Example of an Item Characteristic Curve

Pro

ba

bili

ty o

f Corr

ect

Re

spo

nse

Latent Ability Distribution-3.0 -2.0 -1.0 0.0 1.0 2.0 3.0

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

babi

lity

of a

Co

rre

ct R

esp

onse

-3 -2 -1 0 1 2 3Latent Trait Level

Example of an Item Characteristic CurveP

rob

ab

ility

of C

orr

ect R

esp

on

se


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

A Person with High AbilityHas a High Probability ofResponding Correctly


robabili

ty o

f C

orr

ect

Resp

onse


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

A Person with Low AbilityHas a Low Probability ofResponding Correctly


robabili

ty o

f C

orr

ect

Resp

onse


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Item Difficulty: The level ofability at which a person hasa 50% probability ofresponding correctly.

Example of Two ICCs that Differ in DifficultyP

roba

bili

ty o

f Corr

ect

Re

spo

nse


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00


robabili

ty o

f C

orr

ect

Resp

onse


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Item Discrim ination:How well the item separatespersons of high and low ability;Proportional to the slope of theICC at the point of inflection

Example of Two ICCs that Differ in DiscrimiationP

roba

bili

ty o

f Corr

ect

Re

spo

nse


0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Item Response Function

L o g i s t i c m o d e l :

P ( Y i j = 1 | ) = e D a j ( i - b j )

1 - e D a j ( i - b j )

1

1 + e - D a j ( i - b j )

C u m u l a t i v e n o r m a l p r o b a b i l i t y m o d e l :

dta j(i b j)

P(Y ij=1|i) =-

1

2e- t

2 /2

Extra Creditone way to get estimates of underlying ability

Remember Bayes Theorem

P(AB) = P(A)P(B|A) P(AB) = P(B)P(A|B)

P(A|B) = P(A)P(B|A)

P(B)

Extra Creditone way to get estimates of underlying ability

Bayes modal estimates of latent ability ()(modal a posteriori [MAP] estimates)

likelihood function for response pattern U given ability :

g(U|) = p

iP

yi

i Q1-yi

i

a posteriori likelihood function of given pattern U:

g(|U) = ()g(U|)g(U)

Part 1b

DIF Overview

Identify Biased Test ItemsDifferential Item Functioning (DIF)

• Differences in likelihood of error to a given item may be due to – group differences in ability– item bias– both

• IRT can parse this out• Item Bias = Differential Item Function +

Rationale• Most workers in IRT identify DIF when two

groups do not have the same ICC

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

babi

lity

of a

Cor

rect

Res

pons

e


Example of group heterogeneity but no DIF

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

babi

lity

of a

Cor

rect

Res

pons

e


Example of group heterogeneity and uniform DIF

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

babi

lity

of a

Cor

rect

Res

pons

e


Example of group heterogeneity and non-uniform DIF

Part 2

IRT and Factor Analysis

IRT and Factor Analysis

• IRT describes a class of statistical models• IRT models can be estimated using factor

analysis – Appropriate routines for ordinal dependent

variables (tetrachoric/polychoric correlation coefficients)

• Factor analysis models can be extended in very general ways using structural equation modeling techniques / software

• www.statmodel.com• Used to be LISCOMP, owes lineage to LISREL• Does just about everything other continuous

latent variable / structural equation software implement (LISREL, EQS, AMOS, CALIS)

• Plus, very general latent variable modeling– Continuous latent variables (latent traits)– Categorical latent variables (latent classes, mixtures)– Missing data– Estimation with data from complex designs

• Expensive, demo version available

http://www.statmodel.com/

Mplus approach to IRT Model

• One or Two-parameter IRT models (not explicit)– Discrimination ≈ Factor loadings/slopes– Difficulty ≈ Item thresholds

• Two estimation methods– Weighted Least Squares

• Limited information • Multivariate probit (theta or delta parameterization)• Latent response variable formulation (Assume underlying

continuous variables)– Maximum Likelihood

• Full information• Multivariate logistic• Conditional probability formulation

– More experience, fit statistics with WLS– Some model types require ML, others WLS

Latent Response Variable Formulation (picture)

0

.1

.2

.3

.4

-4 -2 0 2 4

y*

y=0 y=1

Latent Response Variable Formulation (words)

• Assume observed ordinal (dichotomous) y has corresponding underlying continuous normal but unobservable (latent) form (y*)

• When a person’s value for y* exceeds some threshold (), y=1 is observed, otherwise, y=0 is observed

• Analysis is focused on relationship among the y* and estimating the thresholds ()

Latent Response Variable Formulation (equation)

(dichotomous case)

yi= 1 if y*i >0 otherwise (40)†

y*i = xi + i (42)†

P(yi=1|x) = P(y*i >|x) = 1-P(y*i |x) (43)†

P(yi=1|x) = 1-F

-x

V() = F

-+x

V() (45)†1

If we standardize to V(i)=1,

P(yi=1|x) = 1-F[-x] = F[-+x]

1 page 4 in Special Topics in Latent Variable Modeling Using Mplus (2003), which is the day 3 hand-out from the Mplus Short Course series, available

for purchase at www.statmodel.com

Conditional ProbabilityFormulation

Conditional Probability Formulation

P(yi=1|x) = F[+x] (21)†

Recall that the LRV formulation specified

P(yi=1|x) = F[-+x] (45)†1

when we standardize to V(i)=1, so we see that

=- ,

=

† Equation number from Mplus Short Course Handouts, Special Topics in Latent Variable Modeling Using Mplus (2003)

1 page 4 in Special Topics (2003)

Factor Analysis Model

1 1y

4y

1

2y 2

3

4

2

3

4y 3

*

*

*

*

y* = + VAR(y*) = '

VAR() =

assuming VAR() = 1

a = 1-2

b =

Factor Analysis Model

y* = + VAR(y*) = '

VAR() =

assuming VAR() = 1

a = 1-2

b =

0.00

0.50

1.00

P(y

=1

|)

-3 -2 -1 0 1 2 3

Factor Analysis with Covariates

x

1 1y

4y

1

2y 2

3

4

2

3

4y 3

*

*

*

*

1

1 ,1

1 ,1

1

11 , 1

MIMIC Model Multiple Indicators, Multiple Cause

y = + x +

assuming VAR() = 1, =0

a = 1-2 , b =

x

is sufficient to describe uniform DIF

Multiple Group CFA

group = 0 group = 1

1 1y

4y

1

2y 2

3

4

2

3

4y3

*

*

*

*

1 1y

4y

1

2y 2

3

4

2

3

4y3

*

*

*

*

Multiple Group (MG) MIMIC

x

1 1y

4y

1

2y 2

3

4

2

3

4y 3

*

*

*

*

1

1 ,1

1 ,1

1

11 , 1

x

1 1y

4y

1

2y 2

3

4

2

3

4y 3

*

*

*

*

1

1 ,1

1 ,1

1

11 , 1

gro u p = 0 gro u p = 1

MIMIC and MG-MIMIC Model• Disadvantages

– Not so good for factor score generation– Not exactly the IRT model

• different conceptualization of NU-DIF• Some work to get a’s b’s and standard errors

– Relatively little experience / literature in field– Confusing / overlapping measurement

noninvariance literature from SEM field

MIMIC and MG-MIMIC Model• Advantages

– Can be easy to estimate, good for modeling– No need to equate parameters– No data re-arrangements required, missing data tricks– Simultaneous analysis/evaluation of all items and

possible sources of model mis-fit (including potential DIF or bias)

– Multiple independent variables (with DIF)– Y’s and X’s can be categorical or continuous– Anchor items not necessary, but...– Embed in more complex models– Complimentary measurement noninvariance literature

from SEM field

MIMIC Model: how to do it

Title: MIMIC modelData: File is __000001.dat ;Variable: Names are y1 y2 y3 y4 x1; categorical= y1-y4 ;Analysis: type= meanstructure ;MODEL: eta by y1-y4* ; eta@1 ; eta on x1* ; y1 on x1* ;

runmplus y1-y4 x1, categorical(y1-y4) type(meanstructure) model(eta by y1-y4*; eta@1; eta on x1*; y1 on x1*;)

From within STATA using runmplus.ado

Mplus syntax file

Some Applied Examples and Technical Articles

• Muthén, B. O. (1989). Latent variable modeling in heterogeneous populations. Meetings of Psychometric Society (1989, Los Angeles, California and Leuven, Belgium). Psychometrika, 54(4), 557-585.

• McArdle, J., & Prescott, C. (1992). Age-based construct validation using structural equation modeling. Experimental Aging Research, 18(3), 87-116.

• Gallo, J. J., Anthony, J. C., & Muthén, B. O. (1994). Age differences in the symptoms of depression: a latent trait analysis. Journals of Gerontology, 49(6), 251-264.

• Salthouse, T., Hancock, H., Meinz, E., & Hambrick, D. (1996). Interrelations of age, visual acuity, and cognitive functioning. Journal of Gerontology: Psychological Sciences, 51B(6), P317-P330.

• Grayson, D. A., Mackinnon, A., Jorm, A. F., Creasey, H., & Broe, G. A. (2000). Item bias in the Center for Epidemiologic Studies Depression Scale: effects of physical disorders and disability in an elderly community sample. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 55(5), 273-282.

• Jones, R. N., & Gallo, J. J. (2002). Education and sex differences in the Mini Mental State Examination: Effects of differential item functioning. The Journals of Gerontology. Series B, Psychological Sciences and Social Sciences, 57B(6), P548-558.

• Macintosh, R., & Hashim, S. (2003). Variance Estimation for Converting MIMIC Model Parameters to IRT Parameters in DIF Analysis. Applied Psychological Measurement, 27(5), 372-379.

• Rubio, D.-M., Berg-Weger, M., Tebb, S.-S., & Rauch, S.-M. (2003). Validating a measure across groups: The use of MIMIC models in scale development. Journal of Social Service Research, 29(3), 53-68.

• Fleishman, J. A., & Lawrence, W. F. (2003). Demographic variation in SF-12 scores: true differences or differential item functioning? Med Care, 41(7 Suppl), III75-III86.

• Jones, R. N. (2003). Racial bias in the assessment of cognitive functioning of older adults. Aging & Mental Health, 7(2), 83-102.

Part 3

An Applied Example

Jones, R. N. (2003). Racial bias in the assessment of cognitive functioning of older adults. Aging & Mental Health, 7(2), 83-102.

Acknowledgement: R03 AG017680

Example: Racial bias in TICS (HRS/HEAD)

• Nationally representative, very large sample (N=15,257)

• Over-sample of Black or African-Americans (N=2,090)

• Assessment of cognition• Very adequate assessment of SES

(education, income, occupation)

Objective

• Evaluate the extent to which item level performance is due to test-irrelevant variance due to race (White, non-Hispanic vs. Black or African-American participants)

• Control for main and potentially differential effects of background variables

• Sex, Age• Educational attainment• Household income, occupation groups• Health Conditions and Health Behaviors

TICS/AHEAD Measure of Cognitive Function (Herzog 1997)

Points

• Orientation to time (weekday, day, month, year) 4

• Name President, Vice-President 2

• Name two objects (cactus, scissors) 2

• Count Backwards from 20 1

• Serial Sevens 5

• Immediate recall (10 nouns) 10

• Delayed free-recall (10 nouns, 5 min delay) 10

Background Variables

• Sex• Age (9 groups)• Education (6 groups)• Household Income (5

groups)• ‘Highest’ household

occupation (8 groups)

• Health Conditions (HBP, DM, heart, stroke, arthritis, pulmonary, cancer)

• Health Behaviors (current smoking, drinking [three groups])

y 1

y 2

y 3

y 4

y 5

6

1

fe m a le

a g e 8 5 -9 0 a g e 9 0 +a g e 7 5 -7 9 a g e 8 0 -8 4

1

g ra d e 8 g ra d e 9 -1 1 g ra d e 1 3 +

n e v e r d rin ksm o k e r

1 ,1

2 ,1

6 ,1 4

d rin k e r

a g e 6 0 -6 4 a g e 6 5 -6 9a g e 5 0 -5 4 a g e 5 5 -5 9

c le ric a l sa le sm a n a g e rs

in c o m e < $ 5 k $ 1 0 k -< 2 0 k $ 4 0 k o r m o re$ 5 k -< 1 0 k

c ra fts se rv ic e , la b o r m issin go p e ra t iv e s

d ia b e te s h e a rtH B P

stro k e lu n g d ise a se c a n c e ra rth ri t i s

g ra d e 0 g ra d e 1 -71 ,1 4

1 ,2 5

1 ,3 2

1 ,3 5

1 ,5

1 ,9

1 ,1 0

1 ,1 8

1 ,2 1

t im e

p res id en ts

n am e o b jec ts

co u n t b ck w d s

se ria l s even s

d e layed reca l l

**

**

*

1

2

3

4

5

y1

y2

y3

y4

y5

6

1

fe m a le

a g e 8 5 -9 0 a g e 9 0 +a g e 7 5 -7 9 a g e 8 0 -8 4

1



1 ,1

2 ,1

6 ,14

d rin k e r

a g e 6 0 -6 4 a g e 6 5 -6 9a g e 5 0 -5 4 a g e 5 5 -5 9



c ra ft s se rv ic e , l a b o r m issin go p e ra t iv e s


st ro k e lu n g d i se a se c a n c e ra rth ri t i s

g ra d e 0 g ra d e 1 -7 1 ,14

1 ,25

1 ,32

1 ,35

1 ,5

1 ,9

1 ,10

1 ,18

1 ,21

t im e

p res id en ts

n am e o b jec ts

co u n t b ck w d s

seria l s even s

d elayed reca ll

**

**

*

1

2

3

4

5

y1

y2

y3

y4

y5

6

1

fe m a le

a g e 8 5 -9 0 a g e 9 0 +a g e 7 5 -7 9 a g e 8 0 -8 4

1



1 ,1

2 ,1

6 ,14

d rin k e r

a g e 6 0 -6 4 a g e 6 5 -6 9a g e 5 0 -5 4 a g e 5 5 -5 9



c ra ft s se rv ic e , l a b o r m issin go p e ra t iv e s


st ro k e lu n g d i se a se c a n c e ra rth ri t i s

g ra d e 0 g ra d e 1 -7 1 ,14

1 ,25

1 ,32

1 ,35

1 ,5

1 ,9

1 ,10

1 ,18

1 ,21

t im e

p res id en ts

n am e o b jec ts

co u n t b ck w d s

seria l s even s

d elayed reca ll

**

**

*

1

2

3

4

5

W h ite (n o t H isp a n ic )

B la c k o r A fric a n A m e ric a n

Results• All items show DIF by race, some by sex,

age, education• Effect of covariates (age, occupation, income,

smoking status) significantly different across racial group

• Greater variance in latent cognitive function for Black or African-American participants

• No significant race difference in mean latent cognition by race after adjusting for measurement differences

Jones. Aging Ment Health, 2003; 7:83-102.

Differences in Underlying Ability between Whites and African Americans

• 60% is due to measurement differences (DIF, item bias)

• 12% is due to main effect of background variables

• 7% is due to structural differences (i.e., interactions of group and background variables)

• What remains (about .2 SD) is not significantly different from no difference


Differences in Underlying Ability ignoring measurement bias


0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

Den

sity

-3 -2 -1 0 1 2 3Cognitive Function Level

White

Black or African American

HRS/AHEAD data (N=15,257); Jones (2003)

Baseline model-implied distribution of cognitive functioning trait

Differences in Underlying Ability after controlling for measurement bias


0.00

0.05

0.10

0.15

0.20

0.25

0.30

0.35

0.40

0.45

0.50

Den

sity

-3 -2 -1 0 1 2 3Cognitive Function Level

White

Black or African American

HRS/AHEAD data (N=15,257); Jones (2003)

Final model-implied distribution of cognitive functioning trait

Differences in Underlying Ability after controlling for measurement bias

interaction with age group

-3

-2

-1

0

1

2

3

Leve

l of C

ogni

tive

Fun

ctio

n

50 60 70 80 90Age

WhiteB/Af. Am.

Model-Implied Age Differences in Latent Cognitive Function


LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

b. C

OR

RE

CT

resp

onse

-4 -3 -2 -1 0 1 2 3 4Level of Cognitive Function

B/A-A show n w ith dashed line. Jones (2003); data from HRS-AHEAD study (n=15,257)

Name Vice-President (Whites and Black or African-Americans)

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

b. C

OR

RE

CT

resp

onse



Second Word Recognition (Whites and Black or African-Am.)

LatentTraitDensity

0.00

0.10

0.20

0.30

0.40

0.50

0.60

0.70

0.80

0.90

1.00

Pro

b. C

OR

RE

CT

resp

onse



Fifth Serial Subraction (Whites and Black or African-Am.)

Model Fit / Parsimony

• Model fitting accomplished more than shifting group differences in mental status to item-level

• New model provides greater fit to observed data using fit statistics that reward model parsimony

Part 4

Bells and Whistles

Discussion

Latent Growth Model

= + x + y = + x +

x

y

y y y

Multiple Indicator Latent Growth Model

x

y

y y y y

y y y y

y y y y

y y y

y = + x +

= + + x +

Measurement Mixture Models

x 4 x 5

1

x 7

y 1

y 2

y p

x 6

x 1

L a t e n tC la ss

x 2 x 3

p a t t e rn mixtu rep a rt

me a s u re me n tp a rt

C ar d io /C er eb r o -v as c u la r d is eas e o r

r is k f ac to r s

Bac k g r o u n dVar iab les

D iscreteTim e S urviva l

M odel

Grow th Tra j-ec tory c lasses

Im pairm entc lasses

S lope

Leg P ow er

Trunk E ndurance

Leg P ain

B ack P a in

Leg S trength

B alance

A erob ic Capac ity

Obes ity

R ange o f M otion

P eriphera l S ensory Loss

S P P B (base line)

S P P B (year 1)

S P P B (year 2)

In tercep t

Covariates(e.g., age, sex, depression,self-effic ac y , mental state)

1 3 ty y2

y y

(e.g ., falls, m orb id ity,m ortality, d isab ility)

General Latent Variable Fram ework for Probing M echanism sLinking Im pairm ents, M obility, and Discrete Outcom es

D isc re teTim e S urviva l

M odel

Grow th Tra j-ec tory c lasses

Im pairm entc lasses

S lope

Leg P ow er

Trunk E ndurance

Leg P ain

B ack P a in

Leg S trength

B a lance

A erob ic C apac ity

Obes ity

R ange o f M otion

P eriphera l S ensory Loss

S P P B (base line)

S P P B (year 1)

S P P B (year 2)

In tercep t

Covariates(e.g., age, sex, depression,self-effic ac y , mental state)

1 3 ty y2

y y

(e.g ., falls, m orb id ity,m ortality, d isab ility)

I. La ten t C lass (P ro file M ixtu re )M ode l o f Impa irments

II. Random Coeffic ien t (la ten t growth)M ode l o f M ob ility Change

III. D iscre te T ime S urviva l M ode lo f D ista l O u tcome

General Latent Variable Fram ework for Probing M echanism sLinking Im pairm ents, M obility, and Discrete Outcom es

Part 4b

Discussion

Documents

General Latent Variable Modeling Approaches to Measurement Issues using Mplus Rich Jones [email protected] [email protected] Psychometrics