Bayesian factor and structural equation Bayesian factor and structural equation models in spatial applications. models in spatial applications.

Specification, identification and model Specification, identification and model assessment, with case study assessment, with case study

illustrationsillustrations

Peter Congdon, Queen Mary Peter Congdon, Queen Mary University of LondonUniversity of London

Dept of Geography & Centre for Dept of Geography & Centre for StatisticsStatistics

OutlineOutline Background: Bayesian approaches to LV Background: Bayesian approaches to LV

models, advantages & disadvantagesmodels, advantages & disadvantages Computational options including Computational options including

WINBUGSWINBUGS Wider application contexts of Bayesian LV Wider application contexts of Bayesian LV

& SEM models& SEM models Spatial Priors; Common Spatial FactorsSpatial Priors; Common Spatial Factors

Outline (continued)Outline (continued)

Different sorts of spatial factor model Different sorts of spatial factor model (depending on form of manifest variables) (depending on form of manifest variables) and possible identification issuesand possible identification issues

Assessing models, model fit & model Assessing models, model fit & model choice. Possible variable/model choice choice. Possible variable/model choice approachesapproaches

Case studiesCase studies

Case StudiesCase Studies Social capital & mental health, multilevel Social capital & mental health, multilevel

model using Health Survey for England model using Health Survey for England (HSE)(HSE)

Multilevel model for joint prevalence of Multilevel model for joint prevalence of obesity & diabetes, BRFSS respondents obesity & diabetes, BRFSS respondents nested within US counties & states (CDC nested within US counties & states (CDC Behavioral Risk Factor Surveillance Behavioral Risk Factor Surveillance System)System)

Suicide & self-harm, ecological study for Suicide & self-harm, ecological study for small areas (wards) in Eastern Englandsmall areas (wards) in Eastern England

BackgroundBackground SEM and factor models originate in (& still SEM and factor models originate in (& still

most widely used) in psychological, most widely used) in psychological, educational & behavioural applications. educational & behavioural applications.

Recent Bayesian applications to Recent Bayesian applications to psychological & education testing data psychological & education testing data include SEM (e.g. Lee & Song, 2003), LCA, include SEM (e.g. Lee & Song, 2003), LCA, item analysis, and factor analysis per se (e.g. item analysis, and factor analysis per se (e.g. Aitkin & Aitkin, 2005; Press & Shigemasu, Aitkin & Aitkin, 2005; Press & Shigemasu, 1998). 1998).

Also some work on automated Bayesian Also some work on automated Bayesian model choice in normal linear factor modelmodel choice in normal linear factor model

Advantages of Bayesian ApproachAdvantages of Bayesian Approach

Straightforward to depart from standard Straightforward to depart from standard assumptions often built into classical assumptions often built into classical estimation methods (e.g. factor scores estimation methods (e.g. factor scores multivariate normal & independent over multivariate normal & independent over subjects)subjects)

Advantage in generalizations such as Advantage in generalizations such as nonlinear factor effects, multiplicative factor nonlinear factor effects, multiplicative factor schemesschemes

Advantages of Bayesian Approach Advantages of Bayesian Approach (continued)(continued)

Random effect models (of which Random effect models (of which factor/SEM models are subclass) can be factor/SEM models are subclass) can be fitted without relying on numerical methods fitted without relying on numerical methods to integrate out random effectsto integrate out random effects

Potential for Bayesian model choice Potential for Bayesian model choice procedures (e.g. stochastic search procedures (e.g. stochastic search variable selection) in factor/SEM modelsvariable selection) in factor/SEM models

Disadvantages of Bayesian ApproachDisadvantages of Bayesian Approach

Identification issues (re “naming” of Identification issues (re “naming” of factors): can have label switching for latent factors): can have label switching for latent constructs during MCMC updating if there constructs during MCMC updating if there aren’t constraints to ensure consistent aren’t constraints to ensure consistent labelling.labelling.

Slow convergence of model parameters or Slow convergence of model parameters or global model fit measures (e.g. DIC and global model fit measures (e.g. DIC and effective parameter estimate) in large effective parameter estimate) in large latent variable applications (e.g. 1000 or latent variable applications (e.g. 1000 or 10000 subjects)10000 subjects)

Disadvantages of Bayesian ApproachDisadvantages of Bayesian Approach

Formal Bayes model assessment Formal Bayes model assessment (marginal likelihoods/Bayes factors) (marginal likelihoods/Bayes factors) difficult for large realistic applicationsdifficult for large realistic applications

Sensitivity to priors on hyperparameters Sensitivity to priors on hyperparameters (e.g. priors for factor covariance matrix)(e.g. priors for factor covariance matrix)

Bayesian approach may need sensible Bayesian approach may need sensible priors when applied to factor models priors when applied to factor models (“diffuseness“ not necessarily suitable)(“diffuseness“ not necessarily suitable)

Bayesian ComputingBayesian Computing Many Bayesian applications to SEM and Many Bayesian applications to SEM and

factor analysis facilitated by WINBUGS factor analysis facilitated by WINBUGS package. package.

See Congdon (Applied Bayesian See Congdon (Applied Bayesian Modelling, 2003); Lee (Structural Equation Modelling, 2003); Lee (Structural Equation Modeling: a Bayesian Approach, 2007)Modeling: a Bayesian Approach, 2007)

Alternative is R…more programming Alternative is R…more programming involvedinvolved

BayesX can’t model common factorsBayesX can’t model common factors

WINBUGSWINBUGS

Despite acronym, WINBUGS employs Despite acronym, WINBUGS employs Metropolis-Hastings updating where Metropolis-Hastings updating where necessary as well as Gibbs samplingnecessary as well as Gibbs sampling

Program code is essentially a description Program code is essentially a description of the priors & likelihood, but can monitor of the priors & likelihood, but can monitor model-related quantities of interestmodel-related quantities of interest

Computing Illustration: a Normal Computing Illustration: a Normal SEMSEM

Wheaton Study: 3 latent variables, each Wheaton Study: 3 latent variables, each measured by two indicators. Alienation67 measured by two indicators. Alienation67 measured by anomia67 (1967 anomia scale) measured by anomia67 (1967 anomia scale) and powles67 (1967 powerlessness scale). and powles67 (1967 powerlessness scale).

Alienation71 is measured in same way, but Alienation71 is measured in same way, but using 1971 scales. using 1971 scales.

Third latent variable, SES (socio-economic Third latent variable, SES (socio-economic status) measured by years of schooling and status) measured by years of schooling and Duncan's Socioeconomic Index, both in 1967. Duncan's Socioeconomic Index, both in 1967.

Structural model relates alienation in 1971 Structural model relates alienation in 1971 (F(F22) to alienation in 1967 (F) to alienation in 1967 (F11) and SES (G)) and SES (G)

FF2i2i = βF = βF1i1i + + 22GGii+u+u2i2i

FF1i1i = = GGii + u + u1i1i

Measurement model for alienationMeasurement model for alienation

yyjiji==j j ++jjFF1i1i j=1,2j=1,2

yyjiji==j j ++jjFF2i2i j=3,4j=3,4

Measurement model for SES Measurement model for SES

xxjiji==j j ++jjGGii j=1,2j=1,2

WINBUGS for Wheaton studyWINBUGS for Wheaton studymodel { for (i in 1:n) { # structural model

F2[i] ~ dnorm(mu.F2[i],1);

mu.F2[i] <- beta* F1[i]+gam[2]*G[i]

F1[i] ~ dnorm(mu.F1[i],1);

mu.F1[i] <- gam[1]*G[i]}

# priors (normal uses inverse variance)

for (j in 1:2) {gam[j] ~ dnorm(0,0.001)}

beta ~ dnorm(0,0.001)

# measurement equations for alienation for (i in 1:n) { for (j in 1:4) { y[i,j] ~ dnorm(mu[i,j],tau[j])} mu[i,1] <- alph[1]+lam[1]* F1[i]; mu[i,2] <- alph[2]+lam[2]* F1[i] mu[i,3] <- alph[3]+lam[3]* F2[i]; mu[i,4] <- alph[4]+lam[4]* F2[i]}

# PRIORSfor (j in 1:4){ alph[j] ~ dnorm(0,0.001);

# gamma prior on precisions tau[j] ~ dgamma(1,0.001)# alternative prior starts with s.d. of residuals# sd.y[j] ~ dunif(0,100); tau[j] <- 1/(sd.y[j]*sd.y[j])

# identifiability constraint on loadings to ensure # positive alienation measure lam[j] ~ dnorm(1,1) I(0,)}

# measurement of SES (G[i])

for (i in 1:n) { G[i] ~ dnorm(0,1)

for (j in 1:2) { x[i,j] ~ dnorm(mu.x[i,j],tau.x[j])}

mu.x[i,1] <- del[1]+kappa[1]* G[i];

mu.x[i,2] <- del[2]+kappa[2]* G[i]}

for (j in 1:2) {del[j] ~ dnorm(0,0.001);

# gamma prior on precisions

tau.x[j] ~ dgamma(1,0.001)

# identifying constraint ensures +ve SES scale

kappa[j] ~ dnorm(1,1) I(0,)}}

Monitoring model related quantitiesMonitoring model related quantities

Suppose one were interested in posterior Suppose one were interested in posterior probs that Fprobs that F2i2i > F > F1i1i (alienation increasing (alienation increasing

for ifor ith th subject)subject) Add codeAdd code

for (i in 1:n) {delF[i] <- step(F2[i]-F1[i])}for (i in 1:n) {delF[i] <- step(F2[i]-F1[i])} Then posterior means of delF provide Then posterior means of delF provide

required probabilitiesrequired probabilities

Widening Applications of Latent Widening Applications of Latent Variable Methods Variable Methods

In particular: application contexts of Bayes In particular: application contexts of Bayes SEM/factor models now include ecological SEM/factor models now include ecological (area level) studies of health variations. (area level) studies of health variations.

Usually no longer valid to assume units Usually no longer valid to assume units (i.e. areas) are independent. (i.e. areas) are independent.

Instead spatial correlation in latent Instead spatial correlation in latent variable(s) (common spatial factors) over variable(s) (common spatial factors) over the areas should be consideredthe areas should be considered

Multi-Level Latent Variable ModelsMulti-Level Latent Variable Models

Latent variable methods also more widely Latent variable methods also more widely applied in multilevel health studies applied in multilevel health studies

Such models consider joint impact of Such models consider joint impact of individual levelindividual level and and area levelarea level risk factors risk factors on health statuson health status

With several outcomes (data both With several outcomes (data both multivariate & multilevel) can model area multivariate & multilevel) can model area effects using common factor(s)effects using common factor(s)

SOME SPATIAL PRIORS: SOME SPATIAL PRIORS: THE BASIS FOR COMMON THE BASIS FOR COMMON

SPATIAL FACTORSSPATIAL FACTORS

Priors incorporating spatial structure: Priors incorporating spatial structure: basis for common spatial factorsbasis for common spatial factors

May be specified over continuous space May be specified over continuous space (geostatistical models often used for (geostatistical models often used for “kriging”)“kriging”)

OR for discrete sets of areas with irregular OR for discrete sets of areas with irregular boundaries (“lattices” or “polygons”)boundaries (“lattices” or “polygons”)

Major classes: Major classes: Simultaneous Autoregressive (SAR) or Simultaneous Autoregressive (SAR) or

Conditional Autoregressive (CAR) priorsConditional Autoregressive (CAR) priors

Spatial PriorsSpatial Priors

My focus: CAR priors for “lattices” (e.g. My focus: CAR priors for “lattices” (e.g. administrative areas)administrative areas)

These are priors for “structured” effects These are priors for “structured” effects (where labels of area units are important) (where labels of area units are important) as opposed to unstructured effects as opposed to unstructured effects (unaffected or exchangeable over different (unaffected or exchangeable over different labelling scheme for areas)labelling scheme for areas)

Substantive BasisSubstantive Basis Generally taken to represent Generally taken to represent

unmeasured area level risk factors for unmeasured area level risk factors for health that vary relatively smoothly health that vary relatively smoothly over space (regardless of arbitrary over space (regardless of arbitrary administrative boundaries that may administrative boundaries that may define units of analysis)define units of analysis)

Substantive grounding: increased Substantive grounding: increased recognition of genuine spatial effects recognition of genuine spatial effects on health (“contextual” effects)on health (“contextual” effects)

DIFFERENT TYPES OF DIFFERENT TYPES OF COMMON SPATIAL FACTORCOMMON SPATIAL FACTOR

(A) Manifest health variables(A) Manifest health variables

Manifest variables are health outcomes Manifest variables are health outcomes yyij ij (areas i, variable j)(areas i, variable j)

Common residual factor sCommon residual factor sii, expresses , expresses

spatial clustering recurring over several spatial clustering recurring over several outcomes joutcomes j

Interpretable as index of common Interpretable as index of common health risks over outcomeshealth risks over outcomes

Example: Wang & Wall 2003Example: Wang & Wall 2003

(B) Census Indicator Confirmatory (B) Census Indicator Confirmatory Model.Model.

Common Spatial Socioeconomic Factor or Common Spatial Socioeconomic Factor or Factors (deprivation, rurality, etc) based Factors (deprivation, rurality, etc) based on relevant indicators Zon relevant indicators Zikik (k=1,..,K) such as (k=1,..,K) such as

unemployment, low income etc. unemployment, low income etc. Often census indicators form bulk of Often census indicators form bulk of

manifest variablesmanifest variables Example: Hogan & Tchernis JASA 2004Example: Hogan & Tchernis JASA 2004

(C) Two Classes of Manifest (C) Two Classes of Manifest VariableVariable

Common factor(s) used to explain Common factor(s) used to explain variations in observed Y variables (health variations in observed Y variables (health outcomes). outcomes).

But factors mainly measured by But factors mainly measured by socioeconomic indicators Z (e.g. census socioeconomic indicators Z (e.g. census data)data)

Example: my Eastern region suicide studyExample: my Eastern region suicide study Partly confirmatory, partly exploratoryPartly confirmatory, partly exploratory

MANIFEST VARIABLES: MANIFEST VARIABLES: AREA HEALTH VARIABLESAREA HEALTH VARIABLES

(A) Shared Spatial Residual Effects(A) Shared Spatial Residual Effects Unobserved area effects common to Unobserved area effects common to

several health outcomes modelled by several health outcomes modelled by shared spatial effectshared spatial effect

Typical scenario: area counts yTypical scenario: area counts y ijij for for

areas i and outcomes j. Poisson or areas i and outcomes j. Poisson or binomial likelihoodbinomial likelihood

Types of EventTypes of Event

May be deaths, hospitalizations, incidence May be deaths, hospitalizations, incidence counts for different cancer types, counts for different cancer types, prevalence counts, etc prevalence counts, etc

Expected events (offset) EExpected events (offset) Eijij based on based on

standard age rates applied to area standard age rates applied to area populations: ypopulations: yijij ~ Poisson(E ~ Poisson(Eijijijij) )

Can also have populations at risk: yCan also have populations at risk: y ijij ~ ~

Poisson(NPoisson(Niiijij) or y) or yijij ~ Bin(N ~ Bin(Nii,,ijij) )

Multivariate Spatial EffectsMultivariate Spatial Effects One option for such data: no reductionOne option for such data: no reduction Multivariate residual effectsMultivariate residual effects

log(log(ijij)=)=jj+s+sijij

(or log((or log(ijij)=)=jj++jjxxii+s+sijij))

For sFor sijij could use multivariate version of could use multivariate version of

conditional autoregressive prior conditional autoregressive prior

Multivariate Spatial EffectsMultivariate Spatial Effects

Multivariate normal CAR Prior is example Multivariate normal CAR Prior is example of Markov Random Field (Rue & Held, of Markov Random Field (Rue & Held, 2005). 2005).

Easily applied in WINBUGS using mv.car Easily applied in WINBUGS using mv.car prior.prior.

May fit well but proliferation of parameters May fit well but proliferation of parameters (more parameters than data points)(more parameters than data points)

Alternative : common spatial factorAlternative : common spatial factor

log(log(ijij)=)=jj++jjssii

Parsimonious and provides interpretable Parsimonious and provides interpretable summary measure of health risksummary measure of health risk

ssii is univariate CAR (or some other prior is univariate CAR (or some other prior

with spatial dependence) with spatial dependence) Correlation between outcomes within areas Correlation between outcomes within areas

modelled via loadings modelled via loadings jj. .

Identification: Location & ScaleIdentification: Location & Scale

Need Need iissii=0 for location identification. =0 for location identification.

Centre effects at each MCMC iteration.Centre effects at each MCMC iteration. Scale identifiability:

EITHER set var(s)=1 and all j are free loadings (fixed scale),

OR leave var(s) unknown and constrain a loading, e.g. 1=1.0 (anchoring constraint)

Labelling Problems in Repeated Labelling Problems in Repeated SamplingSampling

Even in simple model, labelling may be an issue.

Consider fixed variance identification option, var(s)=1, loadings all unknown.

Suppose diffuse priors are taken on loadings in

log(log(ijij)=)=jj++jjssii

without directional constraint.

Labelling Problems (continued)Labelling Problems (continued)

Then can have:

a) j all positive combined with ssii acting as acting as

positive measure of health risk (higher spositive measure of health risk (higher s ii in in

areas with higher cancer rates) areas with higher cancer rates)

OR OR

b) b) j all negative combined with s all negative combined with sii acting acting

as negative measure of health risk (sas negative measure of health risk (s ii

higher in areas with lower cancer rates)higher in areas with lower cancer rates)

Identifying constraints for Identifying constraints for consistent labellingconsistent labelling

For unambiguous labelling advisable to constrain one or more j to be positive (e.g. truncated normal or gamma prior)

Note that anchoring constraint with var(s) unknown, and preset loading (e.g. 1=1.0), may be intrinsically better identified – steers remaining unknown coefficients to consistent labelling

Loadings and LabellingsLoadings and Labellings May not be sufficient just to rely on May not be sufficient just to rely on

constraining one loading (e.g. assume +ve) constraining one loading (e.g. assume +ve) to ensure consistent labellingto ensure consistent labelling

Sometimes said that constraining direction Sometimes said that constraining direction on one loading ensures consistent on one loading ensures consistent identification…identification…

What if indicator chosen for constrained What if indicator chosen for constrained loading (e.g. loading (e.g. iiii> 0) is poor measure for > 0) is poor measure for

constructconstruct

Loadings and LabellingsLoadings and Labellings

If twenty indicators are measuring a If twenty indicators are measuring a construct, the 19 unconstrained loadings construct, the 19 unconstrained loadings may “fit” a different label (e.g. deprivation) may “fit” a different label (e.g. deprivation) to that implied by the remaining to that implied by the remaining constrained loading (e.g. affluence)constrained loading (e.g. affluence)

Personal View: Much depends on suitable Personal View: Much depends on suitable selection of manifest indicators and which selection of manifest indicators and which (and how many, maybe >1 ) are chosen to (and how many, maybe >1 ) are chosen to have constrained loadingshave constrained loadings

WINBUGS Code for manifest WINBUGS Code for manifest variable scenario Avariable scenario A

Extensions of Spatial Common FactorsExtensions of Spatial Common Factors Product schemes. Consider health Product schemes. Consider health

outcomes arranged by area i and age x. outcomes arranged by area i and age x. Populations at risk NPopulations at risk N ixix

yyixix ~ Poisson(N ~ Poisson(Nixixixix))

log(log(ixix)=)=xx++xxssii

x x show which age groups are most show which age groups are most sensitive to spatial variations in risk sensitive to spatial variations in risk represented by srepresented by sii

Variation on Lee-Carter (JASA 1992) Variation on Lee-Carter (JASA 1992) mortality forecasting modelmortality forecasting model

Random Effect LoadingsRandom Effect Loadings x x potentially random, rather than potentially random, rather than

fixed effects. fixed effects. Identified using sum to 1 or averaging Identified using sum to 1 or averaging

to 1 constraint, e.g. to 1 constraint, e.g. x x multinomial, or multinomial, or

xx~Gamma(h,h)~Gamma(h,h)

Nonlinear effects of common factorNonlinear effects of common factor One possibility: just take powers of sOne possibility: just take powers of s ii, ,

e.g.e.g. log(log(ijij)=)=jj++jjssii++jjss22

ii

Or: spline for nonlinear effects in Or: spline for nonlinear effects in common factor score scommon factor score sii. .

e.g. under fixed variance var(s)=1 e.g. under fixed variance var(s)=1 option, locate knots option, locate knots kk at selected at selected quantiles on cumulative standard quantiles on cumulative standard normal.normal.

Linear SplineLinear Spline

Then linear splineThen linear spline

log(log(ijij)=)=jj++jjssii++kkbbjkjk(s(sii- - kk))++

bbjkjk might be random effects, but might be random effects, but

raises identification issues…?raises identification issues…?

INDICATOR BASED INDICATOR BASED SPATIAL CONSTRUCTSSPATIAL CONSTRUCTS

(B) Indicator Based Spatial Constructs(B) Indicator Based Spatial Constructs

Many studies use latent constructs to Many studies use latent constructs to analyze population health variations. analyze population health variations.

Such constructs (e.g. deprivation) not Such constructs (e.g. deprivation) not directly observeddirectly observed

Instead derived from a collection of relevant Instead derived from a collection of relevant indicator variables that are observed, using indicator variables that are observed, using multivariate techniques or other “composite multivariate techniques or other “composite variable” methodsvariable” methods

Many health outcomes show “deprivation Many health outcomes show “deprivation gradient”gradient”

Latent Constructs in Population Latent Constructs in Population HealthHealth

Example: Townsend deprivation score Example: Townsend deprivation score based on summing standardized census based on summing standardized census area values for 4 input variables (sum of “z area values for 4 input variables (sum of “z scores”)scores”)

% unemployed, % with no car, % % unemployed, % with no car, % households overcrowded, % households not households overcrowded, % households not owner occupiers owner occupiers

Other area constructsOther area constructs

Other examples of latent constructs Other examples of latent constructs relevant to area health variations: relevant to area health variations: rurality/urbanicity, social fragmentationrurality/urbanicity, social fragmentation

Social fragmentation scores used to Social fragmentation scores used to analyze variations in area suicide rates analyze variations in area suicide rates and psychiatric hospitalization ratesand psychiatric hospitalization rates

Confirmatory Indicator Based ModelConfirmatory Indicator Based Model Confirmatory model: indicators k=1,..,K are Confirmatory model: indicators k=1,..,K are

established proxies for latent constructestablished proxies for latent construct e.g. area unemployment rates, welfare e.g. area unemployment rates, welfare

recipients, social housing rates as recipients, social housing rates as indicators of area deprivationindicators of area deprivation

Census rates rCensus rates rikik=z=zikik/D/Dikik where z where zikik are counts are counts

(e.g. unemployed), D(e.g. unemployed), Dikik are relevant are relevant

denominators (e.g. econ active denominators (e.g. econ active populations). populations).

One option for confirmatory modelOne option for confirmatory model

Use Gaussian approximation to binomial Use Gaussian approximation to binomial (Hogan & Tchernis JASA 2004) with (Hogan & Tchernis JASA 2004) with variance stabilizing transformation: Rvariance stabilizing transformation: Rikik==rrikik, ,

var(Rvar(Rikik)=)=kk/D/Dikik. .

→ → normal measurement equationsnormal measurement equations

RRikik ~N( ~N( kkkkFFii, , kk/D/Dik)ik)

where Fwhere Fii scores follow spatial CAR prior scores follow spatial CAR prior

Or use relevant Exponential Family links in Or use relevant Exponential Family links in deriving common spatial factorderiving common spatial factor

P(zP(zikik||ikik) = exp([z) = exp([zikikikik-b(-b(ikik)]/)]/+c(z+c(zikik, , ))))

e.g. ze.g. zikik binomial with populations N binomial with populations Nii, z, zikik ~ ~

Bin(NBin(Nii,,ikik))

Logit link, plus overdispersion effects wLogit link, plus overdispersion effects w ikik

logit(logit(ikik)= )= kkkkFFii+w+wikik

wwik ik : normal and uncorrelated over : normal and uncorrelated over

indicators k.indicators k.

For other indicators transform to For other indicators transform to normalitynormality

For intrinsic proportions (e.g. proportion of For intrinsic proportions (e.g. proportion of area that is green space as indicator of area that is green space as indicator of rurality) take logit transform to rurality) take logit transform to approximate normalityapproximate normality

for population density take log transformfor population density take log transform etcetc

TWO CLASSES OF MANIFEST TWO CLASSES OF MANIFEST VARIABLEVARIABLE

(C) Spatial Factors in Model with 2 (C) Spatial Factors in Model with 2 classes of manifest variableclasses of manifest variable

Health Outcomes YHealth Outcomes Yijij (j=1,…,J); e.g. (j=1,…,J); e.g.

mortality or incidence countsmortality or incidence counts Social Indicators ZSocial Indicators Zikik (k=1,..k); e.g. census (k=1,..k); e.g. census

rates of unemploymentrates of unemployment Typical Scenario: multiple common spatial Typical Scenario: multiple common spatial

factors (Ffactors (F1i1i,..,F,..,FQiQi) primarily measured by Z ) primarily measured by Z

variables (indicators established as variables (indicators established as relevant). relevant).

2 class model2 class model

But Factors F also act to potentially But Factors F also act to potentially explain area variations in health outcomes explain area variations in health outcomes Y. Y.

Z to F links confirmatory, Y to F links Z to F links confirmatory, Y to F links exploratoryexploratory

ExampleExample Four Poisson health outcomes Y1-Y4, Eight Four Poisson health outcomes Y1-Y4, Eight

indicators: Z1-Z4 measure F1; Z5-Z8 measure indicators: Z1-Z4 measure F1; Z5-Z8 measure F2 ; both F1 and F2 F2 ; both F1 and F2 maymay explain Y explain Y

YYijij ~ Po(E ~ Po(Eijijijij))

log(log(ijij)=)=jj++j1j1FF1i1i++j2j2FF2i2i

ZZikik ~ EF( ~ EF(ikik))

g(g(i1i1)= )= 11FF1i1i+w+wi1i1

………… g(g(i5i5)= )= 55FF2i2i+w+wi5i5

………………

MODEL CHOICEMODEL CHOICE

Formal Choice or NotFormal Choice or Not Formal Bayes model criteria (e.g. marginal Formal Bayes model criteria (e.g. marginal

likelihood/Bayes factor) difficult to derive; likelihood/Bayes factor) difficult to derive; also change with priorsalso change with priors

Popular alternative (AIC analogue): Popular alternative (AIC analogue): Deviance Information Criterion (DIC). Deviance Information Criterion (DIC).

Average deviance Dev.bar + effective Average deviance Dev.bar + effective parameter count dparameter count dee

DIC=Dev.bar+ dDIC=Dev.bar+ dee

Model Fit in Realistic ApplicationsModel Fit in Realistic Applications

Multilevel applications to health survey Multilevel applications to health survey data may involve thousands of subjects data may involve thousands of subjects (e.g. HSE study). (e.g. HSE study).

Ecological applications may involve Ecological applications may involve hundreds of small areas (Eastern region hundreds of small areas (Eastern region suicide study)suicide study)

Model Fit in Realistic ApplicationsModel Fit in Realistic Applications

Convergence of DIC and dConvergence of DIC and dee typically slow typically slow

in models with many random effects (such in models with many random effects (such as factor scores)as factor scores)

Slow convergence also applies to other Slow convergence also applies to other measures of fit, e.g. Monte Carlo measures of fit, e.g. Monte Carlo estimates of conditional predictive estimates of conditional predictive ordinatesordinates

Model selection alternatives…Model selection alternatives…

Model Choice using Variable Selection Model Choice using Variable Selection

Model selection potentially for both Model selection potentially for both loadings and factor variance/covariance loadings and factor variance/covariance structure.structure.

Don’t necessarily apply selection for all Don’t necessarily apply selection for all elements in any particular application (e.g. elements in any particular application (e.g. depending whether exploratory or depending whether exploratory or confirmatory)confirmatory)

Apply to selected aspects of spatial SEM Apply to selected aspects of spatial SEM models, e.g. loadings only or correlations models, e.g. loadings only or correlations between factors onlybetween factors only

Selection in 2 manifest variable Selection in 2 manifest variable SEMSEM

Spatial factor models with 2 types of Spatial factor models with 2 types of manifest variable (health outcomes Ymanifest variable (health outcomes Yjj + +

socioeconomic indices Zsocioeconomic indices Zkk))

Apply selection to loadings Apply selection to loadings jqjq linking Y linking Yjj to to

FFq q (exploratory part of model)(exploratory part of model)

But don’t apply selection to Z on F But don’t apply selection to Z on F loadings (confirmatory sub-model based loadings (confirmatory sub-model based on extensive prior knowledge)on extensive prior knowledge)

Mixture Priors for Selecting LoadingsMixture Priors for Selecting Loadings

Random Effects SelectionRandom Effects Selection

Selection procedures for random effects Selection procedures for random effects and/or their variance/covariance structureand/or their variance/covariance structure

e.g. Cai and Dunson (2008), Tüchler & e.g. Cai and Dunson (2008), Tüchler & Frühwirth-Schnatter (2008)Frühwirth-Schnatter (2008)

These extend to factor and SEM models These extend to factor and SEM models as factors are shared random effectsas factors are shared random effects

RE Selection: Multivariate Spatial RE Selection: Multivariate Spatial PriorPrior

Q>1 for shared common spatial Q>1 for shared common spatial factorsfactors

Within area covariance matrix in Within area covariance matrix in MCAR prior denoted MCAR prior denoted FF

Cholesky Decomposition of Covariance Matrix Cholesky Decomposition of Covariance Matrix FF

Selection on variances and/or covariancesSelection on variances and/or covariances

Suppose investigator sure about number Suppose investigator sure about number of factors (confirmatory model based on of factors (confirmatory model based on substantial evidence) substantial evidence)

BUT not sure whether correlations BUT not sure whether correlations between factors are needed between factors are needed

Selection can be applied to relevant Selection can be applied to relevant parameters in decomposition of parameters in decomposition of FF →→ mixture prior selection on mixture prior selection on qrqr parameters parameters to decide whether correlations needed to decide whether correlations needed

CASE STUDIESCASE STUDIES

Social capital and mental health, Social capital and mental health, multilevel model using Health Survey multilevel model using Health Survey for England (HSE)for England (HSE)

Multilevel model, joint prevalence of Multilevel model, joint prevalence of obesity & diabetes, BRFSS subjects obesity & diabetes, BRFSS subjects nested within US counties & statesnested within US counties & states

Suicide & self-harm, ecological (area) Suicide & self-harm, ecological (area) study for wards in Eastern Englandstudy for wards in Eastern England

Case Study 1, Mental Health & Social Case Study 1, Mental Health & Social Capital, Health Survey for EnglandCapital, Health Survey for England

Y is observed mental health status (binary). Y is observed mental health status (binary). Y=1 if subject’s GHQ12 score is 4 or more, Y = 0 otherwise.

Pr(Y=1) related to known socioeconomic risk Pr(Y=1) related to known socioeconomic risk factors X at individual subject levelfactors X at individual subject level

Pr(Y=1) also related to known indicators of Pr(Y=1) also related to known indicators of geographic geographic contextcontext, G (e.g. micro-area , G (e.g. micro-area deprivation quintile, region of residence, urban-deprivation quintile, region of residence, urban-rural residence). Micro-areas (32K in England) rural residence). Micro-areas (32K in England) called Super Output Areas called Super Output Areas

Latent RisksLatent Risks

Finally Pr(Y=1) also related to latent Finally Pr(Y=1) also related to latent subject level risks, {Fsubject level risks, {F1i1i,F,F2i2i,...,F,...,FQiQi}}

Examples: social capital, perceived stress. Examples: social capital, perceived stress. Structural model: Y~f(Y|X,G,F,Structural model: Y~f(Y|X,G,F,))

Health Outcome Sub-ModelHealth Outcome Sub-Model Regression involves 9065 adult subjects.

Yi~ Bin(1,i) .

Use log-link (→relative risk interpretation). Q=1 for single latent risk factor (social

capital) log(i)=βXi+γGi+Fi =β₀+β1,gend[i]

+β2,age[i]+β3,eth[i]+β4,oph[i]+β5,own[i]

+β6,noqual[i]+1,reg[i]+2,dep[i]+3,urb[i]+Fi

Multiple Indicators for Social Multiple Indicators for Social CapitalCapital

Social capital measured by a battery of K Social capital measured by a battery of K survey `items' (e.g. questions about survey `items' (e.g. questions about neighbourhood perceptions, organisational neighbourhood perceptions, organisational memberships etc), {Z₁,...,Zmemberships etc), {Z₁,...,ZKK}}

Z~g(Z|F,Z~g(Z|F,) ) e.g. with binary questions, link probability e.g. with binary questions, link probability

of positive response of positive response kk=Pr(Z=Pr(Zkk=1) to latent =1) to latent construct via construct via

logit(logit(kk)=)=kk++kkFF

Indicators of Social CapitalIndicators of Social Capital Social Support Score (Z1)Social Support Score (Z1) 5 binary items (Z2-Z6) relate to

neighbourhood perceptions (e.g. can people be trusted?; do people try to be helpful?; this area is a place I enjoy living in; etc)

Final item (Z7) relates to membership of organisations or groups.

Multiple Causes of Social CapitalMultiple Causes of Social Capital Social capital varies by demographic Social capital varies by demographic

groups and geographic context (urban groups and geographic context (urban status, region, small area deprivation status, region, small area deprivation category, etc). category, etc).

So have multiple causes of F as well as So have multiple causes of F as well as multiple indicatorsmultiple indicators

F ~ h(F|X*,G*, F ~ h(F|X*,G*, φ)φ) X* and G* are individual and contextual X* and G* are individual and contextual

variables relevant to “causing” social variables relevant to “causing” social capital variationscapital variations

Multiple Cause Sub-ModelMultiple Cause Sub-Model

FFii~N(μ~N(μii,1) μ,1) μii=φ=φ1,gend[i]1,gend[i]+φ+φ2,eth[i]2,eth[i]+φ+φ3,noqual[i]3,noqual[i]

+φ+φ4,urb[i]4,urb[i]+φ+φ5,reg[i]5,reg[i]

+φ+φ6,dep[i]6,dep[i]..

φ: fixed effects parameters with reference φ: fixed effects parameters with reference category (zero coeff) for identificationcategory (zero coeff) for identification

Only small number of regions in HSE Only small number of regions in HSE If had finer spatial detail could take area φ If had finer spatial detail could take area φ

effects spatially random (but weak effects spatially random (but weak identification…?)identification…?)

Effect of F on YEffect of F on Y

Social capital has significant effect in reducing Social capital has significant effect in reducing the chances of psychiatric caseness. the chances of psychiatric caseness.

The effect of social capital apparent in relative The effect of social capital apparent in relative risk 0.35 of psychiatric morbidity for high capital risk 0.35 of psychiatric morbidity for high capital individuals (with score F=+1) as compared to individuals (with score F=+1) as compared to low capital individuals (with F=-1). low capital individuals (with F=-1).

Obtained as exp(-0.525)/exp(0.525)Obtained as exp(-0.525)/exp(0.525) = -0.525 is coefficient for social capital effect.= -0.525 is coefficient for social capital effect.

Geographic Context: Micro-area Deprivation Gradient Geographic Context: Micro-area Deprivation Gradient from Multiple Cause Modelfrom Multiple Cause Model

Case Study 2: Diabetes & Obesity Case Study 2: Diabetes & Obesity in USin US

Data from 2007 Behavioral Risk Factor Data from 2007 Behavioral Risk Factor Surveillance System (BRFSS)Surveillance System (BRFSS)

Multinomial outcome (J=6 categories) Multinomial outcome (J=6 categories) defined by diabetic status and weight defined by diabetic status and weight category (obese, overweight, normal). category (obese, overweight, normal).

Multinomial CategoriesMultinomial Categories

Reference category are subjects with Reference category are subjects with neither condition. All other categories are neither condition. All other categories are “ill” relative to reference category“ill” relative to reference category

Multilevel multicategory regressionMultilevel multicategory regression

Regression includes:Regression includes:o subject level risk factors (age, ethnicity, subject level risk factors (age, ethnicity,

gender, education), gender, education), o known geographic effects (e.g. county known geographic effects (e.g. county

poverty), poverty), o county and state random effects to model county and state random effects to model

unknown geographic influences (e.g. unknown geographic influences (e.g. unknown environmental exposures).unknown environmental exposures).

Regression & LikelihoodRegression & Likelihood

Model FormModel Form

Model includes Model includes knownknown subject risk factors subject risk factors and contextual variables (e.g. county and contextual variables (e.g. county poverty)poverty)

UnknownUnknown contextual risks: assume county contextual risks: assume county and state latent effects, shared over and state latent effects, shared over categories j=1,..,J-1. categories j=1,..,J-1.

Illustrates nested latent spatial effectsIllustrates nested latent spatial effects

County & State EffectsCounty & State Effects

Take county effects vTake county effects vcc (c=1,..,3142) to be (c=1,..,3142) to be

spatially correlated CARspatially correlated CAR But uBut uss (state effects, s=1,..,51) taken to be (state effects, s=1,..,51) taken to be

unstructured. unstructured. Avoids confounding of two spatially Avoids confounding of two spatially

structured effectsstructured effects

Regression Terms for j=1,..J-1Regression Terms for j=1,..J-1

Case Study 3, Suicide & Self Harm: Case Study 3, Suicide & Self Harm: Eastern Region Wards in EnglandEastern Region Wards in England

Two classes of manifest variablesTwo classes of manifest variables YY11-Y-Y44: suicide totals in small areas: suicide totals in small areas

ZZ11-Z-Z1414: Fourteen small area social : Fourteen small area social

indicators indicators Q=3 latent constructs (FQ=3 latent constructs (F11 fragmentation, F fragmentation, F22

deprivation, Fdeprivation, F33 urbanicity). Converse of F urbanicity). Converse of F33

is “rurality”. Common spatial factors. is “rurality”. Common spatial factors.

Local Authority Map: Eastern EnglandLocal Authority Map: Eastern England

Geographic FrameworkGeographic Framework

N=1118 small areas (called wards, N=1118 small areas (called wards, subdivisions of local authorities). subdivisions of local authorities).

Small area focus beneficial: people with Small area focus beneficial: people with similar socio-demographic characteristics similar socio-demographic characteristics tend to cluster in relatively small areas, so tend to cluster in relatively small areas, so greater homogeneity in risk factors related greater homogeneity in risk factors related to social statusto social status

On other hand, health events may be On other hand, health events may be rare…rare…

Confirmatory Sub-ModelConfirmatory Sub-Model

Confirmatory Z-on-F modelConfirmatory Z-on-F model Each indicator ZEach indicator Zkk loads only on one loads only on one

construct Fconstruct Fqq..

Most indicators binomial. A few taken as Most indicators binomial. A few taken as normal after transformation. Mostly 2001 normal after transformation. Mostly 2001 Census, a few non-census (service Census, a few non-census (service access score, proportion greenspace)access score, proportion greenspace)

Exponential Family Model for Exponential Family Model for modelling Z-on-F effectsmodelling Z-on-F effects

For indicator kFor indicator k1,..,14, G1,..,14, Gkk 1,2,3 denotes 1,2,3 denotes

which construct it loads on. which construct it loads on. Regression with link g allows for Regression with link g allows for

overdispersion via “unique” w effectsoverdispersion via “unique” w effects

g(g(ikik)= )= kkk,Gk,GkkF[GF[Gkk,i]+w,i]+wikik

Expected Direction of Expected Direction of Confirmatory Model LoadingsConfirmatory Model Loadings

Health Outcome Sub-Model (Y-on-Health Outcome Sub-Model (Y-on-F effects)F effects)

Model for Y-on-F effectsModel for Y-on-F effects

YYijij ~ Po(E ~ Po(Eijijijij) j=1,..,4) j=1,..,4

log(log(ijij)=)=jj++j1j1FF1i1i++j2j2FF2i2i++j3j3FF3i3i+u+uijij

Coefficient selection on Coefficient selection on jq jq using relatively using relatively

informative priors under “retain” option informative priors under “retain” option when Jwhen Jjqjq=1.=1. Using diffuse priors means null Using diffuse priors means null

model tends to be selectedmodel tends to be selected

Redundant CoefficientsRedundant Coefficients

Some coefficients (e.g. urbanicity on male Some coefficients (e.g. urbanicity on male and female suicide, deprivation on female and female suicide, deprivation on female suicide) not retained under model suicide) not retained under model selectionselection

Four coefficients in the Y-on-F model were Four coefficients in the Y-on-F model were set to zero in at least some MCMC set to zero in at least some MCMC iterations iterations → averaging over 2→ averaging over 244 Y-on-F Y-on-F modelsmodels

Future Directions in Spatial Factor Future Directions in Spatial Factor ModellingModelling

Extend model selection to interactions between Extend model selection to interactions between factors, nonlinear effects etcfactors, nonlinear effects etc

In England, model area socioeconomic structure In England, model area socioeconomic structure (and maybe some health outcomes) at (and maybe some health outcomes) at “neighbourhood” level (32000 “Super Output “neighbourhood” level (32000 “Super Output Areas” with mean population 1500).Areas” with mean population 1500).

In US, similar scope for modelling SES structure In US, similar scope for modelling SES structure in relation to health events for Zip Code in relation to health events for Zip Code Tabulation Areas or ZCTAs (around 31K across Tabulation Areas or ZCTAs (around 31K across US, on average about 10K population)US, on average about 10K population)

More generallyMore generally

Bayesian software options for latent Bayesian software options for latent variable and SEM applications more variable and SEM applications more widely availablewidely available

Potentialities of WINBUGS in this context Potentialities of WINBUGS in this context not always appreciatednot always appreciated

Scope for dedicated Bayesian factor Scope for dedicated Bayesian factor analysis packageanalysis package