Exploratory Factor Analysis

Multivariate Fundamentals: Rotation

PCA Analysis – A Review

Precipitation Temperature Ecosystems

PCA Analysis with Spatial Data

Proportion of variance explained Comp.1 + Comp.2 + Comp.3 ~= 95% Loadings

PCA Analysis with Spatial Data Loadings

Loadings indicate MANY climate variables are associated with Component 1 Intuitively these variables are associated with growth conditions

PCA Analysis with Spatial Data Loadings

Loadings indicate temperature variables are associated with Component 2 Intuitively these variables are associated with continetality

PCA Analysis with Spatial Data Loadings

Loadings indicate moisture variables are associated with Component 3 Intuitively these variables are associated with weather jet stream

PCA Analysis with Spatial Data

Exploratory Factor Analysis (EFA)

Objective - Rotate that data so that new axis explains the greatest amount of

variation within the data (same as PCA)

But, unlike PCA, the key concept of factor analysis is that multiple observed variables have similar patterns of responses because they are all associated with a latent (i.e. not directly measured) variable.

In the context of EFA, PCA can be considered a technique to simply reduce variables

Think of EFA as: “There is a bigger picture controlling the variables I am analyzing, and I want to better understand the relationship with those underlying unobservable factors”

Charles Spearman (1863-1945)

The math behind EFA

Renewable Resources MSc Example:

Student ID

Ecology, Y1

Policy, Y2

Geography, Y3

Design, Y4

Statistics, Y5

1 3 2 3 6 5

2 7 7 7 4 4

3 4 4 4 4 4

4 5 5 5 5 5

5 7 6 6 4 5

Assume Yi is linearly related to F1 and F2 as follows:

Y1 = β10 + β11F1 + β12F2 + ε1 Y2 = β20 + β21F1 + β22F2 + ε2

Y3 = β30 + β31F1 + β32F2 + ε3 Y4 = β40 + β41F1 + β42F2 + ε4 Y5 = β50 + β51F1 + β52F2 + ε5

Fi = unobservable factor (e.g. writing ability, mathematical ability)

ε1 = error term

βi = loadings

Comes from PCA

EFA utilizes the outputs of PCA, but continues to rotate the data to investigate the relationship with underlying unobservable factors Because EFA utilizes PCA components EFA does NOT completely maximize variance explained

Policy

Ecology

Statistics

PC 1

PC

2

Design

Geography

Determining the number of factors

FA should ONLY include important factors

There are a number of different ways to determine the number of factors you should use

But the easiest way is to set the number of factors to the number of PC components that explain a significant portion of the variation in the data

Exploratory Factor Analysis in R

Exploratory FA in R: factanal(dataMatrix,factors=n,rotation="type") (stats package)

Data matrix of original predictor variables NOT principal components

The number of factors you want to include

Note: R will ONLY let you calculate the number of factors where variance explained is sufficient

The statistical method used to rotation the data

Rotation options fall in to 2 categories

Orthogonal rotation – assumes your factors are uncorrelated function options: "varimax", "quatimax" Oblique rotation – assumes your factors are correlated function options: "promax", "oblimin"

Additional function options: "none" , "simplimax", "cluster“

When selecting your rotation method consider correlation between factors Simple criteria commonly used (Tabachnick & Fidell, 2007):

1. Start with an oblique option "promax" and look at the correlation between your factors (assume underlying factors are correlated)

2. If the correlation between factors is <|0.32| then use orthogonal option "varimax" default for R

For more information : Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Upper Saddle River, NJ: Pearson Allyn & Bacon, pg 646-647.

Popular rotation methods

Orthogonal (rotates factors 90°, assumes uncorrelated factors)

Varimax - minimizes the number of variables that have high loadings on each factor and works to make small loadings even smaller

Quartimax - involves the minimization of the number of factors needed to explain each variable

Oblique (rotates factors > or < 90°, assumes correlated factors)

Promax - involves raising the loadings to a power of four which ultimately results in greater correlations among the factors and achieves a simple structure. Preferred methods because its speed with larger datasets

Direct Oblimin - attempts to simplify the structure and the mathematics of the output Oblique rotation is more complex than orthogonal rotation, since it can involve one of two coordinate systems: a system of primary axes or a system of reference axes

Most commonly used are Varimax and Promax

For more information : Tabachnick, B. G., & Fidell, L. S. (2007). Using multivariate statistics (5th ed.). Upper Saddle River, NJ: Pearson Allyn & Bacon, pg 646-647.

Exploratory Factor Analysis in R

First we want to look at the correlation between factors derived from the analysis

Tabachnick & Fidell Criterion:

1. Start with the assumption factors are correlated so we have to use the oblique rotation option "promax"

2. If correlations <|0.32| then the equation solutions remains nearly orthogonal, and we should use the orthogonal rotation option "varimax"

If your data is correlated you should report the correlation value with your results

Exploratory Factor Analysis in R

Uniqueness: tells you if there is a variable that is significantly different than the rest

Factor loadings: tells you the relationship between the calculated factors and the original variables

Variance explained: how much of data variance is explained by each factor

P-value: tests the hypothesis “The number of factors included in this analysis are sufficient enough to capture the underlying unobservable relationships”

Large P-value = do not reject null hypothesis – i.e. YES the number of factors in the analysis are sufficient

Exploratory Factor Analysis in R

Typically weak relationships between factors original variables are generally discarded from the analysis

Debate within the literature to what the cutoff for strong relationship should be

Stringent cutoffs are often not realistic for environmental data

Common Rule of Thumb: A relationship is strong if the absolute value > 0.4 But, this should be evaluated for your data as to what logically makes sense

In the Master Grades example:

Factor 1 has a strong relationship with grades in Ecology, Policy, and Geography Factor 2 has a strong relationship with grades in Statistics and a moderate relationship with Experimental Design

Using rationale and the commonalities between the class subjects we could infer that Factor 1 is likely related to writing ability and Factor 2 is likely related to mathematical ability

PCA EFA

PCA vs EFA output

Renewable Resources MSc Example:

Rotations were in multi-dimensional space (i.e. 5 variables were included)