3. Factor Analysis

Techniques and Applications of Multivariate Analysis

Lecture 3. Factor Analysis (FA)

Lecture3-1 3.1 Introduction 3.2 Orthogonal Factor Model 3.3 Methods of Estimation: PCM vs MLM Lecture3-2 3.4 Factor Rotation 3.5 Factor Scores: WLSM vs RM 3.6 Example : Air-pollution Data 3.7 Strategy for Factor Analysis : Example 9.14

Gaelic English History Arithmetic Algebra Geometry

Gaelic 1.00 . . . . .

English 0.44 1.00 . . . .

History 0.41 0.35 1.00 . . .

Arithmetic 0.29 0.35 0.16 1.00 . .

Algebra 0.33 0.32 0.19 0.59 1.00 .

Geometry 0.25 0.33 0.18 0.47 0.46 1.00

3.1 Introduction of FA

Definition

FA: technique for describing the covariance relationship among many variables in terms of a few factors which are underlying, but unobservable random quantities.

Example (Ex. 9.8, p.502)

Correlation matrix of 6 subjects

Mathematical-ability factor

Verbal-ability factor

3.1 Introduction of FA

History : K. Pearson and Charles Spearman provided beginnings of FA in the early 20th century.

Charles Spearman is known for being the one who coined the term factor analysis

and actually used it to measure children’s cognitive performance.

Spearman, C. (1904). “General intelligence” objectively determined and measured.

"American Journal of Psychology", 15, 201–293.

3.2 Orthogonal Factor Model

Model with m common factors

Properties

Matrix of factor loadings

Vector of specific factors

Assumptions

Common factors decomposition

communality

Specific variance

Loading of the ith the variable on the jth factor

3.3 Methods of Estimation: PCM vs MLM

: Common factor decomposition

with

3.3 Methods of Estimation

[step 3] Obtain the matrix of estimated factor loadings (m<p), the matrix of estimated specific variances and the estimated communalities.

Question : Which communality is estimated very well?

3.3 Methods of Estimation: PCM

How do we select the number of factors m in PCM?

for S, for R

3.3 Methods of Estimation: PCM

Example (Ex. 9.8, p.502)

Correlation matrix of 6 subjects

Program

Gaelic English History Arithmetic Algebra Geometry Gaelic 1.00 . . . . . English 0.44 1.00 . . . . History 0.41 0.35 1.00 . . .

Arithmetic 0.29 0.35 0.16 1.00 . . Algebra 0.33 0.32 0.19 0.59 1.00 .

Geometry 0.25 0.33 0.18 0.47 0.46 1.00

3.3 Methods of Estimation : Results of SAS

They are the only eigenvalues greater than 1.

2 factors account for a cumulative proportion of the total sample variance.

General intelligence factor

Bipolar factor: Half + and half -

All communalities are nearly about 1

All elements are small

3.3 Methods of Estimation: MLM

2) Algorithm for Maximum Likelihood Method

[step 1] Given , consider the likelihood function

[step 3] We have the ml estimators and mles of the communalities

[step 2]With and

obtain the maximization of the likelihood function subject to uniqueness condition

3.3 Methods of Estimation: MLM

How do we select the number of factors m in MLM?

Test the hypotheses with an appropriate m.

: Bartlett’s test statistic based on the chi-square approximation

Residual matrix:

The diagonal elements are zero and the other elements are small: m factor model is appropriate !

3.3 Methods of Estimation: MLM

Program

3.3 Methods of Estimation: Comparison