Chapter II Multiple Correspondance Analysis (MCA)mchave100p/wordpress/wp-content/...Example:TheIrisdata. ## Sepal.Length Sepal.Width Petal.Length Petal.Width Species ## 1 5.1 3.5 1.4

Chapter IIMultiple Correspondance Analysis (MCA)

Master MMAS - University of Bordeaux

Marie Chavent

Chapitre 2 MCA 1/52

Introduction

How to get information from a categorical data table of individuals×variables ?

Example : categorical data table where 27 dogs are described on 6 variables.

#load("chiens.RData")load("dogs.RData")print(data[1:8,])

## Size Weight Velocity Intelligence Affectivity Aggressivness## Beauceron S++ W+ V++ I+ Af+ Ag+## BassetHound S- W- V- I- Af- Ag+## GermanShepherd S++ W+ V++ I++ Af+ Ag+## Boxer S+ W+ V+ I+ Af+ Ag+## Bulldog S- W- V- I+ Af+ Ag-## BullMastiff S++ W++ V- I++ Af- Ag+## Poodle S- W- V+ I++ Af+ Ag-## Chihuahua S- W- V- I- Af+ Ag-

- Which individuals are similar ?

- Which variables are linked ?

Chapitre 2 MCA 2/52

By looking the matrix of the distances between the individuals ?

d <- dist(data)as.matrix(d)[1:5,1:5]

## Beauceron BassetHound GermanShepherd Boxer Bulldog## Beauceron 0 NA NA NA NA## BassetHound NA 0 NA NA NA## GermanShepherd NA NA 0 NA NA## Boxer NA NA NA 0 NA## Bulldog NA NA NA NA 0

How do we measure the distance between two rows of categorical data ?

Chapitre 2 MCA 3/52

By looking the matrix of the χ2 of independance between the pairs ofvariables ?p <- ncol(data) ; chi2 <- matrix(NA,p,p) ; pval <- matrix(NA,p,p)rownames(pval) <- colnames(pval) <- rownames(chi2) <- colnames(chi2) <- colnames(data)for (j in 1:p)

for (k in 1:p) {tab <- table(data[,j],data[,k])chi2[j,k] <- chisq.test(tab)$statisticpval[j,k] <- chisq.test(tab)$p.value

}print(chi2,digit=2) # value of the chi2 statistic

## Size Weight Velocity Intelligence Affectivity Aggressivness## Size 54.0 25.3 15.89 3.6 13.95 2.05## Weight 25.3 54.0 18.47 1.4 9.48 2.55## Velocity 15.9 18.5 54.00 3.2 2.97 0.57## Intelligence 3.6 1.4 3.16 54.0 3.89 1.16## Affectivity 14.0 9.5 2.97 3.9 23.14 0.91## Aggressivness 2.1 2.6 0.57 1.2 0.91 23.14

print(round(pval,digit=3),digit=2) # p-value of the test of independance

## Size Weight Velocity Intelligence Affectivity Aggressivness## Size 0.000 0.000 0.003 0.46 0.001 0.36## Weight 0.000 0.000 0.001 0.85 0.009 0.28## Velocity 0.003 0.001 0.000 0.53 0.227 0.75## Intelligence 0.462 0.852 0.532 0.00 0.143 0.56## Affectivity 0.001 0.009 0.227 0.14 0.000 0.34## Aggressivness 0.359 0.279 0.750 0.56 0.339 0.00

Chapitre 2 MCA 4/52

By applying a multivariate statistical method ?

Multiple Correspondance Analysis (MCA) gives graphical representation of thedistances between the individuals, the links between the categorival variablesand the levels.

library(FactoMineR)res <- MCA(data,graph=FALSE)plot(res,choix="ind",invisible = "var",

title="",cex=1.5)

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

Beauceron

BassetHound

GermanShepherd

Boxer

BulldogBullMastiff

Poodle

Chihuahua

CockerSpaniel

Collie

Dalmatien

Doberman

GermanMastiff

BittanySpaniel

FrenchSpaniel

FoxHoundFoxTerrier

GrdBleuGascon

Labrador

Levrier

MastiffPekingese

Pointer

SaintBernard

Setter

TeckelNewfoundland

plot(res,choix="ind",invisible = "ind",title="",cex=1.5)

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

5−

1.0

−0.

50.

00.

51.

0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

S−

S+

S++

W−

W+

W++ V−

V+

V++

I−

I+I++

Af−

Af+Ag−

Ag+

plot(res,choix="var",invisible = "ind",title="",cex=1.5)

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

Size

WeightVelocity

Intelligence

AffectivityAggressivness

Chapitre 2 MCA 5/52

MCA is also a method of dimension reduction : it gives a small number of newsynthetic numerical variables summarizing the initial variables.

Categorical data : 8 initial categorical data


Numerical data : 3 syntheticnumerical variables

## Dim 1 Dim 2 Dim 3## Beauceron -0.32 -0.418 -0.10## BassetHound 0.25 1.101 -0.19## GermanShepherd -0.49 -0.464 -0.50## Boxer 0.45 -0.882 0.69## Bulldog 1.01 0.550 -0.16## BullMastiff -0.75 0.547 0.50## Poodle 0.91 -0.016 -0.58## Chihuahua 0.84 0.844 -0.47

MCA is then also a method to transform categorical data into numerical data.

Chapitre 2 MCA 6/52

Plan

1 Basic notions

2 The MCA algorithm

3 Different implementations of MCA

4 Interpretation of the results

Chapitre 2 MCA 7/52

1. Basic notions

Let us consider a data table where n individuals are described on p categoricalvariables.

1 . . . j . . . p1...

...i . . . xij . . .

......

n

Let :

- X = (xij )n×p denote the original data matrix whith xij ∈Mj andMj theset of the levels of the jth variable.

- mj = card(Mj ) denote the number of levels of the jth variable.

- m = m1 + . . .+ mp denote the total number of levels.

Chapitre 2 MCA 8/52

Example : categorical data with n = 27 individuals, p = 6 variables and m = 16levels.print(data[1:8,])


Levels of the variables : T-,T+,T++ (taille), P-,P+,P++ (poids), etc.

Two approaches for recoding the categorical data into numerical data :

- build the disjonctive table where each levels is coded is coded as a binaryvariable,

- build the Burt table (anglo-saxon approach) which gathers thecontingency tables of all the pairs of variables.

Chapitre 2 MCA 9/52

The disjonctive table describes the n individuals on the m levels :

1 . . . s . . . m1...

...K = i . . . kis . . .

......

ntotal ns

Each column s is the indicator vector of the level s with :{kis = 1 if individual i has level skis = 0 otherwise

Let ns denote the number of individuals having level s.

Chapitre 2 MCA 10/52

Disjonctive table of the m = 16 levels.

library(FactoMineR)K <- tab.disjonctif(data)print(K[1:4,])

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## Beauceron 0 0 1 0 1 0 0 0 1 0 1 0 0 1 0 1## BassetHound 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1## GermanShepherd 0 0 1 0 1 0 0 0 1 0 0 1 0 1 0 1## Boxer 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 1

Frequencies ns of the levels :

ns <- apply(K,2,sum)print(ns)

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## 7 5 15 8 14 5 10 8 9 8 13 6 13 14 14 13

Relative frequencies nsn of the levels :

ns <- apply(K,2,sum)n <- nrow(K)print(ns/n)

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## 0.26 0.19 0.56 0.30 0.52 0.19 0.37 0.30 0.33 0.30 0.48 0.22 0.48 0.52 0.52 0.48


Centered disjonctive table.

- The n rows of the matrix K (the disjonctive table) define a cloud of npoints in Rm.

- Each individual i is weighted by wi and usually wi = 1n .

Matrix K of the original recoded data

1 . . . s . . . m1...

...i . . . kis . . .

......

nmean . . . ns

n . . .

Matrix Z of the centered data.

1 . . . s . . . m1...

...i . . . zis = kis − ns

n . . .

......

nmean . . . 0 . . .var . . . ns

n (1− nsn ) . . .

Verify that var(zs) = nsn (1− ns

n ) where zs ∈ Rn denotes s-th column of Z.


Distance between two individuals.

- A weight ms is associated with each level s in order to give moreimportance to rare levels :

ms = nns

- The metric M = diag( nns, s = 1 . . . ,m) of the diagonal matrix of the

weights of the columns gives :

d2M(zi , zi′) =

m∑s=1

nns

(zis − zi′s)2

=m∑

s=1

nns

(kis − ki′s)2

Two individuals are different if they have different levels, with more weight inthe distance for rare levels (ns small).


Example :

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## Beauceron 0 0 1 0 1 0 0 0 1 0 1 0 0 1 0 1## BassetHound 1 0 0 1 0 0 1 0 0 1 0 0 1 0 0 1## GermanShepherd 0 0 1 0 1 0 0 0 1 0 0 1 0 1 0 1## Boxer 0 1 0 0 1 0 0 1 0 0 1 0 0 1 0 1## Bulldog 1 0 0 1 0 0 1 0 0 0 1 0 0 1 1 0

Relative frequency of the levels :

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## 0.26 0.19 0.56 0.30 0.52 0.19 0.37 0.30 0.33 0.30 0.48 0.22 0.48 0.52 0.52 0.48

Squared distance between the two first dogs :

d2M(z1, z2) = 1

0.26 (0− 1)2 + 10.19 (0− 0)2 + . . .+ 1

0.48 (1− 1)2


Inertia of the disjonctive table.

We have seen in the slides about the basic notions for PCA that :

- centering the data doesn’t change the distances between the individualsand then the inertia,

- the inertia of a data table is the (weighted) sum of the variances of itscolumns.

In the particular case of a disjonctive table K this gives :

- I(K) = I(Z) where Z is the centered disjonctive table and :

I(Z) =m∑

s=1

msvar(zs),

where ms is the weight of the column (the level) s.


- This gives when the rows are weighted by 1n and the columns are

weighted by ms = nns

:

I(Z) =m∑

s=1

(1− ns

n ).

In practice :

- The contribution of a level s to the inertia of Z is all the moreimportant as the level is rare.

- Too rare levels are then avoided (by pre-processing for instance).


- This gives also :

I(Z) =p∑

j=1

(mj − 1).

In practice :

- The contribution of a variable j to the inertia of Z is all the moreimporant as its number of levels mj is high.

- Variables with too different number of levels are then avoided (bypre-processing for instance).


- This gives also :I(Z) = m − p.

Example of the dogs :

#number of variablesncol(data)

## [1] 6

#number of levelsncol(K)

## [1] 16

I(Z) = 16− 6 = 10.


The correlation ratio The link between a numerical variable y and a categorical

variable x is often measure by :

η2(y |x) = var(y |x)var(y) =

∑ms=1

nsn (ys − y)2∑n

i=11n (yi − y)2

where m is the number of levels of x and ys is the mean value of y performedwith the individuals having the level s.

- This criterion is often named correlation ratio.

- It takes its values in [0, 1].

- It measures the proportion of the variance of the numerical variable yexplained by the categorical variable x .

In which situation is this criterion equal to 0, equal to 1 ?


Example : The Iris data.

## Sepal.Length Sepal.Width Petal.Length Petal.Width Species## 1 5.1 3.5 1.4 0.2 setosa## 2 4.9 3.0 1.4 0.2 setosa## 50 5.0 3.3 1.4 0.2 setosa## 51 7.0 3.2 4.7 1.4 versicolor## 100 5.7 2.8 4.1 1.3 versicolor## 101 6.3 3.3 6.0 2.5 virginica

Correlation ratios between the variable Species and the 4 numerical variables :

eta2 <- function(x, gpe) {moyennes <- tapply(x, gpe, mean)effectifs <- tapply(x, gpe, length)varinter <- (sum(effectifs * (moyennes - mean(x))^2))vartot <- (var(x) * (length(x) - 1))res <- varinter/vartotreturn(res)

}

apply(iris[,-5],2,function(x){eta2(x,iris$Species)})

## Sepal.Length Sepal.Width Petal.Length Petal.Width## 0.62 0.40 0.94 0.93


The variable Species explains :

- 94.1 % of the variance of "Petal Length".

- 40 % of the variance of "Sepal Length".

Pet

al L

engt

h

setosa versicolor virginica

12

34

56

7

Sep

al W

idth

setosa versicolor virginica2.

02.

53.

03.

54.

0


Give an interpretation of the graphical outputs below :

res <- PCA(iris,quali.sup = 5,graph=FALSE)plot(res,choix="ind",habillage=5,

title="",label="none",invisible="quali")

−2 0 2 4

−3

−2

−1

01

23

Dim 1 (72.96%)

Dim

2 (

22.8

5%)

setosaversicolorvirginica

plot(res,choix="var",title="",cex=1.5)

−1.0 −0.5 0.0 0.5 1.0

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (72.96%)

Dim

2 (

22.8

5%)

Sepal.Length

Sepal.Width

Petal.LengthPetal.Width

How is this interpretation coherent with the results of the correlation ratios ?


Plan

1 Basic notions

2 The MCA algorithm




2. The MCA algorithm

Several algorithms exist to perform Multiple Correspondance Analysis (MCA)and MCA can be defined as :

- Correspondance Analysis (CA) applied to the Burt table (anglo-saxonapproach) or to the disjonctive table (french approach),

- Principal Component Analysis (PCA) applied to the centered disjonctivetable (the approach describe in this Chapter).

Because the CA method in not studied in this lecture, the MCA algorithmdescribed hereafter is based on the general framework of PCA with metricintroduced in the section 4 of the Chapter I.


The MCA algorithm.

The data table to be analyzed by MCA comprises n individuals described by pcategorical variables and it is represented by the n× p categorical matrix X. Letm denote the total number of levels of the p categorical variables.

Step 1 : the pre-processing step.

1. Build the real matrix Z of dimension n ×m as follows :

↪→ Each level is coded as a binary variable and the n ×m disjonctive tableK is constructed.

↪→ Z is the centered version of K

2. Build the diagonal matrix N of the weights of the rows of Z. The n rowsare often weighted by 1

n , such that N = 1n In.

3. Build the diagonal matrix M of the weights of the columns of Z :

↪→ The m columns (corresponding to the levels of the categorical variables)are weighted by n

ns, where ns , s = 1, . . . ,m denotes the number of

individuals that belong to the sth level.


The metricM = diag( n

n1, . . . ,

nnm

) (1)

indicates that the distance between two rows of Z is weighted euclideandistance in the spirit of the χ2 distance used in CA. This distance gives moreimportance to rare levels. The total inertia of Z with this distance and theweights 1

n is equal to m − p.


Step 2 : the factor coordinates processing step.

1. The Generalized Singular Value Decomposition (GSVD) of Z with metricsN and M gives the decomposition :

Z = UΛVt (2)

where

- Λ = diag(√λ1, . . . ,

√λr ) is the r × r diagonal matrix of the singular

values of ZMZtN and ZtNZM, and r denotes the rank of Z which canbest be here r = min(n − 1,m − p) ;

- U is the n × r matrix of the first r eigenvectors of ZMZtN such thatUtNU = Ir , with Ir the identity matrix of size r ;

- V is the p × r matrix of the first r eigenvectors of ZtNZM such thatVtMV = Ir .


2. The matrix F of dimension n × r of the factor coordinates of theindividuals is defined by :

F = ZMV, (3)

and we deduce from (2) that :

F = UΛ. (4)

1 . . . α . . . r1...

...i . . . fiα . . .

......

nmean . . . 0 . . .var . . . λα . . .

The columns fα of F are theprincipal components and :

var(fα) = λα.

The columns uα = fα√λα

of U are

the standardized principalcomponents.


The matrix Fres <- MCA(data,graph=FALSE)F <- res$ind$coordF[,1:2]

## Dim 1 Dim 2## Beauceron -0.32 -0.418## BassetHound 0.25 1.101## GermanShepherd -0.49 -0.464## Boxer 0.45 -0.882## Bulldog 1.01 0.550## BullMastiff -0.75 0.547## Poodle 0.91 -0.016## Chihuahua 0.84 0.844## CockerSpaniel 0.73 0.079## Collie -0.12 -0.526## Dalmatien 0.65 -0.990## Doberman -0.87 -0.315## GermanMastiff -1.05 0.507## BittanySpaniel 0.48 -1.037## FrenchSpaniel -0.14 -0.516## FoxHound -0.88 0.025## FoxTerrier 0.88 0.139## GrdBleuGascon -0.52 -0.113## Labrador 0.65 -0.990## Levrier -0.68 -0.083## Mastiff -0.76 0.888## Pekingese 0.84 0.844## Pointer -0.67 -0.424## SaintBernard -0.58 0.594## Setter -0.50 -0.377## Teckel 1.01 0.550## Newfoundland -0.38 0.485

Individuals plotted according to the twofirst PCsplot(res,choix="ind",invisible="var",

cex=1.5,title="")

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

Beauceron

BassetHound

GermanShepherd

Boxer

BulldogBullMastiff

Poodle

Chihuahua

CockerSpaniel

Collie

Dalmatien

Doberman

GermanMastiff

BittanySpaniel

FrenchSpaniel

FoxHoundFoxTerrier

GrdBleuGascon

Labrador

Levrier

MastiffPekingese

Pointer

SaintBernard

Setter

TeckelNewfoundland

res$eig[1:2,1]

## [1] 0.48 0.38


3. The matrix A∗ of dimension m × r of the factor coordinates of the levelsis defined by :

A∗ = MZtNU = MA, (5)

and we deduce from (2) that :

A∗ = MVΛ. (6)

1 . . . α . . . q1...

...s . . . a∗sα . . .

......

m

Each coordinate a∗sα (element of A∗) isthe mean value of the (standardized)factor coordinates of the individuals thatbelong to level s :

a∗sα =1ns

∑i :kis =1

fiα√λα

This relation is called the barycentricproperty. This property is fondamental forthe interpretation of the graphical outputsin MCA.


The matrix A∗

A <- res$var$coordA[,1:2]

## Dim 1 Dim 2## S- 1.18 0.924## S+ 0.85 -1.232## S++ -0.84 -0.021## W- 1.17 0.824## W+ -0.31 -0.819## W++ -1.02 0.974## V- 0.32 1.045## V+ 0.60 -0.888## V++ -0.89 -0.372## I- -0.35 0.809## I+ 0.37 -0.286## I++ -0.34 -0.459## Af- -0.84 0.287## Af+ 0.78 -0.267## Ag- 0.40 -0.194## Ag+ -0.43 0.209

Plot of the levels according to their factorcoordinates on dim1-2

plot(res,choix="ind",invisible="ind",cex=1.5,title="")

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

5−

1.0

−0.

50.

00.

51.

0

Dim 1 (28.90%)D

im 2

(23

.08%

)

S−

S+

S++

W−

W+

W++ V−

V+

V++

I−

I+I++

Af−

Af+Ag−

Ag+

The coordinates of the level W++ are the mean of the standardizedcoordinates of the dogs that belong to W++.

rownames(data)[which(data$Weight=="W++")]

## [1] "BullMastiff" "GermanMastiff" "Mastiff" "SaintBernard" "Newfoundland"


Is it possible to plot both individuals and levels on the same map ? It is possible

to plot the levels at the barycenter of the individuals by using the barycentricproperty

a∗sα = 1ns

∑i :kis =1

fiα√λα.

In that case two dimensions are chosen and :

- the individuals are plotted according to their standardized principalcomponents fα√

λα

,

- the levels are plotted according to their factor coordinates vectors a∗α.


Example of the dogs data :

−1.0 −0.5 0.0 0.5 1.0 1.5 2.0

−1

01

2

Levels at the barycenter of the individuals

first standardized PC

seco

nd s

tand

ardi

zed

PC

S−

S+

S++

W−

W+

W++ V−

V+

V++

I−

I+I++

Af−

Af+Ag−

Ag+

Beauceron

BassetHound

GermanShepherd

Boxer

BulldogBullMastiff

Poodle

Chihuahua

CockerSpaniel

Collie

Dalmatien

Doberman

GermanMastiff

BittanySpaniel

FrenchSpaniel

FoxHoundFoxTerrier

GrdBleuGascon

Labrador

Levrier

Mastiff Pekingese

Pointer

SaintBernard

Setter

TeckelNewfoundland

For instance the level W++ is plotted at the barycenter of the dogs thatbelong to W++.

rownames(data)[which(data$Weight=="W++")]

## [1] "BullMastiff" "GermanMastiff" "Mastiff" "SaintBernard" "Newfoundland"


However this simultaneous representation of the levels at the barycenter of theindividuals is not the standard output of softwares implementing MCA wherethe so-called quasi-barycentric property is usually used.

The quasi-barycentric property is simply the barycentric property written asfollows :

a∗sα = 1√λα

(1ns

∑i :kis =1

fiα

)

This reads : each coordinate a∗sα is the mean value of the factor coordinates ofthe individuals that belong to level s, up to the multiplier coefficient 1√

λα

.


It is then possible to plot the levels at the quasi-barycenter of the individuals :

- the individuals are plotted according to their principal components fα,

- the levels are plotted according to their factor coordinates vectors a∗α.

The representation of the levels at the quasi-barycenter of the individuals :

- is the simultaneous representation usually implemented in the softwares,

- must be interpreted as follows : the cloud of the levels is the dilatation(by 1√

λα

in each dimension) of the cloud of the gravity centers of theindividuals.


Example of the dogs data :

−1.0 −0.5 0.0 0.5 1.0 1.5 2.0

−1

01

2

Levels at the quasi−barycenter of the individuals

first PC

seco

nd P

C

S−

S+

S++

W−

W+

W++ V−

V+

V++

I−

I+I++

Af−

Af+Ag−

Ag+

Beauceron

BassetHound

GermanShepherd

Boxer

BulldogBullMastiff

Poodle

Chihuahua

CockerSpaniel

Collie

Dalmatien

Doberman

GermanMastiff

BittanySpaniel

FrenchSpaniel

FoxHoundFoxTerrier

GrdBleuGascon

Labrador

Levrier

Mastiff Pekingese

Pointer

SaintBernard

Setter

TeckelNewfoundland

For instance the level W++ is plotted at the barycenter of the dogs thatbelong to W++ dilated by 1√

λ1= 1√

0.481 = 2.076 on the first dimension.

res$eig[1:2,1]

## [1] 0.48 0.38

apply(F[which(data$Weight=="W++"),1:2],2,mean)/sqrt(res$eig[1:2,1])

## Dim 1 Dim 2## -1.02 0.97


Step 3 : the squared loadings processing step.

The contribution cjα of the variable xj (j-th column of X) to the variance ofthe principal component fα is defined by :

cjα =∑

s∈Mj

ns

n a∗2sα (7)

The matrix C = (cjα) of dimention p × r is called the squared loadings matrixto draw an analogy with squared loadings in PCA.

1 . . . α . . . r1...

...j . . . cjα . . .

......

p

Each element cjα is equal to thecorrelation ratio between xj and fα :

cjα = η2(fα|xj )


The matrix CC <- res$var$eta2C[,1:2]

## Dim 1 Dim 2## Size 0.89 0.502## Weight 0.64 0.725## Velocity 0.41 0.684## Intelligence 0.13 0.280## Affectivity 0.65 0.077## Aggressivness 0.17 0.041

Variables plotted according to theirsquared loadings

plot(res,choix="var",cex=1.5,title="")

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

Size

WeightVelocity

Intelligence



Plan

1 Basic notions

2 The MCA algorithm




3. Different implementations of MCA

1. Implement MCA as a CA of the Burt table : the “anglo-saxon approach”.

- CA is called simple Correspondance Analysis.- In french CA is AFC = Analyse Factorielle des Correspondances.- CA analyses a simple contingency table obtained by crossing twocategorical variables.

- CA is a two steps procedure with first a PCA of the matrix of therow-profiles of the contingency table and then a PCA of the the matrixof the column-profiles. These PCA use specific weights on the rows andcolumns and then specific metrics.

- Applying CA to the Burt table is then applying a single PCA (withspecific metrics) to the matrix of the row-profiles of the Burt table.Indeed the column-profiles are identical to the row-profiles in the Burttable.

Drawback : This algorithm gives the results (factor coordinates) for thelevels but not for the individuals.Implemented in the procedure CORRESP of the SAS sofware.


The Burt table is a symmetric table of size m ×m which gathers thecontingency tables of all the pairs of variables.

1 . . . s ′ . . . m1...

...B = KtK = s . . . bss′ . . .

......

m

where :

- bss′ =∑n

i=1 kiskis′ is the number of individual having both levels s and s ′

- bss = ns is the number of individuals having s.


Example : Burt table of the m = 16 levels.

K <- tab.disjonctif(data)B <- t(K)%*%Kprint(B)

## S- S+ S++ W- W+ W++ V- V+ V++ I- I+ I++ Af- Af+ Ag- Ag+## S- 7 0 0 7 0 0 5 2 0 3 3 1 1 6 5 2## S+ 0 5 0 1 4 0 1 4 0 0 4 1 0 5 3 2## S++ 0 0 15 0 10 5 4 2 9 5 6 4 12 3 6 9## W- 7 1 0 8 0 0 6 2 0 3 4 1 1 7 5 3## W+ 0 4 10 0 14 0 0 6 8 3 7 4 7 7 8 6## W++ 0 0 5 0 0 5 4 0 1 2 2 1 5 0 1 4## V- 5 1 4 6 0 4 10 0 0 4 5 1 5 5 5 5## V+ 2 4 2 2 6 0 0 8 0 1 5 2 2 6 5 3## V++ 0 0 9 0 8 1 0 0 9 3 3 3 6 3 4 5## I- 3 0 5 3 3 2 4 1 3 8 0 0 6 2 3 5## I+ 3 4 6 4 7 2 5 5 3 0 13 0 4 9 8 5## I++ 1 1 4 1 4 1 1 2 3 0 0 6 3 3 3 3## Af- 1 0 12 1 7 5 5 2 6 6 4 3 13 0 5 8## Af+ 6 5 3 7 7 0 5 6 3 2 9 3 0 14 9 5## Ag- 5 3 6 5 8 1 5 5 4 3 8 3 5 9 14 0## Ag+ 2 2 9 3 6 4 5 3 5 5 5 3 8 5 0 13


2. Implement MCA as a CA of the disjonctive table : the “standardapproach”.

- The disjonctive table is used as a contingency table.- Applying CA to the disjonctive table is then a two steps procedure withfirst a PCA of the matrix of the row-profiles (of the individuall) andthen a PCA of the the matrix of the column-profiles (of the levels).

Advantage : This algorithm gives directly the results (factor coordinates)for the levels and for the individuals.Implemented in the function MCA of the R package FactoMineR.


3 Perform a PCA of the disjonctive table : the “single PCA approach”

- This PCA uses specific weights of the columns (the levels) and then aspecific distance between two rows (individuals).

- Compared to the standard approach :- the factor coordinates of the levels are the same.- the factor coordinates of the individuals are multiplied by √p.- the total inertia is multiplied by p and is equal to m − p.

Advantage : It is not necessary to know the CA method to understandthis algorithm.Implemented in the function PCAmix of the R package PCAmixdata.


Plan

1 Basic notions

2 The MCA algorithm




4. Interpretation of the results

Quality of the dimension reduction.

The quality of the q first principal components is measured by the proportionof the inertia that they explain.

Inertia of the data :

I(Z) = I(F) = λ1 + . . .+ λr = m − p.

Proportion of inertia explaine by the α-th principal component.

λα

λ1 + . . .+ λr.

In MCA, the percentage of inertia explained by the axes are "small" byconstruction. Some authors have proposed corrections of the eigenvalues inMCA (Greenacre, 1993).


Original data (p = 6 et m=16)

## Size Weight Velocity Intelligence## Beauceron S++ W+ V++ I+## BassetHound S- W- V- I-## GermanShepherd S++ W+ V++ I++## Boxer S+ W+ V+ I+## Bulldog S- W- V- I+

Reduction to the 3 first PCs

## Dim 1 Dim 2 Dim 3## Beauceron -0.32 -0.42 -0.10## BassetHound 0.25 1.10 -0.19## GermanShepherd -0.49 -0.46 -0.50## Boxer 0.45 -0.88 0.69## Bulldog 1.01 0.55 -0.16

What is the quality of this reduction ?

## Eigenvalue Proportion Cumulative## dim 1 2.890 28.90 29## dim 2 2.308 23.08 52## dim 3 1.266 12.66 65## dim 4 0.945 9.45 74## dim 5 0.901 9.01 83## dim 6 0.740 7.40 90## dim 7 0.489 4.89 95## dim 8 0.274 2.74 98## dim 9 0.141 1.41 100## dim 10 0.046 0.46 100

- r = 10 non nul eigenvalues because r = min(n − 1,m − p) = 10,- The sum of the eigenvalues is m − p = 10 (total inertia),- 64.6 % of the inertia is exaplined by the 3 first PCs.


Contribution of the individuals and of the levels.

- The relative contribution of an individual i to the variance of an axe α is :

1n

f 2iαλα.

The individuals far from the center of the factor map are those whocontribute the most. They can be a source of instability and can beremoved or used as illustrative.

- The relative contribution of a level s to the variance of an axe α is :

ns

na2

sαλα

.

The levels far from the center of the factor map are not necessary thosewho contribute the most.


The 5 individuals which contribute the most.

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

Chihuahua

DalmatienLabrador

Mastiff Pekingese

The 5 levels which contribute the most.

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.

5−

1.0

−0.

50.

00.

51.

0

Dim 1 (28.9 %)

Dim

2 (

23.0

8 %

)

S−

S+

W−

W+

V−


Contribution of the variables.

The absolute contribution of a categorical variable j to the variance of an axe αis the sum of the contributions of its levels :∑

s∈Mj

ns

n a2sα = η2(fα|xj )

The correlation ratios are signless measure of links used to plot the categoricalvariables on a map.

0.0 0.2 0.4 0.6 0.8 1.0

0.0

0.2

0.4

0.6

0.8

Dim 1 (28.9 %)

Dim

2 (

23.0

8 %

) Size

WeightVelocity

Intelligence


−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

S−S+S++

Beauceron

BassetHound

GermanShepherd

Boxer

BulldogBullMastiff

Poodle

Chihuahua

CockerSpaniel

Collie

Dalmatien

Doberman

GermanMastiff

BittanySpaniel

FrenchSpaniel

FoxHoundFoxTerrier

GrdBleuGascon

Labrador

Levrier

MastiffPekingese

Pointer

SaintBernard

Setter

TeckelNewfoundland


Quality of the projection of the individuals and of the level.

The quality of the projection of the individuals or of the levels is measured as inPCA by the so-called squared cosine.

- If two individuals are well projected their distance on the factor map isnot far from their true distance knowing that in MCA the distancebetween two individuals is small if they have the same levels.

- If two levels are well projected, their distance on the factor map can beinterpreted using the barycentric property :

- two levels of two different variables are close if they are owned by thesame individuals.

- two levels of a same variable are close if the two associated groups ofindividuals are close.

- Take care of the dispertion of the individuals associated with each levelsbefore interpretating of the proximity between two levels.


The 10 individuals best projected

−1.5 −1.0 −0.5 0.0 0.5 1.0 1.5

−1.

0−

0.5

0.0

0.5

1.0

Dim 1 (28.90%)

Dim

2 (

23.0

8%)

BassetHound

Bulldog

Chihuahua

Dalmatien

GermanMastiff

BittanySpanielLabrador

MastiffPekingese

Teckel

Levels having a cos 2 > 0.5

−1.0 −0.5 0.0 0.5 1.0 1.5

−1.

5−

1.0

−0.

50.

00.

51.

0

Dim 1 (28.9 %)

Dim

2 (

23.0

8 %

)

S−

S+

S++

W−

W+

V−

Af−

Af+


Documents

Chapter II Multiple Correspondance Analysis (MCA)mchave100p/wordpress/wp-content/...Example:TheIrisdata. ## Sepal.Length Sepal.Width Petal.Length Petal.Width Species ## 1 5.1 3.5 1.4