Canonical analysis

CANONICAL ANALYSISWei-Jiun, Shen Ph. D.

Purpose

To analyze the relationship between 2 sets of variables

Multiple IVs Multiple DVs

Kinds of research questions

It is considered a descriptive technique or a screening procedure rather than hypothesis-testing procedure Number of canonical variate pairs interpretation of canonical variates Importance of canonical variates Canonical variate scores

Limitations to factor analysis

Theoretical issues Interpretability Linear relationship Sensitivity Causality

Practical issues Ratio of cases to IVs 10:1 Normality, linearity and homoscedasticity (not required) Missing data Absence of outliers Absence of multicollinearity and singularity

Fundamental equation for canonical analysis

Multiple regression

When Y is more than one…

ipipiii xxxy 2211

Rpiiii xxxy 21

piiipiii xxxyyy 2121


Step 1: division of RR=R𝑦𝑦

− 1R 𝑦𝑥R𝑥𝑥−1 R𝑥𝑦

Id TS TC BS BC1 1.0 1.0 1.0 1.02 7.0 1.0 7.0 1.03 4.6 5.6 7.0 7.04 1.0 6.6 1.0 5.95 7.0 4.9 7.0 2.96 7.0 7.0 6.4 3.87 7.0 1.0 7.0 1.08 7.0 1.0 2.4 1.0

TS TC BS BCTS 1.00

0-.16

1.758 -.34

1TC -.16

11.00

0.110 .857

BS .758 .110 1.000

.051

BC -.341

.857 .051 1.000

R𝑥 𝑥

R𝑦 𝑥

R𝑥𝑦

R𝑦𝑦


Step 2: eigenvalue and eigenvector

R=R𝑦𝑦− 1R 𝑦𝑥R𝑥𝑥

−1 R𝑥𝑦

(R− λ I )K=0

(R 𝑦𝑦−1 R𝑦𝑥 R𝑥𝑥

− 1R𝑥𝑦−𝑟 𝑐𝑖2 I )K𝑞=0

𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒=Λ=[𝑟 𝑐12 ⋯ ⋯⋮ ⋱ ⋮⋮ ⋯ 𝑟𝑐 𝑖2 ]

𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟=K=[𝑘1 ⋯ 𝑘𝑞 ]

Do you smell something?


Step 1: division of R1

1

N

P

N*P

X

11

N

Q

N*Q

Y

11

N

Q

N*P

X Y

P1

1

Q

Q

(P+Q)*(P+Q)

P

PR𝑥𝑥

R𝑦 𝑥

R𝑥𝑦

R𝑦𝑦


Step 2: eigenvalue and eigenvector

1

NN*n

Y

1 2 3 n…1

NN*m

X

1 2 3 m…

𝑒𝑖𝑔𝑒𝑛𝑣𝑎𝑙𝑢𝑒=Λ

… …

𝑒𝑖𝑔𝑒𝑛𝑣𝑒𝑐𝑡𝑜𝑟=K


χ1

χ2

χ3

χ4

X1

X2

X3

X4

X5

η1

η2

η3

η4

Y1

Y2

Y3

Y4

𝑟𝑐 1❑

𝑟𝑐 2❑

𝑟𝑐 3❑

𝑟𝑐 4❑

0 0

Canonical variate χ

Canonical variate η

Canonical correlation

Number of set of canonical correlation

𝜒2=−[𝑁−1−(𝑘𝑥+𝑘𝑦+12 )] ln Λ𝑚

Λ𝑚=∏1

𝑚

(1−λ 𝑖 )

F-test Wilk’s lambda Pillai’s trace Hotelling’s trace Roy’s gcr

Canonical weight

Beta in regression Partialed out due to multicollinearity Instability

χn

X1X2X3X4X5

ηn

Y1

Y2

Y3

Y4

𝑟𝑐𝑛❑λ𝑤𝑥𝑛1

❑

λ𝑤𝑥𝑛2❑

λ𝑤𝑥𝑛3❑

λ𝑤𝑥𝑛4❑

λ𝑤𝑥𝑛5❑

λ𝑤𝑦𝑛1❑

λ𝑤𝑦𝑛2❑

λ𝑤𝑦𝑛3❑

λ𝑤𝑦𝑛4❑

χ 𝑛=∑1

𝑖

𝑋𝑖× λ𝑤𝑥𝑛𝑖 η𝑛=∑1

𝑖

𝑌 𝑖× λ𝑤𝑦𝑛𝑖

Canonical loading

Structure factor loading in FA Criterion: >.3

χn

X1X2X3X4X5

ηn

Y1

Y2

Y3

Y4

𝑟𝑐𝑛❑λ𝑥𝑛1

❑

λ𝑥𝑛 2❑

λ𝑥𝑛3❑

λ𝑥𝑛 4❑

λ𝑥𝑛 5❑

λ 𝑦𝑛1❑

λ 𝑦𝑛2❑

λ 𝑦𝑛3❑

λ 𝑦𝑛4❑

Canonical cross-loading

Correlations of each variable and other canonical variate

λ𝑥𝑛𝑖 : 𝑦❑ =𝑟 𝑐𝑛× λ𝑥𝑛𝑖

❑

λ 𝑦𝑛𝑖: 𝑥❑ =𝑟𝑐𝑛×λ 𝑦𝑛𝑖❑

χn

X1X2X3X4X5

ηn

Y1

Y2

Y3

Y4

𝑟𝑐𝑛❑λ𝑥𝑛 1

❑

λ𝑥𝑛 2❑

λ𝑥𝑛3❑

λ𝑥𝑛 4❑

λ𝑥𝑛5❑

λ 𝑦𝑛1❑

λ 𝑦𝑛2❑

λ 𝑦𝑛3❑

λ 𝑦𝑛 4❑

𝑟𝑐𝑛×λ𝑥𝑛 1❑



𝑟𝑐𝑛×λ𝑥𝑛4❑


𝑟𝑐𝑛×λ 𝑦𝑛1❑



𝑟𝑐𝑛×λ 𝑦𝑛 4❑

Which interpretation approach to use

Priority (Hair et al., 2010)1. Canonical cross-loading2. Canonical loading3. Canonical weight

Redundancy (index)

Variance the canonical variates from the IVs and extract from the DVs, and vice versa

𝑝𝑣 𝑥𝑐=∑1

𝑖 λ𝑥𝑛𝑖2

𝑖

𝑝𝑣 𝑦𝑐=∑1

𝑖 λ𝑦𝑛𝑖2

𝑖

𝑟𝑑=𝑝𝑣×𝑟𝑐𝑛2

Adequacy coefficientRedundan

cy index

Redundancy (index)

Variance the canonical variates from the IVs and extract from the DVs, and vice versa

χn

X1X2X3X4X5

ηn

Y1

Y2

Y3

Y4

𝑟𝑐𝑛❑λ𝑥𝑛 1

❑

λ𝑥𝑛 2❑

λ𝑥𝑛3❑

λ𝑥𝑛 4❑

λ𝑥𝑛 5❑

λ 𝑦𝑛1❑

λ 𝑦𝑛2❑

λ 𝑦𝑛3❑

λ 𝑦𝑛 4❑

𝑝𝑣 𝑥𝑐 𝑝𝑣 𝑦𝑐

𝑟 𝑑η𝑛→ X=𝑝𝑣 𝑥𝑐×𝑟𝑐𝑛2 𝑟 𝑑χ𝑛→Y=𝑝𝑣 𝑦𝑐×𝑟𝑐𝑛2

Some important issue

Importance of canonical variates Test for the significance Canonical correlation >.3 Variate and its own variables Redundancy

Interpretation of canonical variates Mathematical resolution of combining variables Loading >.3

Procedure

1. Research question2. Designing a canonical analysis3. Check the assumptions4. Derive canonical analysis and assess overall

fit5. Interpret the canonical variate6. Validation and diagnosis

PRACTICE

過去學業表現與現在學業表現研究生焦育布想瞭解過去學業表現與現在學業表現之間的關係。他的研究問題是，大學生在高中時期的學業表現是否與現階段的學業表現有關？其中，高中學業表現包含國文、英文、三角函數與線性代數等四個科目的評量分數，大學階段的學業表現指標則包含國文、外語、微積分與統計的評量分數。請以典型相關分析解答此問題。


χ1

HS_LAN

HS_ENG

HS_TRI

HS_LIA

η1

=.994-.99

-.99

-.61

-.30

CO_LAN

CO_ENG

CO_CAL

CO_STA

-.94

-.98

-.13

.15

χ1

HS_LAN

HS_ENG

HS_TRI

HS_LIA

η1

=.965-.01

-.06

.75

.65

CO_LAN

CO_ENG

CO_CAL

CO_STA

-.27

-.17

.73

.77

𝑟 𝑑χ𝑛→Y=.58

𝑟 𝑑χ𝑛→Y=.29

𝑟 𝑑η𝑛→ X=.60

𝑟 𝑑η𝑛→ X=.23

身體活動與智能研究生游志繪依想瞭解身體活動型態對於智力的影響。他的研究問題是，青少年的身體活動與智力之間是否有關？其中，身體活動包含坐式生活、健走、中等強度以及高等強度活動量等四項指標，智力的指標則包含語文、數學邏輯、空間、音樂、肢體動覺、內省、人際與自然觀察的測驗表現。請以典型相關分析解答此問題。


χ1

Strenuous

moderate

Walk

Sedentary

η1

Language

Math

Space

Music=.351

-.98

-.74

-.13

.15

Kinesthesis

Introspection

Interpersonal

Nature science

-.43

-.06

.05

-.01

-.70

-.22

-.44

.01

Data & Analytics

Canonical analysis