Lecture 18Eigenvalue Problems II

Shang-Hua Teng

Diagonalizing A Matrix• Suppose the n by n matrix A has n linearly independent

eigenvectors x1, x2,…, xn.

• Eigenvector matrix S: x1, x2,…, xn are columns of S.• Then

n

ASS

11

is the eigenvalue matrix

Matrix Power Ak

• S-1AS = implies A = S S-1

• implies A2 = S S-1 S S-1 = S S-1

• implies Ak = S kS-1

Random walks

How long does it take to get completely lost?

000001

Random walks Transition Matrix1

2

345

6

000001

021

4100

21

310

41000

31

210

21

310

00410

310

0041

210

21

31000

310

100

P

Matrix Powers

• If A is diagonalizable as A = S S-1 then for any vector u, we can compute Aku efficiently

– Solve S c = u– Aku = S kS-1 S c = S k c

• As if A is a diagonal matrix!!!!

Independent Eigenvectors from Different Eigenvalues

• Eigenvectors x1, x2,…, xk that correspond to distinct (all different) eigenvalues are linear independent.

• An n by n matrix that has n different eigenvalues (no repeated ’s) must be diagonalizable

Proof: Show that

implies all ci = 0

011 kk xcxc

Addition, Multiplication, and Eigenvalues

• If is an eigenvalue of A and is an eigenvalue of B, then in general is not an eigenvalue of AB

• If is an eigenvalue of A and is an eigenvalue of B, then in general is not an eigenvalue of A+B

Example

0110

0001

0100

,0010

BA

AB

BA

Spectral Analysis of Symmetric Matrices A = AT (what are special about them?)

Spectral Theorem: Every symmetric matrix has the factorization

A = QQT

with real eigenvalues in and orthonormal eigenvectors in :

A =QQ-1 = QQT with Q-1= QT.

Simply in English

• Symmetric matrix can always be diagonalized

• Their eigenvalues are always real• One can choose n eigenvectors so that they

are orthonormal.

• “Principal axis theorem” in geometry and physics

2 by 2 Case

Real Eigenvalues

222det baccabcacb

ba

cbba

A

2 by 2 Case

so

bc

xxcb

baxIA

ab

xxcb

baxIA

cbba

A

222

2

222

111

1

111

21 xx

The eigenvalues of a real symmetric matrix are real

• Complex conjugate of a + i b is a - i b• Law of complex conjugate :

(a-i b) (c-i d) = (ac-bd) – i(bc+ad)• which is the complex conjugate of

(a+i b) (c+i d) = (ac-bd) + i(bc+ad)

• Claim:

• What can be?

TTTTT xAxxAxxxAxAxAx

AxxT

Eigenvectors of a real symmetric matrix when they correspond to different ’s are

always perpendicular

AyxyAx AA yAyxAx TTTT and and 21

What can the quantity be?

In general, so eigenvalues might be repeated

• Choose an orthogonal basis for each eigenvalue

• Normalize these vector to unit length

Every symmetric matrix has the factorization A = QQT

with real eigenvalues in and orthonormal eigenvectors in :

A =QQ-1 = QQT with Q-1= QT.

Spectral Theorem

Every symmetric matrix has the factorization A = QQT

with real eigenvalues in and orthonormal eigenvectors in :

A =QQ-1 = QQT with Q-1= QT.

Spectral Theorem and Spectral Decomposition

Tnnn

T

Tn

T

n

n xxxxx

xxxA

111

11

1

xi xiT is the projection matrix on to xi !!!!!