Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Inner-outer iterative methods for large sparse

Eigenvalue computations

Melina Freitag

Department of Mathematical Sciences

University of Bath

Informal Postgraduate SeminarUniversity of Bath21st March 2006

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

What is an eigenvalue?

eigenvalue comes from the German word Eigenwert (likeliverwurst, only half of it has been translated).

arises after simplification/discretisation/linearisation of aproblem

can be meaningless intermediate values of a computationmethod in order to find the solution of a problem

sometimes the values have a meaning for the problem(stability analysis)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

What is an eigenvalue?

eigenvalue comes from the German word Eigenwert (likeliverwurst, only half of it has been translated).

arises after simplification/discretisation/linearisation of aproblem

can be meaningless intermediate values of a computationmethod in order to find the solution of a problem

sometimes the values have a meaning for the problem(stability analysis)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Millenium Footbridge

On its opening day, the Millenium footbridge in London startedto wobble under the weight of 100s of people, who, in turn alsostruggled to keep their balance. The bridge had to be closed.After fitting of 37 fluid-viscous dampers and 1 year and £5mlater the problem was fixed.

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Millenium Footbridge and some more applications

What had happened?

Some of the frequencies of the bridge were similar to thecomponents of the pedestrians footsteps, causing vibrationamplification. Finding these natural frequencies amounts tosolving an eigenvalue problem.

Structural dynamics

Quantum Chemistry/Chemical Reactions

Markov chain techniques/Google

Stability analysis of dynamical systems/solution ofdifferential equations

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

A few definitions

Eigenvalues of A ∈ Rn,n are roots of det(A − λI)

Eigenvectors are x 6= 0 such that Ax = λx for someeigenvalue λ ∈ C

Schur Decomposition: There exists an orthogonal matrixQ ∈ C

n,n (QT Q = I) s.t.

QT AQ =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

A few definitions

Eigenvalues of A ∈ Rn,n are roots of det(A − λI)

Eigenvectors are x 6= 0 such that Ax = λx for someeigenvalue λ ∈ C

Schur Decomposition: There exists an orthogonal matrixQ ∈ C

n,n (QT Q = I) s.t.

QT AQ =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Why not calculating the roots of the characteristic

polynomial?

det(A) =∑

σ∈Sn

(

sgn(σ)n∏

i=1

ai,σ(i)

)

σ . . . , permutation

contains n! summands, not very handy

there exists no formula for calculation the roots of apolynomial of degree larger then 5

calculating the roots numerically is unstable

Back to matrices!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Why not calculating the roots of the characteristic

polynomial?

det(A) =∑

σ∈Sn

(

sgn(σ)n∏

i=1

ai,σ(i)

)

σ . . . , permutation

contains n! summands, not very handy

there exists no formula for calculation the roots of apolynomial of degree larger then 5

calculating the roots numerically is unstable

Back to matrices!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Why not calculating the roots of the characteristic

polynomial?

det(A) =∑

σ∈Sn

(

sgn(σ)n∏

i=1

ai,σ(i)

)

σ . . . , permutation

contains n! summands, not very handy

there exists no formula for calculation the roots of apolynomial of degree larger then 5

calculating the roots numerically is unstable

Back to matrices!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Why not calculating the roots of the characteristic

polynomial?

det(A) =∑

σ∈Sn

(

sgn(σ)n∏

i=1

ai,σ(i)

)

σ . . . , permutation

contains n! summands, not very handy

there exists no formula for calculation the roots of apolynomial of degree larger then 5

calculating the roots numerically is unstable

Back to matrices!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Why not calculating the roots of the characteristic

polynomial?

det(A) =∑

σ∈Sn

(

sgn(σ)n∏

i=1

ai,σ(i)

)

σ . . . , permutation

contains n! summands, not very handy

there exists no formula for calculation the roots of apolynomial of degree larger then 5

calculating the roots numerically is unstable

Back to matrices!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Algorithms based on the idea of calculating the

Schur Form

Schur Form:

QT AQ =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Similarity transformation does not change the eigenvalues:

det(QT AQ − λI) = det(QT AQ − QT λQ)

= det(QT )det(A − λI)det(Q)

= det(A − λI)

QR Algorithm

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Algorithms based on the idea of calculating the

Schur Form

Schur Form:

QT AQ =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Similarity transformation does not change the eigenvalues:

det(QT AQ − λI) = det(QT AQ − QT λQ)

= det(QT )det(A − λI)det(Q)

= det(A − λI)

QR Algorithm

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Algorithms based on the idea of calculating the

Schur Form

Schur Form:

QT AQ =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Similarity transformation does not change the eigenvalues:

det(QT AQ − λI) = det(QT AQ − QT λQ)

= det(QT )det(A − λI)det(Q)

= det(A − λI)

QR Algorithm

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

QR-Algorithm I

Basic QR-iteration: given A0 := A compute

Factor Ai = QiRi (QR decomposition)

Ai+1 = RiQi

Ai and A have the same eigenvalues and we hope that

Ai → R =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Work for calculating all the eigenvalues and eigenvectorsof a matrix in O(n3) operations

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

QR-Algorithm I

Basic QR-iteration: given A0 := A compute

Factor Ai = QiRi (QR decomposition)

Ai+1 = RiQi

Ai and A have the same eigenvalues and we hope that

Ai → R =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Work for calculating all the eigenvalues and eigenvectorsof a matrix in O(n3) operations

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

QR-Algorithm I

Basic QR-iteration: given A0 := A compute

Factor Ai = QiRi (QR decomposition)

Ai+1 = RiQi

Ai and A have the same eigenvalues and we hope that

Ai → R =

d11 · · · d1n

0 d11 · · · d2n

0 0. . .

...0 0 · · · dnn

Work for calculating all the eigenvalues and eigenvectorsof a matrix in O(n3) operations

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

QR-Algorithm II

Problem Fill-in for large sparse matrices

0 100 200 300 400

0

50

100

150

200

250

300

350

400

450

nz = 1887

Figure: Sparse matrix

0 100 200 300 400

0

50

100

150

200

250

300

350

400

450

nz = 147114

Figure: One step of QR

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

Only a few eigenvalues (largest, smallest) are required

QR algorithm is too expensive in terms of storage andcomputation time

need iterative methods!

Examples: Power method, Inverse iteration, RayleighQuotient Iteration, Subspace iteration, Lanczos method,Arnoldi’s method

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

Only a few eigenvalues (largest, smallest) are required

QR algorithm is too expensive in terms of storage andcomputation time

need iterative methods!

Examples: Power method, Inverse iteration, RayleighQuotient Iteration, Subspace iteration, Lanczos method,Arnoldi’s method

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

Only a few eigenvalues (largest, smallest) are required

QR algorithm is too expensive in terms of storage andcomputation time

need iterative methods!

Examples: Power method, Inverse iteration, RayleighQuotient Iteration, Subspace iteration, Lanczos method,Arnoldi’s method

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of the Power method

Given x0

yi+1 = Axi

xi+1 = yi+1/‖yi+1‖

λi+1 = xTi+1Axi+1

convergence to λ1 with linear rate|λ2|

|λ1|< 1

only needs one matrix-vector multiplication at each step

Inverse Iteration is power method applied to (A − σI)−1

Rayleigh Quotient iteration with special shiftσi+1 = xT

i+1Axi+1 (quadratic/cubic convergence)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of the Power method

Given x0

yi+1 = Axi

xi+1 = yi+1/‖yi+1‖

λi+1 = xTi+1Axi+1

convergence to λ1 with linear rate|λ2|

|λ1|< 1

only needs one matrix-vector multiplication at each step

Inverse Iteration is power method applied to (A − σI)−1

Rayleigh Quotient iteration with special shiftσi+1 = xT

i+1Axi+1 (quadratic/cubic convergence)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of the Power method

Given x0

yi+1 = Axi

xi+1 = yi+1/‖yi+1‖

λi+1 = xTi+1Axi+1

convergence to λ1 with linear rate|λ2|

|λ1|< 1

only needs one matrix-vector multiplication at each step

Inverse Iteration is power method applied to (A − σI)−1

Rayleigh Quotient iteration with special shiftσi+1 = xT

i+1Axi+1 (quadratic/cubic convergence)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of the Power method

Given x0

yi+1 = Axi

xi+1 = yi+1/‖yi+1‖

λi+1 = xTi+1Axi+1

convergence to λ1 with linear rate|λ2|

|λ1|< 1

only needs one matrix-vector multiplication at each step

Inverse Iteration is power method applied to (A − σI)−1

Rayleigh Quotient iteration with special shiftσi+1 = xT

i+1Axi+1 (quadratic/cubic convergence)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of the Power method

Given x0

yi+1 = Axi

xi+1 = yi+1/‖yi+1‖

λi+1 = xTi+1Axi+1

convergence to λ1 with linear rate|λ2|

|λ1|< 1

only needs one matrix-vector multiplication at each step

Inverse Iteration is power method applied to (A − σI)−1

Rayleigh Quotient iteration with special shiftσi+1 = xT

i+1Axi+1 (quadratic/cubic convergence)

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method I

Power method with initial vector q computesq,Aq, . . . , Akq

idea of Arnoldi’s method: retain past information: after ksteps we have k + 1 vectors q,Aq, . . . , Akq

Given q1, ‖q1‖2 = 1 On subsequent steps k = 1, 2, . . . ,mtake

q̃k+1 = Aqk −

k∑

j=1

qjhjk

where hjk is the Gram-Schmidt coefficienthjk =< Aqk, qj >. Normalise

qk+1 =q̃k+1

hk+1,k

where hk+1,k = ‖q̃k+1‖2

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method I

Power method with initial vector q computesq,Aq, . . . , Akq

idea of Arnoldi’s method: retain past information: after ksteps we have k + 1 vectors q,Aq, . . . , Akq

Given q1, ‖q1‖2 = 1 On subsequent steps k = 1, 2, . . . ,mtake

q̃k+1 = Aqk −

k∑

j=1

qjhjk

where hjk is the Gram-Schmidt coefficienthjk =< Aqk, qj >. Normalise

qk+1 =q̃k+1

hk+1,k

where hk+1,k = ‖q̃k+1‖2

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method II

Definition

For any j the space span{q,Aq, . . . , Aj−1q} is called the jthKrylov subspace associated with A and q and is denoted byKj(A, q).

Matrix representation

The Arnoldi process can be written in the form

AQm = QmHm + qm+1hm+1,meTm

where Hm is square upper Hessenberg.

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method III

For the Lanczos process: Hm is tridiagonal (three termrecurrence)

Approximate eigenvalues and eigenvectors of A can befound from eigenvalues and eigenvectors of much smallermatrix Hm:

Let Qm, Hm and hm+1,m be generated by the Arnoldi process.Let µ be an eigenvalue of Hm with associated eigenvector xnormalised so that ‖x‖2=1. Let y = Qmx ∈ C

n. Then

‖Ay − µy‖2 = |hm+1,m||xm|,

where xm denotes the mth (and last) component of x.

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method III

For the Lanczos process: Hm is tridiagonal (three termrecurrence)

Approximate eigenvalues and eigenvectors of A can befound from eigenvalues and eigenvectors of much smallermatrix Hm:

Let Qm, Hm and hm+1,m be generated by the Arnoldi process.Let µ be an eigenvalue of Hm with associated eigenvector xnormalised so that ‖x‖2=1. Let y = Qmx ∈ C

n. Then

‖Ay − µy‖2 = |hm+1,m||xm|,

where xm denotes the mth (and last) component of x.

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Concepts of Lanczos/ Arnoldi’s method III

For the Lanczos process: Hm is tridiagonal (three termrecurrence)

Approximate eigenvalues and eigenvectors of A can befound from eigenvalues and eigenvectors of much smallermatrix Hm:

Let Qm, Hm and hm+1,m be generated by the Arnoldi process.Let µ be an eigenvalue of Hm with associated eigenvector xnormalised so that ‖x‖2=1. Let y = Qmx ∈ C

n. Then

‖Ay − µy‖2 = |hm+1,m||xm|,

where xm denotes the mth (and last) component of x.

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Example

random complex matrix of dimension n = 144 generated inMatlab:

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

eigenvalues of A

approximation of outer eigenvalues first!

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 5 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 5 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 10 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 10 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 15 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 15 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 20 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 20 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 25 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 25 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 30 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 30 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 35 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 35 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

after 40 Arnoldi steps

−4 −3 −2 −1 0 1 2 3 4

−3

−2

−1

0

1

2

3

Arnoldi after 40 steps

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Outline

1 Motivation

2 Definitions

3 Characteristic Polynomial

4 Eigenvalue algorithms based on similarity transformation

5 Iterative methods

6 Inner-outer iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

interior eigenvalues: Shift-Invert Transformation

Ax = λx

(A − σI)−1x =1

λ − σx

”outer” eigenvalues are the one closest to σ

Problem: requires a solution of a linear system at eachstep:

(A − σI)u = b

Solve is done iteratively (since A is large and sparse) usingMINRES, GMRES, other iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

interior eigenvalues: Shift-Invert Transformation

Ax = λx

(A − σI)−1x =1

λ − σx

”outer” eigenvalues are the one closest to σ

Problem: requires a solution of a linear system at eachstep:

(A − σI)u = b

Solve is done iteratively (since A is large and sparse) usingMINRES, GMRES, other iterative methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

The solve will be inexact

How does the inexact inner solve influence theconvergence property of the outer solve?

For inexact inverse iteration:

(A − σI)yi = xi + resi

If residual resi is chosen to decrease in the same manneras the eigenvalue residual decreases the convergence ratefrom exact solves is recovered

Ongoing research: Situation for Krylov methods

Large sparse

Eigenvalue

computations

Melina Freitag

Outline

Motivation

Definitions

Characteristic

Polynomial

Eigenvalue

algorithms

based on

similarity

transformation

Iterative

methods

Inner-outer

iterative

methods

Large sparse matrices

The solve will be inexact

How does the inexact inner solve influence theconvergence property of the outer solve?

For inexact inverse iteration:

(A − σI)yi = xi + resi

If residual resi is chosen to decrease in the same manneras the eigenvalue residual decreases the convergence ratefrom exact solves is recovered

Ongoing research: Situation for Krylov methods