MAA507, Power method, QR-method and sparse matrix .../Menu/general/column-content/attachment/L7.pdf · Inverse power method QR-method Representing a sparse matrix Cholesky factorization

MAA507,Power

method,QR-methodand sparse

matrix repre-sentation.

ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method

Representing asparse matrix

Choleskyfactorizationand fill-in

TheCuthill-McKeealgorithm

MAA507, Power method, QR-method andsparse matrix representation.

Christopher Engstrom

February 11, 2014

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Lecture 7: Overview

Today we will look at:

I The Power method.

I QR-method.

I If time: A look at sparse matrix representation and ”fillin”.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Why do we need numerical methods?

I think everyone have seen how time consuming it is to find theeigenvalues and eigenvectors analytically by hand.

I In the same way using Gauss-elimination or thecharacteristic polynomial is computationally slow and notvery useful in many real-world systems where we oftenwork with very large matrices.

I We will look at some different methods for different kindsof matrices or depending on if you need all eigenvalues orif it’s enough with one.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Why do we need numerical methods?

If we need to find eigenvalues or eigenvectors of a matrix weshould first ask ourselves:

I Do we already know something about the eigenvalues oreigenvectors we can use? Maybe we know someeigenvalues or have a good approximation already.

I Do we seek only eigenvalues or eigenvectors? Do we needall or only one or a few eigenvalues/eigenvectors?

I Do we have a small or moderately sized matrix or do wehave a large or very large matrix requiring specializedmethods?

I Do we have a full matrix (few zero elements) or sparsematrix with mostly zeros.

I Do our matrix have any useful property or structure wecould use? such as being positive definite or a bandmatrix.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

The Power method is a very simple algorithm for finding theeigenvalue λ1 with the largest absolute value and it’scorresponding eigenvector.

I The method works by iterating xk+1 =Axk||Axk ||

.

I Where we assume there is only 1 eigenvalue on thespectral radius and that A is diagonalizable.

I If the eigenvalue with largest absolute value λ1 6= 0 thenAxk||Axk ||

approaches a vector parallel to the eigenvector of

λ1.

I We get the eigenvalue λ1 from the relationAkx ≈ λ1Ak−1x when k is large.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

We give a short example looking at the following matrix:

I

A =

−2 1 13 −2 01 3 1

I We start with x = [1 1 1]> and iterate:

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

We give a short example looking at the following matrix:

I

Ax =

−2 1 13 −2 01 3 1

111

=

015

I ||Ax|| =

√26

I x1 =Ax

||Ax||= [0, 1, 5]/

√26

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

Continuing for the next iteration we get:

I

Ax1 =

−2 1 13 −2 01 3 1

015

1√26

=

6−28

1√26

I x2 =Ax1

||Ax1||= [6,−2, 8]/

√104

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

If we continue for a couple of more iterations we get:

I x2 = [0.59,−0.17, 0.78]>.

I x3 = [−0.25, 0.91, 0.33]>.

I x4 = [0.41,−0.61, 0.67]>.

I x7 = [−0.28, 0.85,−0.44]>.

I x10 = [0.28,−0.79, 0.55]>.

I We see that it seems to converge, but note that it couldtake some time.

I For reference the true eigenvector isxtrue = [0.267,−0.802, 0.525].

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method: convergence speed

To show why and how fast the method converge we assume Ais diagonalizable with eigenvalues λ1, λ2, . . . , λm such that|λ1| > |λj |, j > 1 and corresponding eigenvectorsv1, v2, . . . , vm

I We write the initial vector x0 as a linear combination ofthe eigenvectors:

x0 = c1v1 + c2v2 + . . .+ cmvm

I Assuming c1 6= 0 (which is true if we choose x0 randomlywith a high probability) after k iterations we get:

Akx0 = c1Akv1 + c2Akv2 + . . .+ cmAkvm

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





Akx0 = c1Akv1 + c2Akv2 + . . .+ cmAkvm

I Since λjvj = Avj we can rewrite this:

Akx0 = c1λk1v1 + c2λ

k2v2 + . . .+ cmλ

kmvm

I Which can be rewritten by:

Akx0 = c1λk1

(v1 +

c2

c1

(λ2

λ1

)k

v2 + . . .+cmc1

(λmλ1

)k

vm

)

I Since |λ1| > |λj |, j > 1 the expression on the right sidewill converge to v1.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





For the power method we had

xk+1 =Axk||Axk ||

Which gives:

xk =Akx0

||Akx0||Since the scaling doesn’t effect the convergence speed it’sobvious from the previous slide that xk converge to somemultiple of the eigenvector v1. We can also see that theconvergence speed depend on the relative size between the

largest and second largest eigenvalue

∣∣∣∣λ2

λ1

∣∣∣∣.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method: Time complexity

We have seen that the method converge geometrically butwhat is the work needed in every iteration?

In every iteration we need to do one matrix × vectormultiplication which can be done in O(n2) time, where n is thelength of the vector.

Calculating the normalization constant is a summation of nelements (O(n)) and normalizing the vector with this constantis done by n multiplications (and one division).

Thus we find that in every iteration we need to do O(n2)operations. Unless λ2 is very close to λ1 we also get O(n2) forthe whole algorithm itself.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method: Time complexity

If the matrix is sparse with on average p non zero elements onevery row the matrix × vector multiplication can instead bedone in O(pn) time.

In practice in many applications p does not increase with n(such as for many real life networks), thise means we essentiallyhave a linear algorithm O(n), at least if we know that λ2 isunlikely to be very close to λ1

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

There are a couple of things to consider when using the Powermethod:

I If there is more than one eigenvalue on the spectral radiusor if it’s multiplicity is larger than one, the method won’tconverge.

I Since the error is of the order |λ2/λ1|k , if λ2 is close to λ1

the convergence can be very slow.

I We only get the (in absolute value) largest eigenvalue andcorresponding eigenvector.

I There are however ways to handle some of theseweaknesses as we will see later.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

While the method does have a couple of negatives, we foundthat it does have a couple of very strong points as well.

I In every step we make only a single vector-matrixmultiplication.

I If A is a sparse matrix the iterations will be fast even forextremely large matrices.

I Since we do no matrix decomposition (as we do in mostother methods) we do not ”destroy” the sparcity of thematrix when calculating the matrix decomposition.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Power method

But how can we handle the problems found earlier?

I We will start by looking at something called the inversepower method.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Inverse power method

We consider the matrix invertible matrix A with one eigenpairλ, x .

I We then have Ax = λx from the definition. This gives:

I

Ax = λx⇔ (A−µI)x = (λ−µ)x⇔ x = (A−µI)−1(λ−µ)x

⇔ (λ− µ)−1x = (A− µI)−1x

I We see that (λ− µ)−1, x is an eigenpair of (A− µI)−1.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





If we instead iterate using this new matrix (A− µI)−1.

I The method becomes:

xk+1 =(A− µI)−1xk||(A− µI)−1xk ||

.

I This will converge to the eigenvector of the dominanteigenvalue of (A− µI)−1.

I We get the dominant eigenvalue (λ− µ)−1 for the λclosest to µ.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





If we look at the error we get:

I ∣∣∣∣∣(λ(second closest to µ) − µ)−1

(λ(closest to µ) − µ)−1

∣∣∣∣∣k

I

⇔∣∣∣∣ (λ(closest to µ) − µ)

(λ(second closest to µ) − µ)

∣∣∣∣k

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





We can easy see that if µ is a good approximation of aneigenvalue of A the error:∣∣∣∣ (λ(closest to µ) − µ)

(λ(second closest to µ) − µ)

∣∣∣∣kwill be very close to zero and the method converge very fast.Often in only one or a couple iteration.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





By using different µ we can find different eigenvectors of A, notonly the dominant one.

I However we still have the problem with eigenvalues withmultiplicity larger than one, or more than one with thesame absolute value.

I The Inverse power method is often used when you have agood approximation of an eigenvalue and need to find anapproximation of the corresponding eigenvector.

I The inverse power method requires us to either solve alinear system or calculate the inverse of the matrix(A− µ). This makes it unsuitable for huge sparse matriceswhere the basic power algorithm works better.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Inverse power method: time complexity

So we found that using the inverse method we could correctsome problems with the method, but this also comes with aprice in that we must be able to calculate the inverse of thematrix (A− µ).

I Inverting a n × n matrix using for example Gauss-Jordanrequires O(n3) operations.

I Inverting a matrix typically destroys the sparcity of thematrix, such that we can no longer use the sparcity of Aduring our iterations anymore.

I Even if we find an effective way to calculate the inversematrix, the iteration steps typically require much moretime than the basic algorithm for sparse matrices.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

Often it’s not enough to find only one eigenvalue or eigenvectorbut you need all of them.

I In general, when you use a program to find eigenvalues andeigenvectors it’s nearly always the QR-method you use.

I But before we look at the method we will take a shortlook at QR-factorization and why this is relevant.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-decomposition

TheoremEvery n ×m matrix A have a matrix decomposition

A = QR

. where

I R is a n ×m upper triangular matrix..

I Q is a n × n unitary matrix.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-decomposition

A unitary matrix is a square matrix satisfying:

I UHU = UUH = I

I In other words U−1 = UH .

I Where UH is the conjugate transpose (AH)ij = (A)ji .

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-decomposition

We let U be a unitary matrix, some important properties are:

I The condition number κ(U) = 1, making for verywell-conditioned and robust systems or algorithms.

I Since U−1 = UH we can very easily find the inverse of U.

I This means similarity transforms UHAU involving U isboth easy to compute and robust.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-decomposition

If A is a square real matrix then Q is an orthogonal matrixQ> = Q−1.

I QR-factorization plays an important role in theQR-method commonly used to calculate eigenvalues andeigenvectors.

I The QR-factorization can be computed using for exampleHouseholder transformations or the Gram-Schmidt process.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

The QR-method is an iterative algorithm for computing theeigenvalues and eigenvectors of a matrix A usingQR-factorization.

I We start by assigning A0 := A

I In every step we then compute the QR-decompositionAk = QkRk .

I We then formAk+1 = RkQk = QH

k QkRkQk = QHk AkQk = Q−1

k AkQk

I So all Ak are similar and therefor have the sameeigenvalues.

I When the method converge Ak will be a triangular matrixand therefor have it’s eigenvalues on the diagonal.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

The main advantage of the QR-method is that it’s not onlyfast when implemented with some necessary modifications.

I It’s also very stable in that a small change in A only givesa small change in the computed eigenvalues.

I This is one of the main reason it’s used over other similarmethods such as for example methods using the LUPdecomposition.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

The QR-method can be seen as a variation of the Powermethod but instead of only one vector, we work with acomplete basis of vectors, using QR-decomposition tonormalize and orthogonalize in every step.

I As with the Power method, the QR-method in it’s basicform have a couple of things to consider.

I Computing the QR-decomposition of a matrix isexpensive, and we need to do it in every step.

I The convergence can be very slow.

I Like the Power method we have problems with complexand non simple eigenvalues.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

I To solve the first problem the matrix A is nearly alwaysfirst brought to upper Hessenberg form.

I This will both decrease the cost of the QR-factorizationsas well as improve the convergence

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Hessenberg matrix

A matrix A is a (upper) Hessenberg matrix if all elementsaij = 0, i > j + 1. In other words A is of the form:

A =

F F F · · · F F FF F F · · · F F F0 F F · · · F F F0 0 F · · · F F F...

......

. . ....

......

0 0 0 · · · F F F0 0 0 · · · 0 F F

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Hessenberg matrix

TheoremEvery square matrix A have a matrix decomposition

A = UHHU

where

I U is a unitary matrix.

I H is a Hessenberg matrix.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Hessenberg matrix

Since U is a unitary matrix, A and H are similar and we canwork with H rather than A in the QR-method.

I This is useful for a couple of reasons.

I Calculating the QR-factorization of a Hessenberg matrix ismuch faster than for a general matrix.

I Since multiplication of a Hessenberg matrix with atriangular matrix (both in the same direction) is itself aHessenberg matrix. We need only do the Hessenbergfactorization once.

I Since a Hessenberg matrix is nearly triangular, itsconvergence towards the triangular matrix will beimproved.

I Since U is unitary, it’s condition number is 1, numericalerros when finding H is not amplified.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

However the convergece can still be very slow even whenworking with the Hessenberg form.

I We remember how we could solve this in the Powermethod by shifting the eigenvalues of the matrix.

I We can do the same in the QR-method using (Ak − σI ).

I Rewriting as before we getAk+1 = QH

k (Ak − σI)Qk + σI = QHk (Ak)Qk .

I We see that Ak ,Ak+1 is still similar and therefor will havethe same eigenvalues.

I As in the Power method we want σ to be as close aspossible to one of the eigenvalues.

I The diagonal elements of the hessenberg matrix prove togive a good approximation.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

I Since the diagonal elements of Ak should give increasinglyaccurate approximation of the eigenvalues we can changeour shift to another eigenvalue or improve the current onebetween iterations as well.

I As fast as a subdiagonal element is close to zero, we haveeither found an eigenvalue if it’s one of the outer elements.

I Or we can divide the problem in two smaller ones since wewould have:

I A =

[A1,1 A1,2

0 A2,2

]

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

I In practice an implicit method using multiple shifts at thesame time is usually used.

I This also solves the problems with complex eigenvalues inreal matrices.

I The Implicit version is also called the Francis algorithm.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

To find the eigenvectors

I The eigenvectors can either be found by multiplyingQ = Q0Q1 . . .Qk with the initial unitary matrix to bringthe problem to Hessenberg from.

I They can also be found through the use of a form ofinverse iteration.

I It’s also possible to use for example Gauss eliminationusing the eigenvalues on our original matrix.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method

We give a summary of the modified method using a singleshift:

1. Compute the Hessenberg form (A = UHHU) using forexample Householder transformations.

2. Assign A0 = H. Then in every step, choose a shift σ asone of the diagonal elements of Ak .

3. Compute the QkRk factorization of (Ak − σI).

4. Compute Ak+1 = RkQk + σI

5. Iterate 2− 4 until Ak is a triangular matrix, optionallydividing the problem in two whenever a subdiagonalelement becomes zero.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




QR-method: time complexity

We said that the QR-method is fast, but how fast is it? Themost expensive steps are:

I Bringing the matrix to Hessenberg form can be done inO(n3) time.

I Finding the QR decomposition of a upper Hessenbergmatrix can be done in O(n2) time (compared to O(n3) fora full matrix).

I If we assume we need O(n) iterations we end up withO(n3) if we bring it to Hessenberg form, and O(n4) if not.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Representing a general sparse matrix

I To store all elements in a n ×m matrix we would need tobe able to store nm elements.

I But what if most of these elements was zero? We wouldlike to store only the non-zero elements.

I Sometimes we have a general sparse matrix and sometimesit might also have some useful structures we can use tostore it more efficiently.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





The Yale format is a format for storing a general sparse matrix.

I Yale is very similar to compressed sparse row (CSR) andcompressed sparse column (CSC).

I Yale stores a sparse matrix M using 3 vectors.

I A containing the non-zero elements.

I IA containing the index in A of the first non-zero elementon every row of M.

I JA contains the column number for all the elements in A.

I For both IA and JA the first row/column is consideredrow/column zero.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





Consider the matrix M:11 22 0 0 33 00 0 44 0 0 0

55 66 0 0 0 00 0 0 77 0 88

I We get:

A =[11 22 33 44 55 66 77 88

]I

IA =[0 3 4 6 8

]I

JA =[0 1 4 2 0 1 3 5

]

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





The Yale format is constructed to allow not only efficientstorage, but also efficient calculations as well.

I Think for a moment how you would implement addition oftwo matrices in Yale format.

I What about multiplication with a vector?

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Representing sparse band matrices

One common type of sparse matrices are band matrices whereall non-zero elements lieas close to the diagonal. Given amatrix A:

I The lower (left) half-bandwidth is the smallest k1 suchthat:

aij = 0, i > j + k1

I The upper (right) half-bandwidth is the smallest k2 suchthat:

aij = 0, i < j − k2

I The matrix bandwidth b is then:

b = k1 + k2 + 1

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





We consider the matrix below.

11 0 0 0 0 021 22 0 24 0 00 0 0 34 0 00 0 43 44 0 460 0 0 54 0 560 0 0 0 65 66

I All elements below the sub-diagonal is zero so we get

lower bandwidth k1 = 1.

I Similarily we get upper bandwidth k2 = 2.

I And bandwidth b = 4.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





A n ×m matrix A with bandwidth b can be represented by amatrix B of size n × b.

I We let every row of our new matrix represent the elementsstarting k1 elements to the left of the diagonal and endingk2 elements to the right.

I This way every column of B represent one of the diagonalsof A.

I We also need to store which column of B contains thediagonal of A.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





The mattrix to the left can be represented by the matrix to theright:

A =

11 0 0 0 0 021 22 0 24 0 00 0 0 34 0 00 0 43 44 0 460 0 0 54 0 560 0 0 0 65 66

B =

0 11 0 021 22 0 240 0 34 0

43 44 0 4654 0 56 065 66 0 0

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cholesky factorization and fill-in

Cholesky factorization is used to decompose a matrix A suchthat A = GGH , where A is Hermitian and positive-definite andG is lower triangular.

I Doing a matrix factorization of a sparse matrix oftendestroys the sparcity of the matrix by adding many newnon-zero elements (so called fill in).

I Permutating rows and columns can greatly reduce this fillin.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





We consider a real matrix matrix A with Cholesky factorizationA = GG>.

A =

F F F 0 0F F F F 0F F F F F0 F F F F0 0 F F F

G =

F 0 0 0 0F F 0 0 0F F F 0 0X F F F 0X X F F F

Where F represent a non-zero element. What happens if anyof the X elements in G would be non-zero?

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





GG> =

F 0 0 0 0F F 0 0 0F F F 0 0X1 F F F 0X2 X3 F F F

F F F X1 X2

0 F F F X3

0 0 F F F0 0 0 F F0 0 0 0 F

=

F F F Y1 Y2

F F F F Y3

F F F F FY1 F F F FY2 Y3 F F F

I If X1 6= 0 then Y1 6= 0 and if X2 6= 0 then Y2 6= 0.

I If X1 = X2 = 0 then if X3 6= 0 then Y3 6= 0.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method





I If we reduce the bandwidth of A, we also increase thenumber of guaranteed zero elements of G.

I If we could find a suitible permutation matrix we couldpotentially greatly reduce the number of non-zeroelements after the Cholesky factorization.

I Storing it as a bandmatrix we could also allocate space forit beforehand.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

The Cuthill-McKee algorithm is used to permutate a sparsesymmetric matrix A into a bandmatrix with a short bandwidth.

I It works by considering A as the adjacency matrix of agraph.

I The new matrix is then created using a breadth-firstalgorithm starting in one of the ”peripheral” vertices.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

Given a n × n sparse symmetric matrix A. At the end we wantto have a set R with n elements containing the order in whichwe want the n vertices.

I Start by selecting one of the vertices with the lowestdegree (peripheral vertex) x and add it to R: R = {x}.

I Then for i = 1, 2, . . . we iterate until |R| = n:I Find the set of vertices adjacent to vertex i in R but not in

R called Ai .Ai := Adj(Ri ) \ R

I Sort Ai in ascending order using the vertex degree.I Add Ai at the end of R.

I In other words we add vertices using breadth first (firstadd all neighbors to the previously added vertex beforegoing to the next) where the vertices are added fromlowest to highest degree.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

Let us consider the following matrix and correspondinggraph(ignoring self loops):

1 2 3

456

A =

F 0 0 0 0 F0 F F F F F0 F F 0 0 00 F 0 F F 00 F 0 F F FF F 0 0 F F

Doing a cholesky factorization we could end up with a fullupper triangular matrix G ! Instead we want to reduce thepotential fill-in.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

We use Cuthil-McKee to create a band matrix:

I We begin by selecting any of the vertices with minimumdegree (vertex 1 or 3). We choose to begin with R = {1}.

I (i = 1) The only neighbor to vertex 1 is vertex 6 so it isadded next. R = {1, 6}.

I (i = 2) vertex 2 and 5 are the neighbors to node 6 notalready in R: A2 = {2, 5}. We sort them in ascendingorder acording to degree and add them to R:R = {1, 6, 5, 2}.

I (i = 3) node 4 is the only vertex adjacent to vertex 5 notalready in R so we add it next. R = {1, 6, 5, 2, 4}

I (i = 4) node 3 is the only vertex adjacent to vertex 2 notalready in R so we add it next. R = {1, 6, 5, 2, 4, 3}

I Since R contains 6 elements we are finished.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

Creating the new matrix by ordering them {1, 6, 5, 2, 4, 3} gives:

1 2 3

456

A =

F F 0 0 0 0F F F F 0 00 F F F F 00 F F F F F0 0 F F F 00 0 0 F 0 F

Doing a cholesky factorization we would now probably end upwith many more zero elements in G than before, saving bothstorage and computation time.

MAA507,Power



ChristopherEngstrom

The Powermethod

Inverse powermethod

QR-method




Cuthill-McKee

Some things to note about the Cuthill-McKee algorithm andfill-in for the Cholesky factorization:

I While the algorithm generally finds a ”good” ordering toreform a matrix as a bandmatrix, it generally does not findthe ”best” ordering.

I Finding the ordering which gives the least possible fill-incan actually be shown to be NP-hard. In other words itprobably doesn’t even have a polynomial time algorithm.

Documents

MAA507, Power method, QR-method and sparse matrix .../Menu/general/column-content/attachment/L7.pdf · Inverse power method QR-method Representing a sparse matrix Cholesky factorization