- Home
- Documents
*Introduction to Iterative Methods for Solving Sparse ... 4 Basic Iterative Methods Splittings and...*

prev

next

out of 70

View

3Download

0

Embed Size (px)

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Introduction to Iterative Methods for Solving Sparse Linear Systems

Dr. Doreen De Leon

Math 191T, Spring 2019

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Outline

1 Motivation 2 Some Linear Algebra Background 3 Discretization of Partial Differential Equations (PDEs)

Discretizations in One Dimension Discretizations in Two Dimensions

4 Basic Iterative Methods Splittings and Overrelaxation Convergence Results

5 Krylov Subspace Methods Orthogonal Krylov Subspace Methods Biorthogonal Krylov Subspace Methods

6 Preconditioning

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Motivation

A sparse matrix is a matrix having a large number of zeros.

Large sparse linear systems arise as a result of a variety of applications.

Examples appear in combinatorics, network theory (when there is a low density of connections), and science and engineering (often in the form of numerical solutions of partial differential equations).

Focus in this talk: numerical solution of differential equations.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Some Linear Algebra Background

M-matrix: A matrix A is an M-matrix if it satisfies the following properties:

1 aii > 0 for i = 1, 2, . . . , n 2 aij ≤ 0 for i 6= j, i, j = 1, 2, . . . , n 3 A is nonsingular 4 A−1 ≥ 0.

Positive definite matrix: A is positive definite if (Au, u) > 0 for all u ∈ Rn such that u 6= 0. Symmetric positive definite (SPD): A is symmetric positive definite if AT = A and A is positive definite.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Some More Linear Algebra

Two flavors of diagonal dominance:

(weak) diagonal dominance: |ajj| ≥ n∑

i=1 i 6=j

|aij|, j = 1, 2, . . . , n

strict diagonal dominance: |ajj| > n∑

i=1 i 6=j

|aij|, j = 1, 2, . . . , n

Irreducible matrix: A is irreducible if there is no permutation matrix P such that PAPT is block upper triangular.

Theorem: If A is a nonnegative irreducible matrix, then λ ≡ ρ(A) is a simple eigenvalue of A.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Partial Differential Equations

Physical phenomena are often modeled by equations relating several partial derivatives of physical quantities.

Such equations rarely have an explicit solution. Model problems to be used in this talk:

Poisson’s equation: ∂2u ∂x2

+ ∂2u ∂y2

= f (x, y) for (x, y) ∈ Ω, where Ω

is a bounded, open domain in R2.

General elliptic equation: ∂

∂x

( a ∂u ∂x

) +

∂

∂y

( a ∂u ∂y

) = f (x, y).

Steady-state conection-diffusion equation: −∇ · (a∇u) + b · ∇u = f .

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 1-D

Based on local approximations of partial derivatives using low order Taylor expansions. Basic approximations (one-dimension), with step size of h:

Forward difference: du dx ≈ u(x + h)− u(x)

h .

Backward difference: du dx ≈ u(x)− u(x− h)

h (use −h).

Centered difference: du dx ≈ u(x + h)− u(x− h)

2h (from

combining Taylor approximations for u(x + h) and u(x− h)). Taylor series expand the right-hand side: error is O(h) for forward and backward difference and O(h2) for centered difference.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 1-D (cont.)

Second derivative approximation (centered difference approximation): add the Taylor approximations for u(x + h) and u(x− h) and divide by h2 to obtain

d2u dx2

= u(x + h)− 2u(x) + u(x− h)

h2 +O(h2).

Note: O(h2) means that the dominant term in the error as h→ 0 is a constant times h2.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

One-Dimensional Example

Consider the equation

−u′′(x) = f (x), for x ∈ (0, 1) u(0) = u(1) = 0.

Discretize [0, 1]: xi = ih, i = 0, 1, . . . , n + 1; so h = 1(n+1) . Values at the boundary are known, so we only number the interior points xi with i = 1, 2, . . . , n At xi, centered-difference approximation gives

1 h2

(−ui+1 + 2ui − ui−1) = fi,

where ui ≈ u(xi), etc. Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

One-Dimensional Example (cont.)

Resulting linear system: Ax = f , where

A = 1 h2

2 −1 −1 2 −1

. . . . . . . . . −1 2 −1

−1 2

.

This is a sparse linear system, because the coefficient matrix consists mostly of zeros.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Upwind Schemes

Consider the 1-D version of the convection-diffusion equation:

−au′′ + bu′ = 0, x ∈ (0, 1) u(0) = u(1) = 0

The exact solution is u(x) = 1− eRx

1− eR , where R =

b a

.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Upwind Schemes (cont.)

Consideration of the various discretization schemes shows that only forward or backward difference gives a non-oscillating solution approximating the true solution, depending on the sign of b.

Solution: Use forward differencing when b < 0 and use backward differencing when b > 0, giving the upwind scheme.

Define the upwind scheme by

bu′(xi) ≈ 1 2

(b− |b|) (

ui+1 − ui h

) +

1 2

(b + |b|) (

ui − ui−1 h

) .

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 2-D

Consider the Poisson problem

− ( ∂2u ∂x2

+ ∂2u ∂y2

) = f in