     # Introduction to Iterative Methods for Solving Sparse ... 4 Basic Iterative Methods Splittings and Overrelaxation

• View
3

0

Embed Size (px)

### Text of Introduction to Iterative Methods for Solving Sparse ... 4 Basic Iterative Methods Splittings and...

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Introduction to Iterative Methods for Solving Sparse Linear Systems

Dr. Doreen De Leon

Math 191T, Spring 2019

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Outline

1 Motivation 2 Some Linear Algebra Background 3 Discretization of Partial Differential Equations (PDEs)

Discretizations in One Dimension Discretizations in Two Dimensions

4 Basic Iterative Methods Splittings and Overrelaxation Convergence Results

5 Krylov Subspace Methods Orthogonal Krylov Subspace Methods Biorthogonal Krylov Subspace Methods

6 Preconditioning

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Motivation

A sparse matrix is a matrix having a large number of zeros.

Large sparse linear systems arise as a result of a variety of applications.

Examples appear in combinatorics, network theory (when there is a low density of connections), and science and engineering (often in the form of numerical solutions of partial differential equations).

Focus in this talk: numerical solution of differential equations.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Some Linear Algebra Background

M-matrix: A matrix A is an M-matrix if it satisfies the following properties:

1 aii > 0 for i = 1, 2, . . . , n 2 aij ≤ 0 for i 6= j, i, j = 1, 2, . . . , n 3 A is nonsingular 4 A−1 ≥ 0.

Positive definite matrix: A is positive definite if (Au, u) > 0 for all u ∈ Rn such that u 6= 0. Symmetric positive definite (SPD): A is symmetric positive definite if AT = A and A is positive definite.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Some More Linear Algebra

Two flavors of diagonal dominance:

(weak) diagonal dominance: |ajj| ≥ n∑

i=1 i 6=j

|aij|, j = 1, 2, . . . , n

strict diagonal dominance: |ajj| > n∑

i=1 i 6=j

|aij|, j = 1, 2, . . . , n

Irreducible matrix: A is irreducible if there is no permutation matrix P such that PAPT is block upper triangular.

Theorem: If A is a nonnegative irreducible matrix, then λ ≡ ρ(A) is a simple eigenvalue of A.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Partial Differential Equations

Physical phenomena are often modeled by equations relating several partial derivatives of physical quantities.

Such equations rarely have an explicit solution. Model problems to be used in this talk:

Poisson’s equation: ∂2u ∂x2

+ ∂2u ∂y2

= f (x, y) for (x, y) ∈ Ω, where Ω

is a bounded, open domain in R2.

General elliptic equation: ∂

∂x

( a ∂u ∂x

) +

∂y

( a ∂u ∂y

) = f (x, y).

Steady-state conection-diffusion equation: −∇ · (a∇u) + b · ∇u = f .

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 1-D

Based on local approximations of partial derivatives using low order Taylor expansions. Basic approximations (one-dimension), with step size of h:

Forward difference: du dx ≈ u(x + h)− u(x)

h .

Backward difference: du dx ≈ u(x)− u(x− h)

h (use −h).

Centered difference: du dx ≈ u(x + h)− u(x− h)

2h (from

combining Taylor approximations for u(x + h) and u(x− h)). Taylor series expand the right-hand side: error is O(h) for forward and backward difference and O(h2) for centered difference.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 1-D (cont.)

Second derivative approximation (centered difference approximation): add the Taylor approximations for u(x + h) and u(x− h) and divide by h2 to obtain

d2u dx2

= u(x + h)− 2u(x) + u(x− h)

h2 +O(h2).

Note: O(h2) means that the dominant term in the error as h→ 0 is a constant times h2.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

One-Dimensional Example

Consider the equation

−u′′(x) = f (x), for x ∈ (0, 1) u(0) = u(1) = 0.

Discretize [0, 1]: xi = ih, i = 0, 1, . . . , n + 1; so h = 1(n+1) . Values at the boundary are known, so we only number the interior points xi with i = 1, 2, . . . , n At xi, centered-difference approximation gives

1 h2

(−ui+1 + 2ui − ui−1) = fi,

where ui ≈ u(xi), etc. Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

One-Dimensional Example (cont.)

Resulting linear system: Ax = f , where

A = 1 h2

 2 −1 −1 2 −1

. . . . . . . . . −1 2 −1

−1 2

 .

This is a sparse linear system, because the coefficient matrix consists mostly of zeros.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Upwind Schemes

Consider the 1-D version of the convection-diffusion equation:

−au′′ + bu′ = 0, x ∈ (0, 1) u(0) = u(1) = 0

The exact solution is u(x) = 1− eRx

1− eR , where R =

b a

.

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Upwind Schemes (cont.)

Consideration of the various discretization schemes shows that only forward or backward difference gives a non-oscillating solution approximating the true solution, depending on the sign of b.

Solution: Use forward differencing when b < 0 and use backward differencing when b > 0, giving the upwind scheme.

Define the upwind scheme by

bu′(xi) ≈ 1 2

(b− |b|) (

ui+1 − ui h

) +

1 2

(b + |b|) (

ui − ui−1 h

) .

Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

• Outline Motivation

Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

Basic Iterative Methods Krylov Subspace Methods

Preconditioning

Discretizations in One Dimension Discretizations in Two Dimensions

Introduction to Finite Difference Discretizations – 2-D

Consider the Poisson problem

− ( ∂2u ∂x2

+ ∂2u ∂y2

) = f in ##### SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS SOLVING SYSTEMS OF EQUATIONS, ITERATIVE METHODS ELM1222
Documents ##### Randomized Iterative Methods for Linear Systems prichtar/papers/random_ ¢  Randomized Iterative Methods
Documents ##### Iterative Methods for Sparse Linear Systems - ??3.6 Sparse Direct Solution Methods ... Iterative methods for solving ... direct solution methods were often preferred to iterative methods
Documents ##### Iterative Methods for Incompressible ybourg/thesis/   Iterative Methods for
Documents ##### Iterative Methods for Image Reconstruction fessler/papers/files/...آ  Iterative Methods for Image Reconstruction
Documents ##### C3 Numerical Methods - Iterative Numerical Methods - Iterative equations The
Documents ##### Iterative projection methods for sparse linear system and ... Iterative projection methods for sparse
Documents ##### DIRECT AND ITERATIVE METHODS FOR BLOCK TRIDIAGONAL reports- .DIRECT AND ITERATIVE METHODS FOR BLOCK
Documents ##### Iterative methods based on splittings for stochastic ... 97)00215-4.pdf ods Jacobi, Gauss-Seidel (GS),
Documents ##### NATURAL PRECONDITIONING AND ITERATIVE METHODS FOR NATURAL PRECONDITIONING AND ITERATIVE METHODS FOR
Documents Documents