Introduction to Iterative Methods for Solving Sparse ... 4 Basic Iterative Methods Splittings and Overrelaxation

  • View
    3

  • Download
    0

Embed Size (px)

Text of Introduction to Iterative Methods for Solving Sparse ... 4 Basic Iterative Methods Splittings and...

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Introduction to Iterative Methods for Solving Sparse Linear Systems

    Dr. Doreen De Leon

    Math 191T, Spring 2019

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Outline

    1 Motivation 2 Some Linear Algebra Background 3 Discretization of Partial Differential Equations (PDEs)

    Discretizations in One Dimension Discretizations in Two Dimensions

    4 Basic Iterative Methods Splittings and Overrelaxation Convergence Results

    5 Krylov Subspace Methods Orthogonal Krylov Subspace Methods Biorthogonal Krylov Subspace Methods

    6 Preconditioning

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Motivation

    A sparse matrix is a matrix having a large number of zeros.

    Large sparse linear systems arise as a result of a variety of applications.

    Examples appear in combinatorics, network theory (when there is a low density of connections), and science and engineering (often in the form of numerical solutions of partial differential equations).

    Focus in this talk: numerical solution of differential equations.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Some Linear Algebra Background

    M-matrix: A matrix A is an M-matrix if it satisfies the following properties:

    1 aii > 0 for i = 1, 2, . . . , n 2 aij ≤ 0 for i 6= j, i, j = 1, 2, . . . , n 3 A is nonsingular 4 A−1 ≥ 0.

    Positive definite matrix: A is positive definite if (Au, u) > 0 for all u ∈ Rn such that u 6= 0. Symmetric positive definite (SPD): A is symmetric positive definite if AT = A and A is positive definite.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Some More Linear Algebra

    Two flavors of diagonal dominance:

    (weak) diagonal dominance: |ajj| ≥ n∑

    i=1 i 6=j

    |aij|, j = 1, 2, . . . , n

    strict diagonal dominance: |ajj| > n∑

    i=1 i 6=j

    |aij|, j = 1, 2, . . . , n

    Irreducible matrix: A is irreducible if there is no permutation matrix P such that PAPT is block upper triangular.

    Theorem: If A is a nonnegative irreducible matrix, then λ ≡ ρ(A) is a simple eigenvalue of A.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Partial Differential Equations

    Physical phenomena are often modeled by equations relating several partial derivatives of physical quantities.

    Such equations rarely have an explicit solution. Model problems to be used in this talk:

    Poisson’s equation: ∂2u ∂x2

    + ∂2u ∂y2

    = f (x, y) for (x, y) ∈ Ω, where Ω

    is a bounded, open domain in R2.

    General elliptic equation: ∂

    ∂x

    ( a ∂u ∂x

    ) +

    ∂y

    ( a ∂u ∂y

    ) = f (x, y).

    Steady-state conection-diffusion equation: −∇ · (a∇u) + b · ∇u = f .

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Introduction to Finite Difference Discretizations – 1-D

    Based on local approximations of partial derivatives using low order Taylor expansions. Basic approximations (one-dimension), with step size of h:

    Forward difference: du dx ≈ u(x + h)− u(x)

    h .

    Backward difference: du dx ≈ u(x)− u(x− h)

    h (use −h).

    Centered difference: du dx ≈ u(x + h)− u(x− h)

    2h (from

    combining Taylor approximations for u(x + h) and u(x− h)). Taylor series expand the right-hand side: error is O(h) for forward and backward difference and O(h2) for centered difference.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Introduction to Finite Difference Discretizations – 1-D (cont.)

    Second derivative approximation (centered difference approximation): add the Taylor approximations for u(x + h) and u(x− h) and divide by h2 to obtain

    d2u dx2

    = u(x + h)− 2u(x) + u(x− h)

    h2 +O(h2).

    Note: O(h2) means that the dominant term in the error as h→ 0 is a constant times h2.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    One-Dimensional Example

    Consider the equation

    −u′′(x) = f (x), for x ∈ (0, 1) u(0) = u(1) = 0.

    Discretize [0, 1]: xi = ih, i = 0, 1, . . . , n + 1; so h = 1(n+1) . Values at the boundary are known, so we only number the interior points xi with i = 1, 2, . . . , n At xi, centered-difference approximation gives

    1 h2

    (−ui+1 + 2ui − ui−1) = fi,

    where ui ≈ u(xi), etc. Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    One-Dimensional Example (cont.)

    Resulting linear system: Ax = f , where

    A = 1 h2

     2 −1 −1 2 −1

    . . . . . . . . . −1 2 −1

    −1 2

     .

    This is a sparse linear system, because the coefficient matrix consists mostly of zeros.

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Upwind Schemes

    Consider the 1-D version of the convection-diffusion equation:

    −au′′ + bu′ = 0, x ∈ (0, 1) u(0) = u(1) = 0

    The exact solution is u(x) = 1− eRx

    1− eR , where R =

    b a

    .

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Upwind Schemes (cont.)

    Consideration of the various discretization schemes shows that only forward or backward difference gives a non-oscillating solution approximating the true solution, depending on the sign of b.

    Solution: Use forward differencing when b < 0 and use backward differencing when b > 0, giving the upwind scheme.

    Define the upwind scheme by

    bu′(xi) ≈ 1 2

    (b− |b|) (

    ui+1 − ui h

    ) +

    1 2

    (b + |b|) (

    ui − ui−1 h

    ) .

    Dr. Doreen De Leon Introduction to Iterative Methods for Solving Sparse Linear Systems

  • Outline Motivation

    Some Linear Algebra Background Discretization of Partial Differential Equations (PDEs)

    Basic Iterative Methods Krylov Subspace Methods

    Preconditioning

    Discretizations in One Dimension Discretizations in Two Dimensions

    Introduction to Finite Difference Discretizations – 2-D

    Consider the Poisson problem

    − ( ∂2u ∂x2

    + ∂2u ∂y2

    ) = f in