С. Т. Kcllcg North Carolina State University

Raleiqh, North Carolina

Solving Nonlinear Eguations with Newton's Method

slam. Society for Industrial and Applied Mathematics

Philadelphia

Contents

Preface xi

How to Get the Software xiii

1 Introduction 1 1.1 What Is the Problem? 1

1.1.1 Notation 1 1.2 Newton's Method 2

1.2,1 Local Convergence Theory 3 1.3 Approximating the Jacobian 5 1.4 Inexact Newton Methods 7 1.5 Termination of the Iteration 9 1.6 Global Convergence and the Armijo Rule 11 1.7 A Basic Algorithm 12

1.7.1. Warning! 14 1.8 Things to Consider 15

1.8.1 Human Time and Public Domain Codes 15 1.8.2 The Initial Iterate 15 1.8.3 Computing the Newton Step 16 1.8.4 Choosing a Solver 16

1.9 What Can Go Wrong? 17 1.9.1 Nonsmooth Functions 17 1.9.2 Failure to Converge 18 1.9.3 Failure of the Line Search 19 1.9.4 Slow Convergence 19 1.9.5 Multiple Solutions 20 1.9.6 Storage Problems 20

1.10 Three Codes for Scalar Equations 20 1.10.1 Common Features 21 1.10.2 newtsol.m 21 1.10.3 chordsol.m 22 1.10.4 secant.m 23

1.11 Projects 24

VII

Contents

1.11.1 Estimating the q-order 24 1.11.2 Singular Problems 25

Finding the Newton Step with Gaussian Elimination 27 2.1 Direct Methods for Solving Linear Equations 27 2.2 The Newton-Armijo Iteration 28 2.3 Computing a Finite Difference Jacobian 29 2.4 The Chord and Shamanskii Methods 33 2.5 What Can Go Wrong? 34

2.5.1 Poor Jacobians 34 2.5.2 Finite Difference Jacobian Error 35 2.5.3 Pivoting 35

2.6 Using nsold.m 35 2.6.1 Input to nsold.m 36 2.6.2 Output from nsold.m 37

2.7 Examples 37 2.7.1 Arctangent Function 38 2.7.2 A Simple Two-Dimensional Example 39 2.7.3 Chandrasekhar H-equation 41 2.7.4 A Two-Point Boundary Value Problem 43 2.7.5 Stiff Initial Value Problems 47

2.8 Projects 50 2.8.1 Chandrasekhar H-equation 50 2.8.2 Nested Iteration 50

2.9 Source Code for nsold.m 51

Newton-Krylov Methods 57 3.1 Krylov Methods for Solving Linear Equations 57

3.1.1 GMRES 58 3.1.2 Low-Storage Krylov Methods 59 3.1.3 Preconditioning 60

3.2 Computing an Approximate Newton Step 61 3.2.1 Jacobian-Vector Products 61 3.2.2 Preconditioning Nonlinear Equations 61 3.2.3 Choosing the Forcing Term 62

3.3 Preconditioners 63 3.4 What Can Go Wrong? 64

3.4.1 Failure of the Inner Iteration 64 3.4.2 Loss of Orthogonality 64

3.5 Using nsoli .m 65 3.5.1 Input to nsol i .m 65 3.5.2 Output from nsol i .m 65

3.6 Examples 66 3.6.1 Chandrasekhar H-equation 66 3.6.2 The Ornstein-Zernike Equations 67 3.6.3 Convection-Diffusion Equation 71

Contents ix

3.6.4 Time-Dependent Convection-Diffusion Equation . . 73 3.7 Projects 74

3.7.1 Krylov Methods and the Forcing Term 74 3.7.2 Left and Right Preconditioning 74 3.7.3 Two-Point Boundary Value Problem 74 3.7.4 Making a Movie 75

3.8 Source Code for nsol i .m 76

4 Broyden's Method 85 4.1 Convergence Theory 86 4.2 An Algorithmic Sketch 86 4.3 Computing the Broyden Step and Update 87 4.4 What Can Go Wrong? 89

4.4.1 Failure of the Line Search 89 4.4.2 Failure to Converge 89

4.5 Using brsola.m 89 4.5.1 Input to brsola.m 90 4.5.2 Output from brsola.m 90

4.6 Examples 90 4.6.1 Chandrasekhar H-equation 91 4.6.2 Convection-Diffusion Equation 91

4.7 Source Code for brsola.m 93

Bibliography 97

Index 103