Numerical Analysis

A B Tumwesigye CSC2103 2008/2009 1

NUMERICAL METHODS I

A.B. TUMWESIGYE

Contents

1 Introduction 21.1 What is Numerical Analysis? . . . . . . . . . . . . . . . . . . . . . . . . . 41.2 Two issues of Numerical Analysis: . . . . . . . . . . . . . . . . . . . . . . . 41.3 Advantages of Numerical Analysis: . . . . . . . . . . . . . . . . . . . . . . 41.4 Important Notes: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5 Numerical Errors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

1.5.1 Sources of Errors: . . . . . . . . . . . . . . . . . . . . . . . . . . . . 51.5.2 Types of Errors: . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5

2 Numerical Integration 72.1 Manual Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 82.2 Trapezoidal/Trapezium rule . . . . . . . . . . . . . . . . . . . . . . . . . . 8

2.2.1 Composite Trapezoidal Rule . . . . . . . . . . . . . . . . . . . . . . 82.2.2 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.3 Simpsons rule . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122.3.1 Composite Simpsons rule . . . . . . . . . . . . . . . . . . . . . . . 132.3.2 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 162.3.3 Program-FORTRAN (an alternative to MAPLE) . . . . . . . . . . 17

3 Solution to Non-Linear Equations 223.1 Successive Substitution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

3.1.1 Background knowledge . . . . . . . . . . . . . . . . . . . . . . . . . 223.1.2 Successive Substitutions . . . . . . . . . . . . . . . . . . . . . . . . 223.1.3 Convergence criterion . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3.2 Secant Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 273.2.1 Derivation of the secant method . . . . . . . . . . . . . . . . . . . . 273.2.2 Advantages and Disadvantages of the secant method . . . . . . . . 293.2.3 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 The Regular False method . . . . . . . . . . . . . . . . . . . . . . . . . . . 313.3.1 Geometric representation and derivation of the regular false algorithm. 313.3.2 Order of convergence of the Regular algorithm . . . . . . . . . . . . 333.3.3 Advantages and disadvantages of the regular false algorithm . . . . 333.3.4 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

2


3.4 Bisection Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.2 Implementation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.3 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 353.4.4 Explanation on the bisection method . . . . . . . . . . . . . . . . . 353.4.5 Advantages of the bisection method . . . . . . . . . . . . . . . . . . 363.4.6 Disadvantages of the bisection method . . . . . . . . . . . . . . . . 363.4.7 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.5 Newton Raphsons method . . . . . . . . . . . . . . . . . . . . . . . . . . . 383.5.1 Derivation of the Newtons method . . . . . . . . . . . . . . . . . . 383.5.2 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 423.5.3 General Newtons algorithm for extracting roots of positive numbers. 423.5.4 Using Newtons general formula for roots in finding reciprocals of

numbers. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 443.5.5 Some limitations of the Newton Raphsons method . . . . . . . . . 453.5.6 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4 Interpolation 484.1 Review- Linear interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . 484.2 Application . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 514.3 History . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524.4 Extensions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 524.5 Lagrange interpolation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

4.5.1 Alternatively . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 534.5.2 Examples of interpolating polynomials . . . . . . . . . . . . . . . . 544.5.3 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 584.5.4 Error analysis in Lagranges interpolation . . . . . . . . . . . . . . 594.5.5 Rounding errors in Lagrange polynomials . . . . . . . . . . . . . . . 624.5.6 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5 Numerical Differentiation 715.1 Why Numerical techniques for finding derivatives. . . . . . . . . . . . . . . 715.2 Analytic definition of a derivative as compared to a numerical definition . . 715.3 Forward difference approximation . . . . . . . . . . . . . . . . . . . . . . . 72

5.3.1 Analytical derivation of the forward difference approximation. . . . 745.4 Backward difference approximation . . . . . . . . . . . . . . . . . . . . . . 74

5.4.1 Analytical derivation of the backward difference approximation. . . 765.5 The Central difference approximation . . . . . . . . . . . . . . . . . . . . . 77

5.5.1 Analytical derivation of the central difference approximation . . . . 785.6 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 795.7 Comparision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 805.8 The second derivative approximation . . . . . . . . . . . . . . . . . . . . . 80

5.8.1 Error analysis in numerical differentiation. . . . . . . . . . . . . . . 82


5.8.2 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6 Ordinary Differential Equations 866.1 Different forms of ordinary differential equations . . . . . . . . . . . . . . . 87

6.1.1 Initial-value problems . . . . . . . . . . . . . . . . . . . . . . . . . . 876.1.2 Single step methods . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

6.2 Taylor series method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 876.3 Eulers Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

6.3.1 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 926.4 Runge Kutta Methods -The Improved Euler method . . . . . . . . . . . . 95

6.4.1 Runge-kutta two stage method of order two . . . . . . . . . . . . . 966.4.2 Runge-kutta classical four stage method of order two . . . . . . . . 1006.4.3 Text Questions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

7 Sample Questions 107

8 Further Reading 116

Chapter 1

Introduction

Most real mathematical problems do not have analytical solutions. However, they dohave real solutions. In order to obtain these solutions we must use other methods such asgraphical representations, or numerical analysis. Numerical analysis is the mathematicalmethod that uses numerical approximations to obtain numerical answers to the problem.Numerical analysis also considers the accuracy of an approximation, and when the ap-proximation is good enough. Numerical answers are useful because we use numbers tobuild our world, not with the exact analytical solution, such as e

pi27

The ever-increasing advances in computer technology has enabled many in science andengineering to apply numerical methods to simulate physical phenomena. Numericalmethods are often divided into elementary ones such as finding the root of an equation,integrating a function or solving a linear system of equations to intensive ones like thefinite element method. Intensive methods are often needed for the solution of practi-cal problems and they often require the systematic application of a range of elementarymethods, often thousands or millions of times over. In the development of numericalmethods, simplifications need to be made to progress towards a solution: for examplegeneral functions may need to be approximated by polynomials and computers cannotgenerally represent numbers exactly anyway. As a result, numerical methods do not usu-ally give the exact answer to a given problem, or they can only tend towards a solutiongetting closer and closer with each iteration. Numerical methods are generally only usefulwhen they are implemented on computer using a computer programming language .

The study of the behavior of numerical methods is called numerical analysis. This is amathematical subject that considers the modeling of the error in the processing of nu-merical methods and the subsequent re-design of methods.

Numerical analysis involves the study of methods of computing numerical data. In manyproblems this implies producing a sequence of approximations; thus the questions involvethe rate of convergence, the accuracy (or even validity) of the answer, and the complete-

2


ness of the response. (With many problems it is difficult to decide from a programstermination whether other solutions exist.) Since many problems across mathematics canbe reduced to linear algebra, this too is studied numerically; here there are significantproblems with the amount of time necessary to process the initial data. Numerical so-lutions to differential equations require the determination not of a few numbers but ofan entire function; in particular, convergence must be judged by some global criterion.Other topics include numerical simulation, optimization, and graphical analysis, and thedevelopment of robust working code.

Numerical linear algebra topics: solutions of linear systems AX = B, eigenvalues and eigen-vectors, matrix factorizations. Calculus topics: numerical differentiation and integration,interpolation, solutions of nonlinear equations f(x) = 0. Statistical topics: polynomialapproximation, curve fitting.

Further information on the elementary methods can be found in books on numericalmethods or books on numerical analysis. Dedicated text books can be found on each ofthe intensive methods. Details of available books can be accessed through www.science-books.net .

Need help understanding numerical methods?

(1) What is the use of numerical methods in real life application?

(2) Need a brief explanation of numerical methods

(3) Fixed Point Iteration, Linear Interpolation and Newton-Raphson Method, what arethe differences to their uses?

Best Answer

1. um, everywhere? From a cash machine, to calculating how much chemicals to putto produce laundry detergent, to construction of buildings and bridges.

2. The ever-increasing advances in computer technology has enabled many in scienceand engineering to apply numerical methods to simulate physical phenomena. Nu-merical methods are often divided into elementary ones such as finding the root ofan equation, integrating a function or solving a linear system of equations to in-tensive ones like the finite element method. Intensive methods are often needed forthe solution of practical problems and they often require the systematic applicationof a range of elementary methods, often thousands or millions of times over. Inthe development of numerical methods, simplifications need to be made to progresstowards a solution: for example general functions may need to be approximated bypolynomials and computers cannot generally represent numbers exactly anyway. Asa result, numerical methods do not usually give the exact answer to a given prob-lem, or they can only tend towards a solution getting closer and closer with each

4 MAK-ICT

iteration. Numerical methods are generally only useful when they are implementedon computer using a computer programming language .

3. Visit these sites: http://math.fullerton.edu/mathews/n2003/FixedPointMod.htmlhttp://en.wikipedia.org/wiki/Linear-interpolationhttp://mathworld.wolfram.com/NewtonsMethod.html

Other answers

2. Numerical Methods refers to procedures to find approximate solutions when exactsolutions cannot be found in a straightforward manner.

3. Linear interpolation assumes that if two points on a graph are given, any point inbetween them can be found by connecting the original two points by a straight line.

Newton Raphson is a method to find approximate solutions to an equation throughan iterative process where each calculated value is used as the starting point for thenext calculated value. NRM requires that you can evaluate the first derivative ofthat equation.

1.1 What is Numerical Analysis?

- It is a way to do highly complicated mathematics problems on a computer.

- it is also known as a technique widely used by scientists and engineers to solve theirproblems.

1.2 Two issues of Numerical Analysis:

- How to compute? This corresponds to algorithmic aspects;

- How accurate is it? That corresponds to error analysis aspects.

1.3 Advantages of Numerical Analysis:

- It can obtain numerical answers of the problems that have no analytic solution.

- It does NOT need special substitutions and integrations by parts. It needs only thebasic mathematical operations: addition, substitution, multiplication and division,plus making some comparisons.


1.4 Important Notes:

- Numerical analysis solution is always numerical.

- Results from numerical analysis is an approximation.

1.5 Numerical Errors

When we get into the real world from an ideal world and finite to infinite, errors arise.

1.5.1 Sources of Errors:

- Mathematical problems involving quantities of infinite precision.

- Numerical methods bridge the precision gap by putting errors under firm control.

- Computer can only handle quantities of finite precision.

1.5.2 Types of Errors:

- Truncation error (finite speed and time)

Truncation error is a consequence of doing only a finite number of steps in a cal-culation that would require an infinite number of steps to do exactly. A simpleexample of a calculation that will be affected by truncation error is the evaluationof an infinite sum using the NSum function. The computer certainly isnt going tocompute values for all of the terms in an infinite sum. The terms that are left outlead to truncation error.

Truncation Error: The essence of any numerical method is that it is approximatethis usually occurs because of truncation, e.g., cosx = 1 x22 or terminating aninfinite sequence of operations after a finite number have been performed.

It is not possible by numerical techniques alone to get an accurate estimate of thesize of the truncation error in a result. It is possible for any purely numerical al-gorithm, including the algorithms used by numerical functions in Mathematica, toproduce incorrect results, and to do so without warning. The only way to be certainthat results from functions like NIntegrate and NDSolve are correct is to do someindependent analysis, possibly including detailed investigation of the algorithm thatwas used to do the calculation. Such investigations are an important part of thefield of numerical analysis.

6 MAK-ICT

For example, after using Taylor expansion

ex = 1 +x

1!+x2

2!+x3

3!+

cosx = 1 +x2

2!+x4

4!+x6

6!+x8

8!+

sinx = x x3

3!+x5

5! x

7

7!+x9

9!+

You could realize that there are many terms you have truncated off in the expansion,thats why

- Round off errors

Numbers can be stored only up to a fixed finite number of digits: Additional digitsmay be rounded or chopped. Rounding error is sometimes characterized by machine,the largest (positive) number that the machine (computer) cannot distinguish be-tween 1 and 1+ machine.

Roundoff error, or representation error, is the error associated with the fact thatthe computer keeps only a finite number of digits in calculations with inexact num-bers. Since it is not possible (except in special cases) to represent all of the digitsin numbers like 1/3 or pi or

2, computers store only the first few digits in numeri-

cal approximations of these numbers. In typical situations, the computer will storeonly the first 16 decimal digits or the first 53 binary digits. The remaining digitsare discarded. The discarded digits lead to errors in the result. One of the moreconspicuous symptoms of roundoff error is the appearance of tiny non-zero numbersin results that would otherwise be zero.

Although the ability to reduce the effects of roundoff error by raising the precisionof a calculation is certainly very useful, it is far from a universal solution to allproblems with numerical error.

1.8625 to three decimal places, it becomes 1.8631.8625 to two decimal places, it becomes 1.861.8625 to one decimal place, it become 1.9

- Human Errors Such as Computing tools/machines, Mathematical equation/model,propagated error.

Chapter 2

Numerical Integration

There are two main reasons for you to need to do numerical integration: analytical in-tegration may be impossible or infeasible, or you may wish to integrate tabulated datarather than known functions. In this section we outline the main approaches to numericalintegration. Which is preferable depends in part on the results required, and in part onthe function or data to be integrated.

That isThis will be useful when we cannot find an elementary antiderivative for f(x) or if thefunction is defined using data obtained from some experiment.

Numerical integration is the numerical approximation of the integral of a function. For afunction of one variable, it amounts to finding the area under the graph of the function.That is finding I where

I =

ba

f(x) dx

Methods generally replace the integral by a weighted sum of n weights and n functionevaluations, so that

I =ni=1

Wif(xi) dx

For a function of two variables it is equivalent to finding an approximation to the vol-ume under the surface. Numerical integration is often also referred to as quadrature orsometimes cubature for functions of two or more variables. Returning to the one variablecase, numerical integration involves finding the approximation to an integral of a functionf(x) through its evaluation at a set of discrete points. There are two distinct approachesto this. Firstly methods like the trapezium rule or Simpsons rule determine the integralthrough evaluating f(x) at regularly spaced points. These are generally referred to as

7

8 MAK-ICT

Newton-Cotes formulae.

Alternative methods termed Gaussian Quadrature methods have arisen that select irregularly-placed evaluation points, chosen to determine the integral as accurately as possible witha given set of points.

Gaussian Quadrature methods are important as they often lead to very efficient methods.In numerical integration the efficiency of the method relates to the accuracy obtainedwith respect to the number of evaluations of the function f(x). In intensive methods suchas the boundary element method integrations may need to be performed millions of timesso the efficiency of the methods needs to be considered sometimes.

In general, care must be taken to match the numerical integration method to the expectednature of the function f(x). For example it may be known that f(x) is regular. On theother hand f(x) may be singular or oscillatory and will then need special treatment.Often a special method called a product integration method can be developed for theintegration of functions of the form f(x) = w(x)g(x) where w(x) is a pre-set function andthe function and g(x) is known to be a relatively nice function.

There are books devoted to numerical integration. Numerical integration is a basic nu-merical method and the subject is generally covered in books on numerical methods ornumerical analysis (see below). Numerical methods for carrying out numerical integrationcan often be easily programmed and can also be found in general numerical libraries.

2.1 Manual Method

If you were to perform the integration by hand, one approach is to superimpose a gridon a graph of the function to be integrated, and simply count the squares, counting onlythose covered by 50% or more of the function. Provided the grid is sufficiently fine, areasonably accurate estimate may be obtained.

2.2 Trapezoidal/Trapezium rule

2.2.1 Composite Trapezoidal Rule

To derive the rule, we use the following figure

Figure 2.1: Illustration of Composite Trapezoidal rule.


We divide [a, b] into n equal intervals of width h = (ba)n

Let

I =

ba

f(x)dx =

x1a

f(x)dx+

x2x1

f(x)dx+ . . .+

xi+1xi

f(x)dx+ . . .+

xnxn1

f(x)dx.

Applying the trapezium equation , we get,

I =h

2[f0 + 2f1 + 2f2 + . . .+ 2fn1 + fn] + Etrunc,

where

Etrunc = h3

12[f (c1) + f (c2) + . . .+ f (cn)]

|Etrunc| Mh3

12n2

such that the second derivative f is continuous on [a, b] and that |f (x)| < M for all xin [a, b].

Example 2.2.1 Approximate the integral

I =

10

x2dx

using the composite trapezoidal rule with step length h = 0.2

SolutionSince

I =

10

f(x)dx ' h2{f0 + 2f1 + 2f2 + 2f3 + 2f4 + f5}

Partitioning the interval [0, 1], we get,

f0 = f(x0) = f(0) = 02 = 0

10 MAK-ICT

f1 = f(x1) = f(0.2) = (0.2)2 = 0.04

f2 = f(x2) = f(0.4) = (0.4)2 = 0.16

f3 = f(x3) = f(0.6) = (0.6)2 = 0.36

f4 = f(x4) = f(0.8) = (0.8)2 = 0.64

f5 = f(x5) = f(1) = 12 = 1

Therefore

I ' 0.22{0 + 2[0.04 + 0.16 + 0.36 + 0.64] + 1}

= 0.1{0 + 2[1.2] + 1} = 0.340.Absolute error committed is

0.340 0.333 ' 0.00667We have noted that the error obtained is much smaller than that obtained with the puretrapezoidal rule in the previous example. In fact the smaller the error, i.e the better theapproximation.

The truncation error is

|Etrunc| Mh3

12n2=

2(0.2)3

(12)52since f (x) = 2, |f (x)| < 2 on [0, 1]

Example 2.2.2Approximate the integral

I =

22e

x2

2 dx

by the composite trapezoidal rule with h = 1.0. The exact value of the integral I to 4decimal places is 2.3925.SolutionSince h = 1.0, then ,

I =1.0

2(e

(2)22 + 2e

(1)22 + 2e

(0)2

2+ 1

2e(1)2

2 + e(2)2

2 )


0.5(0.13534 + 2 0.60653 + 2 1.00000 + 2 0.60653 + 0.73534)= 2.3484

with error of 0.0441.

Example 2.2.3Estimate

21

1x+1

dx using

(i) Trapezium rule

h

2[f0 + f1] =

1

2[1

2+

1

3] = 0.416

(ii) Composite trapezium rule with h = 0.2

' h2{f(1.0) + 2f(1.2) + 2f(1.4) + 2f(1.6) + 2f(1.8) + f(2.0)}

' 0.22{0.5 + 2(0.45) + 2(0.42) + 2(0.38) + 2(0.36) + 0.33} = 0.4057

Example 2.2.4 Evaluate 3

0(2x + 3) dx by the Trapezium rule with four intervals (5

ordinates).

' h2{f(0) + 2f(0.75) + 2f(1.5) + 2f(2.25) + f(3.0)}

' 0.752{3 + 2(4.5) + 2(6) + 2(7.5) + 9} = 18

Example 2.2.5 Evaluating 3

1sinx dx by the Trapezium rule with 100 points, it gives

the answer as 1.5302437923But can you think of a way of programme this easily??

Example 2.2.6 Evaluating 3

0sinx2 dx by the Trapezium rule

n Sum of areas of trapezoids4 0.433588 0.7040416 0.7572332 0.7695464 0.77256128 0.77331256 0.77350512 0.773551024 0.773562048 0.77356

12 MAK-ICT

0.77356 appears to be a reasonable estimate of our integral.

2.2.2 Text Questions

1. Compute the approximate value of 1

0(x2+1)1dx by using the trapezoidal rule with

ten subintervals. Then compare with the actual value of the integral. Determinethe truncation error bound and compare with the actual error.

2. If the trapezoidal rule is used to compute 5

2sinxdx with h = 0.01, obtain an upper

bound error.

3. How large must n be if the trapezoidal rule is to estimate 2

0exp{x2}dx with an

error not exceeding 106?

4. Consider the integral 1

0sin(pi x

2

2)dx. Suppose that we wish to integrate numerically

with error < 105. What interval width h is needed if the trapezoidal rule is to beused?

5. Approximate 3

11xdx by the trapezoidal rule with an error of utmost 0.1.

2.3 Simpsons rule

We can obtain the Simpsons rule by various ways. One of the most popular ways isfrom Lagranges quadratic interpolation polynomial. The Simpsons rule approximatesthe area under the curve y = f(x) from x0 to x2 by the area under a parabolic curve.The figure below illustrates parabolic fitting on to the curve y = f(x) from x0 to x2.Interpolating f(x) by a Lagrange polynomial of degree 2 i.e. P2(x) then

f(x) = P2(x) + Etrunc(x)

So x2x0

f(x)dx =

x2x0

P2(x)dx+

x2x0

Etrunc(x)dx (2.1)

ResultsSumming up all the three cases, equation (2.1) becomes

x2x0

P2(x)dx =h

3[f0 + 4f1 + f2] + Etrunc(x) Thus

x2x0

P2(x)dx =h

3[f0 + 4f1 + f2] (2.2)


Relation equation (2.2) is the Simpsons rule for approximating the integral. Theintegral for the error in equation (2.1), becomes

x2x0

Etrunc(x)dx =

x2x0

(x x0)(x x1)(x x2)3!

f (4)(c(x))dx

This can be shown (with difficulty!) to be

Mh5

180n4(2.3)

That the fourth derivative f 4 is continuous on [a, b] and that |f 4(x)| < M for all x in[a, b].

Example 2.3.1Use Simpsons rule to approximate the integral I =

10x2dx.

Solution

Since I =

10

f (x )dx ' h3

[f (x0 ) + 4f (x1 ) + f (x2 )]

But x0 = 0 , x1 =1

2, x2 = 1 , h =

(1 0 )(2 )(1 )

=1

2=

(b a)2n

and n = 1.

Therefore

I ' 16

[f(0) + 4f(1

2) + f(1)] =

1

6[02 + 4.(

1

2)2 + 12] =

1

3= 0.33

But the exact value of the integral is 13

= 0.33. It should not surprise you that theSimpson rule has generated the exact value of the integral. In fact the general result isthat for f(x) a polynomial of degree two or less, the Simpson rule will always generatethe exact value of the integral. This will later be stated as a theorem.

2.3.1 Composite Simpsons rule

Lets consider the Figure (8.2),

Figure 2.2: Illustration of the composite Simpsons rule.

14 MAK-ICT

We divide the interval [a, b] into 2n equal intervals of width h = (ba)2n

. Thus the integral,

I =

ba

f(x)dx

becomes,

I =

ba

f(x)dx =

x2a

f(x)dx+

x4x2

f(x)dx+ . . .+

2nx2n2

f(x)dx

=h

3{[f0 + 4f1 + f2] + [f2 + 4f3 + f4]+

[f4 + 4f5 + f6] + . . .+ [f2n2 + 4f2n1 + f2n]}

=h

3[f0 + 4(f1 + f3 + . . .+ f2n1) + 2(f2 + f4 + . . .+ f2n2) + fn] + Etrunc

Where the truncation error,

Etrunc = h5

90[f (4)(c1) + f

(4)(c2) + . . .+ f(4)(cn)]

= h5

90f (4)(c2) n

= (b a)h4

180f (4)(cf )

where a cf b.Example 2.3.2Use the Simpsons rule to compute the integral

I =

22e

x2

2 dx

Using step size h = 1.0. Recall the exact value of I to 4 decimal places is 2.3925.


SolutionUsing the Simpsons rule, we have,

I ' 1.03

(e

(2)22 + 4e

(1)22 + 2e

(0)2

2 + 4e(1)2

2 + e(2)2

2

)= 2.3743

The error committed is 0.0182. We note that is error is much smaller than that obtainedwhen using Trapezoidal rule in the previous lecture though same step size is used.

Example 2.3.3It is required to obtain 2

0

ex2

dx

exact to 4 decimal places. What should h be for Simpsons rule.

SolutionSince the error term is

(b a)180

h4f (4)(cf )

But

f(x) = ex2

therefore

f (x) = 2xex2

f (x) = 2(ex2

+ 2xex2

) = 2ex2

(1 + 2x2) = ex2

(2 + 4x2)

therefore, f (x) = 2xex2

(2 + 4x2) + 8xex2

= ex2

(4x+ 8x3 + 8x) = ex2

(12x+ 8x3)

and f(iv)(x) = 8ex2

(2x4 + 5x2 + 1) < 424e4

So h!2h4

180 424e4 < (0.5)104 = 0.057

Say choose h = 0.05.

16 MAK-ICT


1. Compute an approximate value of

10

(x2 + 1)1

Using Composite Simpsons rule with

(i) h = 0.1,

(ii) h = 0.5.

Compare with the actual value of the integral in each case. Next, determine thetruncation error bound and compare with the actual error.

2. If the Simpsons rule is used to compute

52

sinxdx

with h = 0.75, obtain an upper bound on the error.

3. Establish the composite Simpsons rule over (n 1) even subintervals

ba

f(x)dx =h

3[(f(a) + f(b)) + 4

(n1)2i=1

f(a+ (2i 1)h) + 2(n3)

2i=1

f(a+ 2ih)] +Etrunc

where, h = (ba)(n1) and

Etrunc = (b a)180

h4f (4)(c)

for some c [a, b]4. Consider the integral 1

0

sin(pix2

2)dx.

Suppose that we wish to integrate numerically with error < 105. What intervalwidth h is needed if the Simpsons rule is to be used?


5. Compute 20

(x3 + 1)dx

by using h = 14

and compare with the exact value of the integral.

Example 2.3.4 Using Maple; use Simpsons Rule with n = 100 to approximate theintegral 1

0

1

1 + x2dx

> with(student);> simpson( 1

1+x2, x = 0..1, 100);

1

200+

1

75

(50i=1

1

1 +(

150i 1

100

)2)

+1

150

(49i=1

1

1 + 12500

i2

)

> evalf();.7853981634

> evalf(Pi/4);.7853981635

When trying for the trapezium rule, instead of simpson, you replace it with trape-zoid in step II

2.3.3 Program-FORTRAN (an alternative to MAPLE)

Note that this program is written for clarity rather than speed. The number of functionevaluations actually computed may be approximately halved for the Trapezium rule andreduced by one third for Simpsons rule if the compound formulations are used. Note alsothat this example is included for illustrative purposes only. No knowledge of Fortran orany other programming language is required in this course

REAL*8 FUNCTION TrapeziumRule(x0,x1,nx)C=====parametersINTEGER*4 nxREAL*8 x0,x1C=====functionsREAL*8 fC=====local variablesINTEGER*4 iREAL*8 dx,xa,xb,fa,fb,Sum

18 MAK-ICT

dx = (x1 - x0)/DFLOAT(nx)Sum = 0.0DO i=0,nx-1xa = x0 + DFLOAT(i)*dxxb = x0 + DFLOAT(i+1)*dxfa = f(xa)fb = f(xb)Sum = Sum + fa + fbENDDOSum = Sum * dx / 2.0TrapeziumRule = SumRETURNEND

Assigment to Hand in 2.1 Evaluate

21

x2 cosxdx, f(x) = x2 cosx with h =2 1

6=

1

6. [a, b] = [1, 2]

using (i)Trapezium rule, (ii) Simpsons rule

Example 2.3.5

(a) (i) State one reason to justify numerical techniques of integration. 2 Marks

The analytic integral can be very complicated, and some times impossible. Forexample

30ex

2dx, but also The function f(x) might not be known, but can

be defined on some discrete points.

(ii) One of the commonly used numerical methods of integrations is the Simpsonsrule, state the rules. 1 Marks

IS =h3[f0 + 4(f1 + f3 + . . .+ f2n1) + 2(f2 + f4 + . . .+ f2n2) + fn] + Etrunc

(b) Derive Trapeziums rule for the integration of a function f(x) between a and b,

IT =h

2[f0 + 2f1 + 2f2 + . . .+ 2fn1 + fn] + Etrunc

State its truncation error. 3 Marks

Figure 2.3: Illustration of Trapezoidal rule.


We divide [a, b] into n equal intervals of width h = (ba)n

Let

I =

ba

f(x)dx =

x1a

f(x)dx+

x2x1

f(x)dx+ . . .+

xi+1xi

f(x)dx+ . . .+

xnxn1

f(x)dx.

Applying the trapezium equation , we get,

I =h

2[f0 + 2f1 + 2f2 + . . .+ 2fn1 + fn] + Etrunc,

where

Etrunc = h3

12[f (c1) + f (c2) + . . .+ f (cn)]

|Etrunc| Mh3

12n2

such that the second derivative f is continuous on [a, b] and that |f (x)| < M forall x in [a, b].

(c) Numerically approximate 20

[2 + cos(2

x)]dx

by using the Trapezium rule with

(i) n = 4 = h = 12

4 Marks

I h2

[f0 + 2f1 + 2f2 + . . .+ 2fn1 + fn]

1

4

[(2 + cos(2

0))

+(

2 + cos(2

2))

+ 2[(

2 + cos(2

0.5))

+(

2 + cos(2

1))

+(

2 + cos(2

1.5))]]

20 MAK-ICT

14

[3 + 2 + cos(2

2) + 2

[(2 + cos

2) + (2 + cos 2) + (2 + cos

6)]]

14

[5 + cos(2

2) + 2

[(2 + cos

2) + (2 + cos 2) + (2 + cos

6)]]

=1

4[13.98841913]

3.4971(ii) n = 8 = h = 1

44 Marks

I =h

2[f0 + 2f1 + 2f2 + . . .+ 2fn1 + fn]

=(

2 + cos(2

0))

+(

2 + cos(2

2))

+ 2(

2 + cos(2

0.25))

+ 2(

2 + 2 cos(2

0.5))

+ 2(

2 + cos(2

0.75))

+ 2(

2 + cos(2

1))

+ 2(

2 + cos(2

1.25))

+ 2(

2 + cos(2

1.5))

+ 2(

2 + cos(2

1.75))

3.46928

(d) Considering your results in part (c) above, state any two reasons on how to reducethe errors in numerical integration. 2 Marks

(e) (i) Solve the integral in part (c) above using the Simpsons rule with n = 4. 4Marks

I =h


16

[(2 + cos(2

0))

+(

2 + cos(2

2))

+ 2[(

2 + cos(2

0.5))

+(

2 + cos(2

1.5))]

+ 4(

2 + cos(2

1))]

16

[3 + 2 + cos(2

2) + 2

[(2 + cos

2) + (2 + cos

6)]

+ 4(2 + cos 2)]

16

[5 + cos(2

2) + 2

[(2 + cos

2) + (2 + cos

6) + 4(2 + cos 2)

]] 3.46008250981

(ii) Show that using the Simpsons rule with n = 8, the integral in part (c) aboveis I 3.460002979.

I =h



112

[(2 + cos(2

0))

+(

2 + cos(2

2))

+ 2[(

2 + cos(2

0.5))

+(

2 + cos(2

1))

+(

2 + cos(2

1.5))]

+ 4[(

2 + cos(2

0.25))

+(

2 + cos(2

0.75))

+(

2 + cos(2

1.25))

+(

2 + cos(2

1.75))]]

3.460002979

4 Marks

(f) Which of the two techniques of integration is superior. 1 Marks

It is the Simpsons rule that is more superior.

Chapter 3

Solution to Non-Linear Equations

3.1 Successive Substitution

3.1.1 Background knowledge

Successive substitution is one of the iterative techniques for solving nonlinear equations.Iterative techniques start with an initial value/guess x0 to the root and then using asuitable recurrence relation we generate a sequence of approximations {xk}k=o. If thesequence {x0, x1, . . .} converges, then it does so on the required root. Iterative techniquesare written in the form

xr+1 = g(xr), r = 0, 1, 2, . . . ,

if the next iterate xr+1 depends on the previous one xr.

or xr+1 = gr(xr, xr1), r = 1, 2, . . . ,

if the next iterate depends on the previous two i.e. xr and xr1.

3.1.2 Successive Substitutions

Masenge (1989) called this method the successive substitutions. Sometimes it is calledGeneral Iteration or fixed point method. In the method, we seek the roots of,

f(x) = 0. (3.1)

We try to split f(x) in the form,

f(x) = x g(x) (3.2)However, this splitting may not be unique. But not all the different splittings may beuseful to us. We can determine the type of splitting which is useful to a numerical analyst.Now, instead of solving equation (3.1) we now solve x = g(x). The scheme for solvingthis problem is given by the algorithm;

xn+1 = g(xn), n = 0, 1, 2, . . .

22


Thus we start with a suitable value x0, and generate the sequence of approximations

x1 = g(x0)

x2 = g(x1)

x3 = g(x2)

x4 = g(x3)

xn+1 = g(xn)

That is, the sequence is {x1, x2, . . . , xn, . . .}

Example 3.1.1Find the real root of the equation

x2 2x 3 = 0

in the interval [2, 4].

SolutionSplitting

f(x) = x2 2x 3 = 0in the form

f(x) = x g(x) = 0.We get the schemes,

x = g(x) =3

x 2

i.e x =3

x 2

Giving the iterative scheme.

xr+1 =3

xr 2

24 MAK-ICT

Taking the initial approximation x0 = 4, we get the iterates

x1 =3

x0 2 = 1.5

x2 =3

x1 2 = 6

x3 =3

x2 2 = 0.375

x4 =3

x3 2 = 1.263158

x5 =3

x4 2 = 0.919355

x6 =3

x5 2 = 1.02762

x7 =3

x6 2 = 0.990876

According to the behavior of the iterates, there is no hope for convergence in theinterval [2, 4]. Hence such a rearrangement is no good. Splitting f(x) = 0 in the form,

x = g2(x) =x2 3

2

giving the iterative scheme,

xr+1 =x2r 3

2

therefore with x0 = 4 we get,

x1 =x20 3

2= 6.5

x2 =x21 3

2= 19.625


x3 =x22 3

2= 191.070

This show that the iterates are obviously diverging. Hence such a rearrangement isno good. Splitting f(x) = 0 in the form,

x = g3(x) =

(2x+ 3)

Giving the iteration formula,xr+1 =

(2xr + 3)

Thus with x0 = 4 we get,

x1 =

(2x0 + 3) = 3.31662

x2 =

(2x1 + 3) = 3.10375

x3 =

(2x2 + 3) = 3.03439

x4 =

(2x3 + 3) = 3.01144

In fact this is an arrangement which is giving a sequence of iterates which are convergingto the root . The sequence is converging to the root x = 3.

Note 3.1.1One would actually wonder as to whether you have to keep trying the splitting until youget one which converges to the root. We can test and know the splitting which givesa convergence sequence of approximations before starting to compute the iterates. Thiswonderful criterion is called the convergence criterion for the iterative scheme of the form

xr+1 = g(xr).

However, before stating the criterion, lets first formerly state what is meant by an iterativescheme

xr+1 = g(xr).

being convergent.

Definition 3.1.1An iterative scheme/process. xr+1 = g(xr) is convergent if,

limr|xr+1 xr| = 0,

otherwise we say that the scheme is divergent.

26 MAK-ICT

3.1.3 Convergence criterion

Let the function g(x) be continuous in a small interval [a, b] containing a simple (single)root of the function f(x). Let also g(x) be differentiable in the open interval (a, b). Ifthere exists a real number L such that 0 |g(x)| L 1, a g(x) b x (a, b),then for an arbitrary starting value x0 taken from (a, b) the iteration formula xr+1 = f(xr)will converge. The rate of convergence of the iteration will depend on the smallness ofthe constant L relative to unity.

Example 3.1.2 x3 sinx = 0, the root is = 0.929i)x = (sinx)

13 = g(x) g(x) = 1

3(sinx)

23 cosx

g() = 0.23 so |g()| < 1 and iteration is convergent .ii) x = sinx

x2= g(x) g(x) = x2 cosx2x sinx

x4

g() = 1.3 so |g()| > 1 and iteration is divergent.Is there a way to choose g(x) so that the iteration will always work?+ Is there a way to improve the rate of convergence?

Note 3.1.2Sometimes this theorem is known as the fixed point theorem, i.e. a fixed point is a numberx? such that x? = g(x?). Thus a root of f(x) = 0 is a fixed point of the scheme x = g(x).

Theorem 3.1.1 Fixed point theoremIf g(x) and g(x) are continuous in I = (x , x + ), where x is a fixed point, andif |g(x)| k < 1, x I, xr+1 = g(xr) converges to x (attractive fixed point). If|g(x)| > 1 x I, the process does not converge to x? (repulsive).Proof

[x? = g(x?)] [xr+1 = g(xr)]x xr+1 = g(x?) g(xr)

= g(cn)(x xn), cn (x?xn)(mean value theorem) Writing, x? xn+1 = en+1 etc

|en+1| = |g(c0)||e0| k|en|If x0 I,

|e0| = |g(c0)||e0| k|e0||e2| = k|e1| k2|e0|

= |en+1| kn+1|e0|So as n, kn+1 0 and |en+1| 0 i.e xn+1 x?.Example 3.1.3

1. Verify for the following example that x = 3 is a solution of x = g(x).


(a) g(x) = 18x(x2+9)

(b) g(x) = x3 24(c) g(x) = 2x

3

(3x29)

(d) g(x) = 81(x2+18)

Starting with x0 = 3.1, calculate the first few iteration and justify theoretically theapparent behavior.

2. Consider the fixed iteration xk = (xk1) for g(x) = 2(x 1) 12 for x 1. Showthat only one fixed point exists (at x = 2) and that g(2) = 1. Compute iterationsstarting from

(a) x0 = 1.5 and

(b) x0 = 2.5 and show them on a plot of x and g(x).

3. By splitting f(x) = x3 x 1 = 0 in the form f(x) = x g(x) = 0 for finding theroot [1, 2],

(i) get three different splitting and with their corresponding iterative formulae.

(ii) Test using the convergence criterion which of the splitting lead to a convergentsequence.

(iii) For the scheme leading to a convergent sequence, start with a suitable initialapproximation and find the root correct to 3 decimal places.

3.2 Secant Method

The secant method needs two points near the root before the algorithm can be applied.Thus it is of the form

xr+1 = f(xr, xr1).

3.2.1 Derivation of the secant method

We linear approximate the graph of y = f(x) in figure (10.1), by a chord passing throughthe points A and B. The equation of this chord is,

28 MAK-ICT

y f(x1) = f(x0) f(x1)(x x1)x0 x1 .

The chord cuts the x - axis at x2 thus

f(x1) = f(x0) f(x1)(x2 x1)x0 x1

Giving,

x2 = x1 f(x1)(x1 x0)f(x1) f(x0)

or in general,xn+1 = g(xn, xn1)

xn+1 = xn f(xn)(xn xn1)f(xn) f(xn1) (3.3)

Note 3.2.1 Realise we have been able to generate Equation 3.3 by comparing two tan-gents at x0 and x1. But if we compare the two tangents at x0 and x2 we generate

xn+1 = xn1 f(xn1)(xn1 xn)f(xn1) f(xn) (3.4)

The error in the (n + 1)th iterate is related to the error in the nth iterate en by therelation en+1 ' Aekn where k ' 1.618 . . . and A is a constant. This relation suggests thatthe method has order of convergence 1.618.


Example 3.2.1Use the secant method to find the root near 2 of the equation x3 2x 5 = 0. Start theiteration with x0 = 1.9 and x1 = 2.0.

Solution Recall

xn+1 = xn1 f(xn1)(xn1 xn)f(xn1) f(xn)

Since f(x) = x2 2x 5. We find

x0 = 1.9, f0 = 1.941x1 = 2.0, f1 = 1.000

x2 = x0 f(x0)[x0 x1]f(x0) f(x1)

= 1.9 0.1(1.941)0.941

= 2.1063 f2 = 0.1320

x3 = 2.0 0.1063(1.000)1.1320

= 2.0939 f3 = 0.0073

x4 = 2.1063 (0.0124)(0.1320)0.1393 = 2.09455 f4 = 0.0002

finally we have

x4 = 2.0939 0.00065(0.0073)0.00728

= 2.09455

Thus since x4 and x5 are identical to 5 decimal places, so x5 = 2.09455 is the value ofthe root correct to five decimal places.

3.2.2 Advantages and Disadvantages of the secant method

The method;

(i) can work for double roots.

(ii) has order of convergence of 1.618.

(iii) is not always convergent.

30 MAK-ICT

The above are advantages or disadvantages depending on the comparison in question.

Example 3.2.2Find the real root of f(x) = x3 + x2 3x 3 = 0 using secant method.

SolutionFirst we must find an interval where the real root x? lies. Now since, f(1) = 1+133 =4 < 0 and f(2) = 8 + 463 = 3 > 0 and f(x) is continuous for all real x, there existsx? (1, 2) such that f(x?) = 0 (the intermediate value theorem).

f (x) = 3x2 + 2x 3 = 0

when x = 0.7208 or 1.3874. Since these values of x do not belong to (1, 2) we concludethat x is the only real root in (1, 2) (Application of Rolles theorem). Thus since, |f(1)| >|f(2)|, we let x0 = 1 and x1 = 2. To get,

x2 = 1 (4)(1)7 = 1 +4

7= 1.571432f(1.571429) = 1.364432,

So |f(1.571429)| < |f(2)|, hence let x0 = 2, x1 = 1.571429 to get,

x2 = 2 (3)2 1.5714293 + 1.364432

= 1.705411 f(1.705411) = 0.2477445

So that |f(1.705411)| < |f(1.541429)| So we let, x0 = 1.571429, x1 = 1.705411 to get,

x2 = 1.571429 (1.364432) 1.571429 1.7054111.364432 + 0.247745 = 1.735136

and the next iterate is 1.732051. Thus continuing this process gives, the fourth and fifthapproximations to x? equal to 1.731996 and 1.732051 respectively. Therefore 1.732051 isa root correct to 2 decimal places.


1. Show that there is a root of the equation

f(x) = 3x sinx ex = 0

in the interval (0, 1). Estimate this root to 2 decimal places using the secant method.

2. Find the roots of the following equations, using the methods of secant.


(i) ex = cosx

(ii) x3 2x+ 1 = 0(iii) sin 2x ex 1 = 0(iv) ln(x 1) = x2

3. Use the secant method to find the real root of the equation

x3 + 2x2 x+ 5 = 0.

4. Consider obtaining the root of;

f(x) =ex + 1 sinx

(x 2)

Show that f(1.9) < 0, f(2.1) > 0 and use the secant method to obtain the root.

3.3 The Regular False method

The regular false algorithm uses a similar geometric approach like the secant method(the regular false method is an associate of the secant method) with the exception that,f(xr)f(xr1) < 0 at each stage of the algorithm. Thus to start the method, you need twopoints x0 and x1 near the root such that f(x0)f(x1) < 0.

3.3.1 Geometric representation and derivation of the regularfalse algorithm.

Figure 3.1: Geometrical representation of the regular false method.

From Figure above, we note that the produce f(x0)(f(x1) < 0, which is in conformitywith the regular false. This was not necessarily the case for the secant method. Theequation of the Chord CD is,

y = f(x1) =f(x0 f(x1))(x x1)

x0 x1

This Chord cuts the x-axis at x2 i.e.

f(x1) = f(x0) f(x1)(x0 x1) (x2 x1)

32 MAK-ICT

giving, x2 = x1 f(x1)(x1 x0)f(x1) f(x0)

or in general we have that, xn+1 = g(xn, xn1)

That is, xn+1 = xn f(xn)(xn xn1)f(xn) f(xn1) . (3.5)

Equation (3.5) is the popular Regular false/falsi position method with condition that ateach stage of the algorithm, f(xr)f(xr1) < 1.

Example 3.3.1Find the real root of f(x) = x3 2x 2 = 0 using Regular-falsi method.

SolutionFirst we must find an interval where the real root x? lies. Now since, f(1) = 3 < 0 andf(2) == 2 > 0 and f(x) is continuous for all real x, there exists x? (1, 2) such thatf(x?) = 0 (the intermediate value theorem). We let x0 = 1 and x1 = 2. To get,

x2 = 2 (2)(1)6 = 2.333333,

since f(x2) = f(2.33333) = 6.0370 > 0

Thus x3 = x2 f(x2)(x2x0)f(x2)f(x0)

x3 = 2.3333 6.0370(2.3333 1)9.0370

= 1.4426

since f(x3) = f(1.4426) = 1.88296 < 0

Thus x4 = x3 f(x3)(x3x2)f(x3)f(x2)

x4 = 1.4426 1.8829(0.8904)7.91996 = 1.6542914 = f(x4) = 0.781316 < 0

x5 = x4 f(x4)(x4 x2)f(x4) f(x2) = 1.73206 = f(x5) = 0.26785 < 0


x6 = x5 f(x5)(x5 x2)f(x5) f(x2) = 1.757605 = f(x6) = 0.085654 < 0

x7 = x6 f(x6)(x6 x2)f(x6) f(x2) = 1.76565 = f(x7) = 0.02682 < 0

x8 = x7 f(x7)(x7 x2)f(x7) f(x2) = 1.76816 = f(x8) = 0.008356 < 0

x9 = x8 f(x8)(x8 x2)f(x8) f(x2) = 1.76894 = f(x9) = 0.002595 < 0

x10 = x9 f(x9)(x9 x2)f(x9) f(x2) = 1.76918

We could continue our iterations, though the regular-falsi takes long to converge. Butclearly the exact value is 1.7693

Note 3.3.1 In every next iteration, the previous point must be part of it.

3.3.2 Order of convergence of the Regular algorithm

The error in the (n + 1)th iterate (denoted en+1) is related to the error in the nth iterate

en by the equation,en+1 ' Aekn

where k = 1. This suggests that although the regular false uses the same formula as thesecant method, the order of convergence of the regular false is one compared to ' 1.618for the secant. Thus, the method is slower at converging to the root compared to thesecant method. However, with the condition f(xr)f(xr1) < 0 at each stage, ensures thatthe regular false is always convergent which is not the case with secant method.

3.3.3 Advantages and disadvantages of the regular false algo-rithm

(i) The regular false algorithm is always convergent.

34 MAK-ICT

(ii) The order of convergence of the method is one.The two basic points on the advantages and disadvantages. Whether it is an advan-tage or a disadvantage it all depends on the comparison in question. For instancein comparison with the secant method, it is disadvantageous that the regular falsehas order of convergence one. While it is advantageous that it is always convergent.


1. Use the regular false algorithm to approximate the root in the interval (1, 2) of theequation x3 2x 2 = 0. Start is x0 = 1 and x1 = 2.

2. Use the regular false algorithm to find the root of

f(x) = x2 4x+ 2 = 0

that lies in the interval (0, 1) and state your answer correct to three decimal places.

3. Verify that x = 3 is a solution of x = g(x) where

g(x) =18x

(x2 + 9).

Use the regular false to approximate this root.

4. Consider the equation

f(x) =ex + 1 + sinx

(x 2) = 0

whose root you would want to find. Show that f(1.9) < 0, f(2.1) > 0 and use theregular false algorithm to compute this root.

5. Approximate to three decimal places the roots of the following equations using theregular false algorithm.

(i) x3 = 2

(ii) x2 = 3

(iii) x4 = 2

(iv) x5 = 3

6. (a) Derive the regular false algorithm by clearly giving its geometrical illustration.

(b) What advantages and disadvantages does the secant method enjoy over othermethods so far considered for solving nonlinear equations.


3.4 Bisection Method

3.4.1 Background

The bisection method is one of the bracketing methods for finding roots of equations.

3.4.2 Implementation

Given a function f(x) and an interval which might contain a root, perform a predeterminednumber of iterations using the bisection method.

3.4.3 Limitations

Investigate the result of applying the bisection method over an interval where there is adiscontinuity. Apply the bisection method for a function using an interval where thereare distinct roots. Apply the bisection method over a large interval.

3.4.4 Explanation on the bisection method

The bisection method takes a similar geometrical approach with the regular false algo-rithm. You need two initial guesses x0 and x1 to the root x

? of the nonlinear equationf(x) = 0, such that f(x0)f(x1) < 0. The next approximation is obtained by getting thearithmetic mean of the previous two. However, the pair xr, xr+1 to be used to get xr+2must satisfy the condition f(xr)f(xr+1) < 0. Masenge (1989) called this method a trivialsimplification of the regular false.

In mathematics, the bisection method is a root-finding algorithm which works by re-peatedly dividing an interval in half and then selecting the subinterval in which the rootexists.

Example 3.4.1 Use the Bisection method to find the root of x2 = 3

x2 3 = 0 = f(x) = x2 3. Since f(1) < 0, f(2) > 0, the root lies in (1, 2) iex0 = 1, x1 = 2

x2 =x1 + x0

2=

1 + 2

2= 1.5 = f(x2) = 0.75 < 0

x3 =x2 + x1

2= 1.75 = f(x3) > 0

x4 =x3 + x2

2= 1.625 = f(x4) < 0

36 MAK-ICT

x5 =x4 + x3

2= 1.6875 = f(x5) < 0

x6 =x5 + x3

2= 1.71875

etc.... process still slow but convergent.

Example 3.4.2Use the bisection algorithm to approximate the root in the interval (1, 2) of the equation

x3 2x 2 = 0

SolutionLet x0 = 1 and x1 = 2 therefore

f(x0) = 1 2 2 = 3 < 0f(x1) = 8 4 = 2 = 2 > 0 therefore

= (3)(2) < 0.

Hence root exists in (1, 2). Therefore,

c =a+ b

2=

1 + 2

2= 1.5

and f(x2) = f(1.5) = 1.53 2(1.5) 2. and the process goes on until convergence to the

solution. Complete this as an exercise.

3.4.5 Advantages of the bisection method

The method is simple.

The method is always convergent.

3.4.6 Disadvantages of the bisection method

It requires the values of a and b.

The convergence of interval halving is very slow (slow at converging to the root x*)

The method fails in case of approximating a double root or a root of even multiplicity.



1. Approximate to 2 decimal places the roots of the following equations using thebisection method.

(i) x2 = 3

(ii) x3 = 2

(iii) x4 = 2

2. The function h(x) = x sinx occurs in the study of damped forced oscillation. Findthe value of x that lies in the interval [0, 2] where the function takes on the valueh(x) = 1. Use interval bisection.

3. If a = 0.1 and b = 1.0, how many steps of the bisection method are needed todetermine a root in this interval with an error of at most 1

2 108?

4. Consider obtaining the root of:

f(x) =ex + 1 + sinx

(x 2) .

Show that f(1.9) < 0, f(2.1) > 0 and use the bisection method to obtain the root.

5. Find the real root of the equation

x3 x2 x+ 1 = 0using the bisection algorithm.

6. The bisection method generates intervals [a0, b0], [a1, b1], and so on, which of theseinequalities are true for the root r that is being calculated?

(a) |r an| 2|r bn|(b) |r an| 2n1(b0 a0)(c) |r bn| 2n1(b0 a0)(d) 0 r an2n(b0 a0)(e) |r 1

2(an + bn)| 2n2(b0 a0)

Example 3.4.3 Find all the real solutions to the cubic equation x3 + 4x2 10 = 0 inthe interval [1, 2].

Example 3.4.4 Use Newtons method to find the roots of the cubic polynomial x3 3x+ 2 = 0 in the interval

(i) [0, 2]

(ii) [3,1]

38 MAK-ICT

3.5 Newton Raphsons method

3.5.1 Derivation of the Newtons method

If we are given a non-linear equation f(x) = 0 and we are to apply the Newton Raphsonsmethod, we linear approximate the graph of y = f(x) by a straight line passing throughthe point (x0, f0) and tangential to the graph of y = f(x). Take the slope of this line tobe p. Geometrically this is given in figure below.

Figure 3.2: Geometrical representation of a tangent to a curve at a point.

The equation of the line with slope p and passing through the point (x0, f0) is

y f0x x0 = p (3.6)

However, we know that p is the slope of the tangent to y = f(x) at (x0, f0). This is givenby;

p = f (x0) = f 0 (3.7)

Substituting equation (3.7) in equation (3.6) we get

y f0x x0 = f

0

y f0 = (x x0)f 0 (3.8)From figure 13.1, line of equation 13.3 cuts the x-axis at the point (x1, 0) i.e when x = x1and y = 0. Substituting in equation (3.8) we get;

0 f0 = (x1 x0)f 0Making x1 the subject, we get,

x1 = x0 f0f 0

or

x1 = x0 f(x0)f x0

(3.9)


equation (3.9) is actually the Newtons method for obtaining the next iterate x1 from theprevious iterate x0. The equation (3.9) is generalized and written;

xn+1 = xn f(xn)f (xn)

, (3.10)

since the linear approximation of the curve is done at each of the iterates xn, xn+1, xn+1, . . .as reflected in figure above.

Example 3.5.1Use Newton Raphsons method to find the root of

x2 3 = 0 on [1, 2].

f(xn) = x2n 3

therefore f (xn) = 2xn

But Raphsons formula is xn+1 = xn f(xn)f (xn)

Substituting in the Raphsons formula we get

xn+1 = xn (x2n 3)2xn

=2x2n x2n + 3

2xn

=(x2n + 3

2xn

)Taking the initial guess/approximation as x0 = 2, but you could also consider x0 = 1 youcome up with the same answer.

x1 = x20 + 3

2x0=

22 + 3

2(2)= 1.75

x2 =x21 + 3

2x1=

(1.75)2 + 3

2(1.75)= 1.7321

40 MAK-ICT

x3 =x22 + 3

2x2=

(1.7321)2 + 3

2(1.7321)= 1.7320508

x4 =x23 + 3

2x3=

(1.7320508)2 + 3

2(1.7320508)= 1.7320508

Thus the root is 1.7320508

Example 3.5.2Use Newton Raphsons method to find the root of

f(x) = x3 + 3x 1On [0, 1].

Solution

Clearly f(0) = 03 + 3(0) 1 = 1f(1) = 13 + 3(1) 1 = 3

Therefore f(0)f(1) < 1 implying a real root exists in the interval [0, 1]. But

f(xn) = x3n + 3xn 1

therefore f (xn) = 3x2n + 3 = 3(x2n + 1).

But Raphsons formula is xn+1 = xn f(xn)f (xn)

Substituting in the Raphsons formula we get

xn+1 = xn (x3n + 3xn 1)

3x2n + 3

=3x3n x3n 3xn + 1

3x2n + 3

=1

3

(2x3n 3xn+1x2n + 1

)Taking the initial guess/approximation as x0 = 0

x1 = 13

(2x30 3x0 + 1

x20 + 1) =

1

3

(1

1

)= 0.333


x2 =1

3

(2(1

3)3 3(1

3) + 1

(13)2 + 1

)= 0.0222

x3 =1

3

(2(0.0222)3 3(0.0222) + 1

(0.0222)2 + 1

)= 0.3110

x4 =1

3

(2(0.3110)3 3(0.3110) + 1

(0.3110)2 + 1

)= 0.0386

x5 =1

3

(2(0.0386)3 3(0.0386) + 1

(0.0386)2 + 1

)= 0.2943

Example 3.5.3Find the root of

f(x) = 3x+ sinx ex = 0in the interval (0, 1). Using Newton Raphsons method.

SolutionSince f(0) = 1 < 0 and f(1) = 3 + sin(1) e > 0, so there is a real root in (0, 1). Using


= xn (3xn + sinxn exn)

(3 + cos xn exn)

=3xn + xn cosxn xnexn 3xn sinxn + exn

3 + cos xn exn

=xn(cosxn exn) sinxn + exn

3 + cos xn exn

With x0 = 0 (initial guess), therefore

x1 =0(cos 0 e0) sin 0 + eo

3 + cos 0 eo =1

3= 0.3333

x2 = . . . . . . . . .

x3 = . . . . . . . . .

x4 = . . . . . . . . .

Compute x2, x3 and x4 as an exercise.

42 MAK-ICT


1. Verify that when Newtons method is used to computex (by solving the equation

x2 = R), the sequence of iterates is defined by,

xn+1 =(xn +

Rxn

)

2.

Hence find

3 correct to six decimal places.

2. Find the root of the equation

x2 + x 1 = 0in the interval [0, 1], giving your answer correct to 4 decimal places.

3. Show that the cubic equation

2x3 + 3x2 3x 5 = 0

has a real root in the interval [1, 2]. Approximate this root correct to five decimalplaces using Newton Raphsons method.

4. Use Newtons method to approximate the root of the equation

g(x) = x3 2 sinx

on [0.5, 2].

5. The convergence rate m of an iterative process is given by

en+1 = Aemn .

Obtain experimentally the convergence rate for Newtons method applied to findinga root of

(i) tanhx x2 = 0 starting at x = 1(ii) x3 x2 x+ 1 = 0 starting at x = 1.5. (use computer program)

3.5.3 General Newtons algorithm for extracting roots of posi-tive numbers.

Suppose that our interest is to find the rth root of a real positive number A. If x is thevalue of this root, then x is related to A by the equation, xr = A. or xr A = 0


Let f(x) = xr A, then x is the root of the nonlinear equation,

f(x) = xr A = 0.

Applying the Newton Raphsons method, we have;


But f(xn) = xrn A

f (xn) = rxr1n

therefore xn+1 = xn (xrn A)rxr1n

=rxrn xrn + A

rxr1n

=(r 1)xrn + A

rxr1n

=1

r{(r 1)xn + A

xr1n} (3.11)

Equation (3.11) was also given by Masenge (1987) in his book on Fundamentals of Numer-ical methods. Equation (3.11) is a general formula from which we can obtain quadraticallyconvergent iterative processes for finding approximations to arbitrary roots of numbers.

Note 3.5.1 The root, or answer interested in is the x. And also what is f(x) is thefunction f(x) = 0

Example 3.5.4When r = 2 we have x2 = A x = A. Thus for r = 2 in the general formula, we get,

xn+1 =1

2(xn +

A

xn)

Which is the Newtons square root algorithm for extracting roots of positive numbers.

Example 3.5.5Use Newtons square root algorithm to find the square root of 5 correct to six decimalplaces.

44 MAK-ICT

Solution

Substituting in the general formula r = 2 and A = 5 we get,

xn+1 =1

2(xn +

A

xn)

Starting with x0 = 2 we get,

x1 =1

2(x0 +

5

x0) = 2.25

x2 =1

2(x1 +

5

x1) = 2.236111111

x3 =1

2(x2 +

5

x2) = 2.236067978

x4 =1

2(x3 +

5

x3) = 2.2360798

Since x0 and x1 agree, in one decimal place, then x1 = 2.25 is the value of the rootcorrect only to one decimal place. Likewise x2 = 2.236111111 is also correct only to onedecimal place since it agrees with the previous iterate x1 = 2.25 only in one decimal place.However, x3 = 2.236067978 is correct to three decimal places since it is in agreement withthe previous iterate x2 in exactly three places of decimal. But x4 = 2.236067978 is exactlythe same as x3. In fact they are exactly the same up to nine decimal places. This meansthat x4 = 236067978 is the value of the root correct to nine decimal places. Thus x4, mustalso be correct up to six decimal places. Hence, the value of the root that you state asbeing correct to six decimal places or nine decimal places is x4 = 2.236067978. Comparewith the value obtained from calculator.

3.5.4 Using Newtons general formula for roots in finding recip-rocals of numbers.

If we have that r = 1 then x1 = A (positive number), this means x = 1A

(reciprocal ofA) with r = 1 in the general formula in equation (3.11) then we get,

xn+1 = xn(2 Axn).


This formula is quadratically convergent and can suitably be applied to calculate thereciprocal of numbers.

Example 3.5.6Use Newtons reciprocal algorithm to find the reciprocal of 3.

SolutionUsing the algorithm,

xn+1 = xn(2 Axn)Since A = 3, let x0 = 0.5. Therefore,

x1 = x0[2 3(x0)] = 0.5[2 3(0.5)] = 0.25x2 = x1[2 3(x1)] = 0.3125x3 = x2[2 3(x2)] = 0.33203125x4 = x3[2 3(x3)] = 0.3333282447x5 = x4[2 3(x4)] = 0.333333333x6 = x5[2 3(x5)] = 0.333333333

Thus x6 = 0.333333333 is the value of the reciprocal of 3 i.e13

correct to nine decimalplaces. But we know that 1

3= 0.33.

3.5.5 Some limitations of the Newton Raphsons method

Good though it is, the method has some limitations.

(a) If in the immediate neighborhood of a root of f(x), f (x) vanishes or is very small, theNewton Raphsons method will not converge. Masenge (1987) explained the reason

for this failure as; since f (x) is very small, the quantity, f(xn)f (xn) becomes very large.

The consequent is that we are thrown away from the root we are approximating.

(b) The Newton Raphsons method may also fail if f(x) has a point of inflection in theneighborhood of the root.


1. Use Newtons square root algorithm to find the square root of 2 correct to 6 decimalplaces.

2. Use cube root Newtons algorithm to find the cube root of 7 correct to four decimalplaces.

46 MAK-ICT

3. Use Newtons reciprocal algorithm to find

(i) the reciprocal of the square root of 2.

(ii) The reciprocal of the cube root of 4.

4. State the advantages and disadvantages of the Newtons method for nonlinear equa-tions as compared to the other methods considered in the previous lectures.

Assigment to Hand in 3.1 Compute the roots (zeros) of f(x) = ex 4 2x usingsuccessive substitution methodwith (i)x0 = 2, (ii) x0 = 0Example 3.5.7

(a) (i) State the convergence criterion of successive substitution technique of solvinga non-linear equation f(x) = 0. 2 Marks

Let the function g(x) be continuous in a small interval [a, b] containing a simple(single) root of the function f(x). Let also g(x) be differentiable in the openinterval (a, b). If there exists a real number L such that 0 |g(x)| L 1, a g(x) b x (a, b), then for an arbitrary starting value x0taken from (a, b) the iteration formula xr+1 = f(xr) will converge. The rateof convergence of the iteration will depend on the smallness of the constant Lrelative to unity.

(ii) Given the function f(X) = x3 sinx = 0, [0, 1], using successive substitutiontechnique, we can generate

xn+1 = (sinx)13 & xn+1 =

sinx

x2

Which of the two methods converge? Why? 2 Marks

Since for g(x) = (sinx)13 , g(x) = 1

3(sin)

23 cosx = 0 < g(x) < 1, thus

the converging formula.

(iii) Use the converging formula in (ii) above, to approximate the root off(x) = x3 sinx = 0 in [0, 1] to 3 decimal places.[Hint: Let x0 = 1 and use radians.] 5 Marks

n (sinx)13

1 1.02 0.9443 0.9324 0.9295 0.929


(b) (i) Define the Newton-Raphson method formula for finding the root of a non-linearequation f(x) = 0 2 Marks


,

(ii) The convergence of the Newton-Raphson method technique highly depends onthe initial guess. Discuss. 2 Marks

Yes, when the initial guess is in the interval given, the iterations converge fasterthan otherwise.

(iv) Use Newton Raphson method to estimate one of the solutions of x2 4 = 0using x0 = 6 to 2 decimal places. 5 Marks

xn xn+1x0 = 6 x1 = 3.33x1 = 3.33 x2 = 2.27x2 = 2.27 x3 = 2.01x3 = 2.01 x4 = 2.00x4 = 2.00 x5 = 2.00

(iii) Newtons Raphsons method is one of the popular schemes for solving a non-linear equation f(x) = 0. Prove that the Newton Raphsons method for findingthe square root of a positive number A is given by,

xn+1 =1

2

(xn +

A

xn

)

Use the scheme above to approximate the square root of 5(

5) to three decimalplaces with x0 = 2. 5 Marks

xn xn+1x0 = 2 x1 = 2.25x1 = 2.25 x2 = 2.236x2 = 2.236 x3 = 2.2.236x3 = 2.236 x4 = 2.236

(d) With any simple example, write short notes on the Bisection method for solvingnon-linear equations. 2 Marks

Chapter 4

Interpolation

Linear interpolation is often used to approximate a value of some function f using twoknown values of that function at other points.

4.1 Review- Linear interpolation

Using the similarities of triangles,

f(b) f(a)b a =

f(c) f(a)c a

f(b) f(a)b a (c a) = f(c) f(a)

thus

f(c) = f(a) +f(b) f(a)

b a (c a) (4.1)

and

c = a +f(c) f(a)f(b) f(a)(b a) (4.2)

Example 4.1.1 Given the data below

Time 0 1 2 3 4

Distance 0 6 39 67 100

48


(i) Find the distance traveled when t = 2.3 hrs.Then a = 2, b = 3,& c = 2.3f(a) = 39, f(b) = 67, & f(c)?

f(c) = f(a) +f(b) f(a)

b a (c a) = 39 +(67 39)(3 2) (2.3 2) = 47.4

(ii) The time taken when the distance traveled is 80 miles.Then a = 3, b = 4,& c =?f(a) = 67, f(b) = 100, & f(c) = 80

c = a +f(c) f(a)f(b) f(a)(b a) = 3 +

(80 67)(100 67)(4 1) = 3.39394

Example 4.1.2 The bus stages along Kampala-Jinja are 10 km apart. An express bustraveling between the two towns only stops at these stages except in case of an emergencywhen its permitted to stop at a point between the two stages.The fares (fee) between the first, second, third and fourth stages from Jinja are Sh 110, Sh 150, Sh 185and Sh 200 respectively. On a certain day, a passenger paid to travel from Jinja in thebus upto the fourth stage, but he fell sick and had to be left on a health center 33 kmfrom Jinja.

Given that he was refunded money for the distance he had not traveled, find theapproximate amount of money he received.

Distance (x) 10 20 30 40

Amount Paid 110 150 185 200

Then a = 30, b = 40,& c = 33f(a) = 185, f(b) = 200, & f(c)?

f(c) = f(a) +f(b) f(a)

b a (c a) = 185 +(200 185)(40 30) (33 30) = 189.5

The journey he had moved costed him Sh 189.5, thus he was refunded200 189.5 = 10.5 shillings

50 MAK-ICT

Another person who had only Sh.165 was allowed to board a bus but would be leftat a point worth his money, how far from Jinja would he be left.Then a = 20, b = 30,& c =?f(a) = 150, f(b) = 185, & f(c) = 165

c = a +f(c) f(a)f(b) f(a)(b a) = 20 +

(165 150)(185 150)(30 20) = 24.286 km

Example 4.1.3 The table below shows the values of cosx.

80 x 0 10 20 30 40 50

cos 80 x 0.1736 0.1708 0.1679 0.1650 0.1622 .01593

(i) Find the value of cos 8035.Then 30 = 2, b = 40,& c = 35

f(a) = 0.1650, f(b) = .01622, & f(c)?

f(c) = f(a) +f(b) f(a)

b a (c a) = 0.1650 +(0.0028)

105 = .01636

(ii) Find the cos1 0.1655.Then a = 20, b = 30,& c =?f(a) = 0.1679, f(b) = 0.1650, & f(c) = 0.1655

c = a+f(c) f(a)f(b) f(a)(ba) = 20

+(0.1655 0.1679)(0.1650 0.1679)(30

20) = 20+8.276 = 28.276

cos1 0.1655 = 8028.3

Example 4.1.4 Use Linear interpolation to find the root of the equation x3x 1 = 0which lies between (1, 2)

x 1 2

f(x) 1 5


Recall that a value x is a root or a solution to an equation f(x) if x satisfy it, i.e iff(x) = 0. The question is to find the value of x at f(x) = 0.Then a = 1, b = 2,& c =?f(a) = 1, f(b) = 5, & f(c) = 0

c = a +f(c) f(a)f(b) f(a)(b a) = 1 +

(01)(51)(2 1) = 1.16667

Example 4.1.5 Use linear interpolation to estimate the root of the equationx3 2x 5 = 0 which lies in the interval (2, 3). 2.0588Example 4.1.6 Use linear interpolation to estimate the root of the equation x2 2 = 0which lies in the interval (2,1). 1.3333Example 4.1.7 The following data gives the distance covered by a particle for a certainperiod of time.

Time (s) 0 1 2

Distance(m) 0 5 7

Estimate the time taken by a particle to cover a distance of 6 m. 1.0833 s

4.2 Application

Linear interpolation is often used to fill the gaps in a table. Suppose you have a tablelisting the population of some country in 1970, 1980, 1990 and 2000, and that you wantto estimate the population in 1994. Linear interpolation gives you an easy way to do this.

The basic operation of linear interpolation between two values is so commonly used incomputer graphics that it is sometimes called a lerp in the jargon of computer graphics.The term can be used as a verb or noun for the operation. e.g. Bresenhams algorithmlerps incrementally between the two endpoints of the line.

Lerp operations are built into the hardware of all modern computer graphics processors.They are often used as building blocks for more complex operations: for example, abilinear interpolation can be accomplished in three lerps. Because this operation is cheap,its also a good way to implement accurate lookup tables with quick lookup for smoothfunctions without having too many table entries.

52 MAK-ICT

4.3 History

Linear interpolation has been used since antiquity for filling the gaps in tables, oftenwith astronomical data. It is believed that it was used in the Seleucid Empire (lastthree centuries BC) and by the Greek astronomer and mathematician Hipparchus (secondcentury BC). A description of linear interpolation can be found in the Almagest (secondcentury AD) of Ptolemy.

4.4 Extensions

In demanding situations, linear interpolation is often not accurate enough (since not allpoints can be approximated to be on a straight line). In that case, it can be replaced bypolynomial interpolation or spline interpolation.

Linear interpolation can also be extended to bilinear interpolation for interpolating func-tions of two variables. Bilinear interpolation is often used as a crude anti-aliasing filter.Similarly, trilinear interpolation is used to interpolate functions of three variables. Otherextensions of linear interpolation can be applied to other kinds of mesh such as triangularand tetrahedral meshes.

4.5 Lagrange interpolation

We may know the value of a function f at a set of points x1, x2, . . . , xN . How do we esti-mate the value of the function at any point x; how do we compute f(x). In general this isdone by developing a smooth curve though the points (x1, f(x1)), (x2, f(x2)), ...(xN , f(xN)).

Lagrange interpolation is a way to pass a polynomial of degree N 1 through Npoints.

Lagrange polynomials are the interpolating polynomials that equal zero in all givenpoints, save one. Say, given points x1, x2, . . . , xN , Lagranges polynomial number k is theproduct

Pk(x) =(x x1)(xk x1)

(x x2)(xk x2)

(x xk1)(xk xk1)

(x xk+1)(xk xk+1)

(x xN)(xk xN)

such that Pk(xk) = 1 and Pk(xj) = 0, for j different from k. In terms of Lagrangespolynomials the polynomial interpolation through the points (x1, y1), (x2, y2), . . . , (xN , yN)could be defined simply as

P (x) = y1P1(x) + y2P2(x) + ...+ yNPN(x)

Definition 4.5.1 Polynomial interpolation is a method of constructing a function andestimating values at non-tabular points between x0 and xn.


4.5.1 Alternatively

Let f(x) be a continuous function on [a, b], such that

f(xk) = fk, k = 0, 1, ..., n,

with xk [a, b]. We call the set of points {xk} tabular points (or interpolating points),while the set of values {fk} are called the tabular values of f(x). We seek for a polynomialof degree n, such that

Pn(xk) = fk, for k = 0 , 1 , 2 , . . . n. (4.3)

Such a polynomial is called Lagranges interpolation polynomial. However, the process ofcalculating f(x), x [a, b], x 6= xk k from 0 to n, is called interpolation.We now proceed to derive a formula for Pn(x). However, there is need to introduce somekey definitions.

Definition 4.5.2Let {Dk : k = 0, 1, ..., n} be any set of numbers. We define the products;

nk=0

Dk = D0.D1...Dn

andn

k=0 ,k 6=jDk = D0 .D1 ...Dj1 .Dj+1 ...Dn .

This definition is important in the following defining equation for Lagranges interpolatingpolynomial.

Theorem 4.5.1The Lagranges interpolation polynomial Pn(x) is given by,

Pn(x) =nk=0

Lk(x)f(xk) (4.4)

where Lk(x ) =n

j=0 ,j 6=k

x xjxk xj

and fk f (xk)

54 MAK-ICT

ProofBecause Pn(x) is of degree n, we may write

Lk(x) = k

nj=0,j 6=k

(x xj),

k constant.For Pn(x) to satisfy equation (4.3) we must have,

Lk(xj) = kj( Kroneker delta)

with

kj =

1, j = k

0, j 6= k(4.5)

With the condition (1.3) we have,

k =1

nj=0,j 6=k

(xk xj)

This completes the proof.

4.5.2 Examples of interpolating polynomials

(i) Linear interpolating polynomialsWhen n = 1, in equation (4.4) we have the polynomial as

P1(x) =1

k=0

Lk(x)f(xk)

= L0(x)f(x0) + L1(x)f(x1)

But Lk(x) =n

j=0 j 6=k

(x xj)(xk xj)

therefore L0(x) =

(x x1x0 x1

)

L1(x) =

(x x0x1 x0

)


Thus,

P1(x) =

(x x1x0 x1

)+

(x x0x1 x0

)f(x1) (4.6)

Which is known as Lagranges interpolating polynomial of degree one, popularlyknown as Linear interpolating polynomial.

Example 4.5.1Construct a linear interpolation polynomial for the data,

x 0 1f(x) 1.0000 2.7183

Hence interpolate f(0.5)

SolutionLet x0 = 0 and x1 = 1therefore f(0) = 1.0000 and f(1) = 2.7183Substituting in equation (4.6) for linear interpolation we have,

P1(x) = (1 x)(1.0000) + (x)(2.7183)= 1.0000 + 1.7183x

and hence, P1(0.5) = 1.0000 + (1.7183)(0.5)

= 1.0000 + 0.8592 = 1.8592

In fact the data in this particular example describes the graph of f(x) = ex on [0, 1].The geometrical interpretation of the linear interpolation on [0, 1] .

(ii) Quadrative interpolating polynomialsWhen n = 2 in equation (4.4) we get

P2(x) =2

k=0

Lk(x)fk

= L0(x)f(x0) + L1(x)f(x1) + L2(x)f(x2)

with Lk(x) =n

j=0,6=k

(x xj)(xk xj)

56 MAK-ICT

then L0(x) =(x x1)(x x2)

(x0 x1)(x0 x2)

L1(x) =(x x0)(x x2)

(x1 x0)(x1 x2)

L2(x) =(x x0)(x x1)

(x2 x0)(x2 x1)

Thus, we have,

P2(x) =(x x1)(x x2)

(x0 x1)(x0 x2)f(x0)

+(x x0)(x x2)

(x1 x0)(x1 x2)f(x1) +(x x0)(x x1)

(x2 x0)(x2 x1)f(x2)

Which is called a quadratic interpolating polynomial. Generally they are betterinterpolating polynomials than the linear ones.

Example 4.5.2You are given that, f(0) = 2, f(2) = 4, and f(3) = 10. Find a Lagrange polyno-mial of degree 2 that fits the data.

SolutionSince x0 = 0, x1 = 2 and x2 = 3 therefore f(x0) = 2, f(x1) = 4, f(x2) = 10.But

P2(x) = L0f(x0) + L1f(x1) + L2f(x2)

L0 =(x x1).(x x2)(x0 x1)(x0 x2) =

(x 2)(x 3)(2)(3)

=1

6x2 5

6x+ 1

L1 =(x x0).(x x2)(x1 x0)(x1 x2) =

(x 0)(x 3)(2 0)(2 3)

= 12x2 +

3

2x

L2 =(x x0).(x x1)(x2 x0)(x2 x1) =

(x 0)(x 2)(3 0)(3 2)


=1

3x2 2

3x

P2(x) = f(x) = L0fx0 + L1fx1 + L2fx2

f(x) = (2)(

1

6x2 5

6x+ 1

)+ (4)

(1

2x2 +

3

2x

)+ (10)

(1

3x2 2

3x

)= x2 + x 2

Thus P2(x) can be used to interpolate f(x) at any of the non tabular points.

(iii) The Cubic interpolating polynomial

When n = 3 in equation (4.4) we get,

P3(x) =3

k=0

Lk(x)fk

= L0(x)f(x0) + L1(x)f(x1) + L2(x)f(x2) + L3(x)f(x3)

and with, Lk(x) =n

j=0,j 6=k

(x xj)(xk xj)

then, L0(x) =(x x1)(x x2)(x x3)

(x0 x1)(x0 x2)(x0 x3)

L1(x) =(x x0)(x x2)(x x3)

(x1 x0)(x1 x2)(x1 x3)

L2(x) =(x x0)(x x1)(x x3)

(x2 x0)(x2 x1)(x2 x3)

L3(x) =(x x0)(x x1)(x x2)

(x3 x0)(x3 x1)(x3 x2)

Thus,

P3 =(x x1)(x x2)(x x3)

(x0 x1)(x0 x2)(x0 x3)f(x0) +(x x0)(x x2)(x x3)

(x1 x0)(x1 x2)(x1 x3)f(x1)

58 MAK-ICT

+(x x0)(x x1)(x x3)

(x2 x0)(x2 x1)(x2 x3)f(x2) +(x x0)(x x1)(x x2)

(x3 x0)(x3 x1)(x3 x2)f(x3)

Construction of the cubic polynomials from available data will be tested in thetext questions at the end of this lecture. Note that the higher degree Lagrangepolynomials can be constructed with ease.

Theorem 4.5.2Lagranges interpolation polynomial Pn(x) is unique.

Note 4.5.1The uniqueness means that you cannot find any other polynomial of the same degreewhich can interpolate the data.ProofThe proof proceeds by contradiction. Let Pn(x) and Qn(x) be two different poly-nomials which interpolate f(x) over the set of points {xk : k = 0, 1, ..., n} whichbelong to the interval [a, b], then

Pn(xk) = Q(xk) = fk, for k = 0 , 1 , 2 , . . . , n.

Lets define,r(x) = P (x)Q(x) x [a, b],

then r(x) has at most degree n. Since r(xk) = 0, for k = 0, 1, 2, ..., n, it has n + 1distinct zeros in [a, b]. This contradicts the fundamental law of algebra which statesthat a non-zero polynomial of degree n cannot have more than n zeros and so Pn(x)and Qn(x) are the same polynomials.


1. Construct a linear interpolating polynomial P1(x) for the function f(x) =1x

on theinterval [1, 2]. Use your polynomial to interpolate f(x) at x = 1.2 .

2. Given that x0 = 0, x1 =12

and x2 = 1 for f(x) = ex. Construct a Lagrange

polynomial that agrees with f(x) at the interpolating points.

3. Find a third degree Lagrange polynomial that goes through the points (0, 0), (1, 1), (8, 2)and (27, 3).Use the polynomial to find q for (20, q). Also construct a linear interpolating poly-nomial using only (8, 2) and (27, 3), then use the linear polynomial to estimate q.Compare the estimated qs and comment on your results given that the data is ofthe function y = x

13 .

4. Find the interpolating polynomials going through

(i) (0, 1) and (2, 3)


(ii) (1, a), (0, b) and (1, c)(iii) (0, 1), (1, 0), (2, 1) and (3, 0)

5. Given the table below, use Lagranges interpolation polynomials of degree one, twoand three to find f(2.5)

x 2.0 2.2 2.4 2.6f(x) 0.5102 0.5208 0.5104 0.4813

Solution

(i) For n = 1, since need to predict f(2.5) then x0 = 2.4, x2 = 2.6

P1(x) = (x x1x0 x1 )f(x0) + (

x x0x1 x0 )f(x1)

= (x 2.6

2.4 2.6)0.5104 + (x 2.4

2.6 2.4)0.4813

= 0.49585

so for x = 2.5, f(2.5) = 0.49585

(ii) For n = 2, x0 = 2.2, x1 = 2.4, x2 = 2.6

=(x x0)(x x2)

(x1 x0)(x1 x2)f(x1) +(x x0)(x x1)

(x2 x0)(x2 x1)f(x2)

= 0.53814

6. Repeat questions in linear-interpolation (review-section) using the Lagrange linearinterpolating polynomial. comment on the accuracy of the two techniques.

4.5.4 Error analysis in Lagranges interpolation

Truncation errors in Lagrange interpolation

We assume that fk f(xk), k = 0, 1, 2, . . . , n are exact and we consider the truncationerror e(x) = f(x) Pn(x) for x [a, b] and Pn(x) the Lagrange polynomial of degree nas defined in Lecture 1. Apart from the fact that e(xk) = 0, for k = 0, 1, 2, . . . , n withxk [a, b], we can say nothing more about e(x) for any x 6= xk. In addition we have thatf(x) has at least n+ 1 continuous derivatives on [a, b], then it is possible to express e(x)in terms of fn+1(x). We now state without proof two necessary Lemmas.

60 MAK-ICT

Lemma 4.5.1Given

qn+1(x) =nk=0

(x xk), x [a, b]

of degree (n+ 1),qn+1(x) is of degree n such that

qn+1(xj) =n

k=0,k 6=j(xj xk)

Lemma 4.5.2Let a function g(x) be defined on [a, b]. Let {Sk : k = 0, 1, . . . , n} be a set of distinctpoints each belonging to [a, b], with S0 < S1 < S2 < . . . < Sn. Suppose that:

(i) g(n)(x)(n integer) is continuous on [S0, Sn].

(ii) g(Sk) = 0, k = 0, 1, . . . , n.

Then there is at least a number (S0, Sn) such that , g(n)() = 0

Lemma 4.5.3The expression in the truncation error e(x) is given by:

e(x) =nk=0

(x xk)f(n+1)()

(n+ 1)!

with = (x) and mink(x, xk) < < maxk(x, xk)


Proof

For points x = xk, k = 0, 1, 2, . . . , n, the theorem is trivially satisfied.Suppose x 6= xk, we define

q(t) =nk=0

(t xk)

and

g(t) = f(t) Pn(t) q(t)q(x)

(f(x) Pn(x)) for t , x [a, b].

Now g(xk) = 0, k = 0, 1, 2, . . . n Also g(x) = 0 (x 6= xk). So for each fixed x 6= xk, g(t)has n+ 2 distinct zeros.Since g(t) satisfies all the conditions of Lemma 1.2, we deduce that there is a number such that g(n+1)() = 0 where

mink

(x, xk) < < maxk

(x, xk).

Now

g(n+1)(t) = f (n+1)(t) P (n+1)n (t)(f(x) Pn)q(n+1)(t)

q(x)

= f (n+1)(t) (n+ 1)!(f(x) Pn(x))q(x)

SinceP n+1(t) = 0

andqn+1(t) = (n+ 1)!

The result of the theorem follows when t = .

Example 4.5.3Using two point linear interpolation of ex on [0, 1] therefore

ex =x(x 1)

2e, (0, 1).

But x(x 1) is maximum or minimum at x = 12

with maximum absolute value equal to1. So e(x) has a maximum value equal to e

8= 0.3398 (4dp)

62 MAK-ICT

4.5.5 Rounding errors in Lagrange polynomials

Errors known as rounding errors are usually introduced in the functional evaluation off(xk). Suppose rounding errors Ek occur in the data fk = f(xk), k = 0, 1, 2, . . . , nrespectively.Let P (x) and P ?(x) denote, respectively, the interpolating polynomials using exact andinexact data.Thus,

Pn(x) =nk=0

(fk + Ek)Lk(x)

P ?n(x) =nk=0

fkLk(x)

so |Pn(x ) P?n(x )| nk=0

|Ek||Lk(x)|

Which is the rounding error bound. If data was rounded to m decimal places, then theabsolute maximum error is |Ek| 12 10m for each k = 0, 1, . . . , n.

Example 4.5.4Find the rounding error bound in linear interpolation, of ex for x [0, 1], when data wererounded to four digits.

Solution

|P1(x) P1 (x)| 12

104(|L0(x)|+ |L1(x)|)

Now, L0(x) = 1 xand L1(x) = x

and so on [0, 1], |L0(x)|+ |L1(x)| = 1 x+ x = 1.

Thus, |P1 (x ) P?1 (x )| 1

2104,

which says that the effect of rounding errors in the data, on P1(x) maintains the samemaximum magnitude.

Example 4.5.5


The table below gives the tabulated values of the probability integral,

I =

2

pi

x0

et22 dt

Use linear interpolation to find the value of I when x = 1.125. Estimate also the errorbound on the truncation error over [1, 1.25]

x 1.00 1.05 1.10 1.15 1.20 1.25I 0.683 0.705 0.729 0.750 0.770 0.789

Solution

Note 4.5.2At the tabular points x0, x1, ..., xn, the truncation error is zero, hence we can write

f(x) = Pn(x) +Rni=0

(x xi) (R constant depending on x)

Let,

F (x) = f(t) Pn(t) {f(x) Pn(x)}

ni=0

(t xi)ni=0

(x xi)()

then

F (xi) = f(xi) Pn(xi) = 0, i = 0, 1, 2, . . . nFurther

F (x) = f(x) Pn(x) f(x) + Pn(x) = 0, x (a, b), x 6= xi,

necessarily and i = 0, 1, 2, . . . n.Thus, F (t) vanishes in (a, b) at n + 2 distinct points. On applying Rolles theorem re-peatedly we conclude that there exists c (a, b) such that, F (n+1)(c) = 0.Differentiating equation () (n+ 1) times and putting t = c, gives,

f(x) = Pn(x) +ni=0

(x xi)f(n+1)(c)

(n+ 1)!

64 MAK-ICT

For linear interpolation, n = 1, so,

P1(x) =x x1x0 x1f(x0) +

x x0x1 x0f(x1)

withx0 = 1.10, x1 = 1.15 f(x0) = .729, f(x1) = 0.750

thus ,

P1(1.125) =(1.125 1.15)(0.729)

0.05 +(1.125 1.10)(0.75)

0.05

= 0.5(0.729 + 0.750) = 0.7395

Truncation error1i=0

(x xi)f (2)(c)2! , Now (x x0)(x x1) is minimum at x = x0+x12 and is

equal to (x1x0)2

4in magnitude.

f (x) =

2

piex22 , f (x) =

2

pixe

x2

2

f (x) =

2

piex22 +

2

pix2e

x22

and is 0 if x = 1. Thus ,

|f (x)|

2

pie

A bound on the truncation error is given by,

(0.05)2

8pi

2

e' 0.00009



1. Compute a bound on the truncation error for ex on [1, 1.4] when fourth degreepolynomial is used to interpolate ex.

2. Obtain error bounds for both linear and quadratic interpolation for sinhx over theinterval [1.90, 2.10]

3. A table of values for x4x12

is constructed for 0 x 1 in such a way that the errorin linear interpolation does not exceed if rounding errors were neglected. Showthat for uniform spacing h, then h does not exceed the value 2

2.

4. Find the rounding error bound when quadratic interpolation, of ex for x [0, 1],when data were rounded to four digits.

Note 4.5.3 Use Lagrange interpolation to find an appropriate function passing throughthe given points. Sketch a graph of this function based only on the given points and whatyou think the curve must be. Compare your sketch with the graph created by graphingtechnology.

(a) A linear function passing through the points (1, 3) and (2, 1).(b) A quadratic func

Documents

Numerical Analysis