Methods - MacHighwayfrink.machighway.com/~dynamicm/Numerical-Methods.pdf · In all such cases, numerical methods have to be resorted to. The mathematics or science student is therefore

Prepared by Dr. Ralph W.P. MASENGE

African Virtual universityUniversité Virtuelle AfricaineUniversidade Virtual Africana

Numerical Methods

African Virtual University �

Notice

This document is published under the conditions of the Creative Commons http://en.wikipedia.org/wiki/Creative_Commons Attribution http://creativecommons.org/licenses/by/2.5/ License (abbreviated “cc-by”), Version 2.5.


I. NumericalMethods_________________________________________ 3

II. PrerequisiteCourseorKnowledge_____________________________ 3

III. Time____________________________________________________ 3

IV. Materials_________________________________________________ 3

V. ModuleRationale __________________________________________ 3

VI. Content__________________________________________________ 4

6.1 Overview___________________________________________ 4 6.2 Outline_____________________________________________ 5 6.3 GraphicOrganizer_____________________________________ 6

VII. GeneralObjective(s)________________________________________ 7

VIII. SpecificLearningActivities___________________________________ 8

IX. TeachingandLearningActivities_______________________________ 9

X. LearningActivities_________________________________________ 16

XI. CompiledListofallKeyConcepts(Glossary)___________________ 107

XII. CompiledListofCompulsoryReadings_______________________ 113

XIII. CompiledListofMultimediaResources_______________________ 114

XIV. SynthesisoftheModule___________________________________ 115

XV. SummativeEvaluation_____________________________________ 116

XVI.References_____________________________________________ 130

XVII.MainAuthoroftheModule_________________________________ 131

Table of ConTenTs


I. numerical Methodsby Dr. Ralph W.P Masenge

II. Prerequisite Courses or KnowledgeCalculus 2 is prerequisite.

III. TimeThe total time for this module is 120 study hours.

IV. MaterialStudents should have access to the core readings specified later. Also, they will need a computer to gain full access to the core readings. Additionally, students should be able to install the computer software wxMaxima and use it to practice algebraic concepts.

V. Module RationaleA key attribute of mathematics is its applicability in problem solving. The history of the subject is full of evidence that the driving force in its early development was based in trying to solve problems in plane geometry, celestial mechanics and in navigation.

Unfortunately, mathematical formulations (models) of most problems in science and engineering are, in general, difficult to solve analytically either because of the com-plex nature of the analytical solutions or because such solutions cannot be expressed in terms of combinations of known mathematical functions.

In all such cases, numerical methods have to be resorted to. The mathematics or science student is therefore expected to have a working knowledge of and ability to apply numerical methods in solving some basic mathematical problems such as interpolation, numerical integration and finding roots of functions.

Figure 3: Surfing on the Internet is a key to e‑learning


VI. Content

6.1 Overview

First Learning Activity: Types and Causes of Errors

The first learning activity aims at making the learner appreciate the need for numerical methods. It is also felt that this is the right moment to define the concept of a mathe-matical error, point out the sources and types of errors and mention some practical ways of reducing their cumulative effect on the numerical solution.

Second Learning Activity: Interpolation

The second learning activity deals with the concept of interpolation. Both linear and higher order polynomial interpolation methods, based on Lagrange, Newton’s divided differences and finite difference interpolation techniques are presented.

Third Learning Activity: Numerical Integration

The third learning activity looks at the problem of numerical integration. The discus-sion is limited to Newton‑Cotes formulae. Specific attention is given to the Trapezoidal and Simpson’s rules and the application of Richardson’s extrapolation technique on both the Trapezoidal and Simpson’s rules in derivingRomberg integration schemes.

Fourth Learning Activity: Roots of functions

The fourth and final learning activity presents the root finding problem associated

with solving the nonlinear equation f (x) = 0 and solving the coupled system of two

nonlinear equations f (x, y) = 0 , g(x, y) = 0 .

Figure 4: A Baobab Tree. Number of roots equals number of twigs


6.2 Outline: Syllabus (include specific timings here if needed).

This is a two unit course offered at Level 2 with priority rating A. Mathematics Module 3 is its prerequisite course.

The following is a detailed outline of the contents of each Learning Activity. The timings shown are suitable for students following a unit level course allowing 35 hours per unit. The timings show the length of time the Learner is recommended to spend learning each of its components.

[Students following a module level course should allow 120 hours for the complete module].

Preperation, taking and review for Pre-Assessment (4 hrs)

Learning Activity No.1: Types and Causes of Errors (25 hrs)Types and sources of errorsNeed for numerical methodsError sources and typesStrategies for reducing errors

Learning Activity No.2: Interpolation (35 hrs)

Linear interpolationLagrangian interpolationNewton’s divided differencesFinite difference operatorsFinite difference tablesFinite difference interpolation polynomials

Learning Activity No.3: Numerical integration (25 hrs)Newton Cote’s formulaeDerivation of the trapezoidal and Simpson’s rulesRomberg integrationGaussian quadrature formulae

Learning Activity No. 4: Roots of functions. (25 hrs)Bisection methodConvergence of the bisection methodThe Regula Falsi or False Position methodSecant methodNewton-Raphson’s methodSolving a coupled system of two nonlinear equationsFixed point iterations


Preperation and taking Summative Assessment (6 hrs)

6.3 Graphic Organiser

Figure 5: Graphic Organizer


VII. General objective(s)

At the end of this module:

You will be equipped with knowledge and understanding of the properties of elementary functions and their various applications necessary to confidently teach these subjects at the secondary school level.

You will have secure knowledge of the related contents of school mathematics to enable you confidently teach these subjects at the secondary school level.

You will acquire knowledge of and the ability to apply available ICT to improve the teaching and learning of school mathematics.

Specifically you will be able to:

(1) Distinguish between numerical and analytical methods/solutions(2) Appreciate the need for learning and applying numerical methods(3) Identify the main sources of errors and take measures to eliminate or reduce

such errors(4) Derive and apply a number of interpolation methods(5) Derive and apply a number of numerical integration methods(6) Derive and apply a number of numerical methods for finding roots of func-

tion(7) Solve a coupled system of two nonlinear equations in two variables.

Figure 6: Internet Dish at the AVU-Learning Center, University of Dar Es Salaam


VIII. specific learning objectives As already mentioned in the Module Outline given in Section VI, this is a two units module and its contents will be presented using four learning activities. Each learning activity has a set of specific learning objectives. We are stating them here in advance with a view to enable the learner have a global overview of what lies ahead in terms of what s/he will be able to do on completion of the module.

At the end of the module, the learner will be able to:

S/N Lear-ning Activity

Specific Learning Objectives

1 Types and sour-ces of errors

• List the chief sources of computational errors and the practical steps to be taken to eliminate or reduce their cumulative effect on the numerical solution

• Appreciate the difference between the size (accuracy) and the seriousness (precision) of an error.

• Understand and apply linear interpolation on a given table of function values

• Estimate the error in linear interpolation for a known smooth function

2 Interpo-lation

• Explain why numerical methods are essential in solving mathe-matical problems

• Write down and apply the Lagrange interpolation polynomial for equally spaced data

• Write down and apply Newton’s interpolation based on divided differences

• Define and manipulate the shift, forward, backward, central and mean difference operators

• Construct tables of differences for a given function or given data• Derive and apply Newton’s forward, Newton’s backward, and

Stirling’s central difference interpolation polynomials.

3 Numeri-cal Inte-gration

• Derive, understand and apply the Trapezoidal rule, Simpson’s rule and any other Newton-Cotes type numerical integration for-mula

• Derive, understand and apply Romberg’s numerical integration scheme based on either the Trapezoidal or Simpson’s rule.

4 Roots of functions

• Derive and apply the bisection method• Prove convergence of the bisection method• Derive, understand and apply both the Secant and the Regula

Falsi methods• Derive, understand and apply the Newton Raphson method• Derive, understand and apply the Newton’s method on a pair of

nonlinear simultaneous equations.


IX. Teaching and learning activities

9.1 Pre-assessment

1. If x is an exact measurement of a certain quantity and x0 is its approximation, the concept of error in the approximation is defined by

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x0 − xx − x0

x − x0

x − x0

2. The absolute error in approximating the exact quantity x = 104 by the approxi-

mate value x0 = 107 is given by

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

437

104

The relative error in the approximation x0 given in Question 1 is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

0.0288460.2994620.0028850.038462

3. One valid reason why the function

f (x) =1 if x > 0

0 if x = 0

⎧

⎨⎪

⎩⎪

African Virtual University �0

is not continuous at x = 0 is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

limx→ 0

limx→ 0

limx→ 0

limx→ 0

f (x) = f (0)f (x) = 0f (x) = 1

f (x) ≠ f (0)

4. The continuous function x3 − 3x − 3 must have a zero (root) at some point in the

interval 2 < x < 3 because

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

f (2) f (3) = 0f (2) f (3) < 0

f (2) > 0f (2) = 0

f (3) < 0f (3) = 0

5. The first derivative of the function y = xx is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

1+ ln(x)( )xx

xx

xx ln(x)(1− ln(x))xx

6. The truncated cubic Maclaurin series expansion of the function f (x) = 1+ x is

(a) :1− x2+

x2

4−

3x3

8

(b) :1+ x2−

x2

4+

3x3

8

(c) :1− x2+

x2

8−

x3

16

(d) :1+ x2−

x2

8+

x3

16

African Virtual University ��

7. The general anti-derivative of the function f (x) = ln(x) is given by

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x ln(x) + Cln(x) − x + C

x ln(x) − x + Cx ln(x) + x + C

8. The value of the integral 1+ x

x2 + 2x + 531

2

∫ dx is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

1.1465811.1456811.1461851.164581

9. Working throughout with six decimal places accuracy, the approximate

value of 1+ x

x2 + 2x + 531

2

∫ dx using the Trapezoidal rule with h = 0.25 is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

1.1468501.4165601.1460381.146580

10. Working under the same conditions, the approximate value of the integral given in Question 8 using Simpson’s rule is found to be

(a) :(b) :(c) :(d) :

⎧

⎨⎪⎪

⎩⎪⎪

1.4160391.1460831.1460381.146580


11. If the 3rd and 13th terms of an arithmetic sequence of numbers are 7 and 27 , respectively, then the 52nd element of the sequence is

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

103105107106

12. Starting with x0 = 2 , the value of x4 using the iteration formula xn+1 =3(1+ xn )

xn

is

(a) :(b) :(c) :(d) :

⎧

⎨⎪⎪

⎩⎪⎪

2.1042552.1037312.1307312.103713

13. Given two points P (x0 , y0 ) andQ(x1 , y1 ) , the x-coordinate of the point where the

secant line AB cuts the x - axis is

(a) x1y0 − x0 y1

y1 − y0

(b) y1 − y0

x1y0 − x0 y1

(c) x0 y1 − x1y0

y1 − y0

(d) y1 − y0

x0 y1 − x1y0

⎧

⎨

⎪⎪⎪⎪⎪

⎩

⎪⎪⎪⎪⎪


14. The curve of a function y = f (x) passes through two points whose coordinates

are (0.2,1.183) and (0.4,1.342) . By using the straight line which joins the two points to approximate the curve of the function in the given interval, the approximate value of y at x = 0.3 is found to be

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

y = 1.2265y = 1.6225y = 1.5262y = 1.2625

15. Given a point P (x0 , f (x0 )) on the curve of a differentiable function y = f (x) , the x coordinate of the point where the tangent at P cuts the x - axis is

(a) : x0 −f (x0 )f ' (x0 )

(b) : x0 +f (x0 )f ' (x0 )

(c) : x0 −f ' (x0 )f (x0 )

(d) : x0 +f ' (x0 )f (x0 )

16. The Bisection method is based on the principle that a continuous function which is positive at one point x = a and negative at another point x = b has a root

(zero) at some point x = c in the interval a < c < b and that the point x =a + b

2, which bisects the interval[a,b] , is a reasonable approximation of c . Starting

with the two points a = 2, b = 3 , application of the bisection method on

the function x3 − 3x − 3 gives the value

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x3 =

x3 =

x3 =

x3 =

2.2152.1152.1252.225

after the third bisection process.


17. The Newton Raphson method for approximating a root of a function f (x) uses the iteration formula

xn+1 = xn −

f (xn )f ' (xn ) n = 0,1,2,...

If x0 = 2 is an approximation of one of the roots of the function f (x) = x3 − 3x − 3 , then application of the Newton-Raphson method on the function gives the value of

x2 as:

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x2 =

x2 =

x2 =

x2 =

2.1308362.1033862.3018362.103836

18. The coordinates of the point where the curves representing the functions

x2 + y2 = 4 ; x2 − y2 = 1 intersect each other and which lies in the first qua-drant are

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x = 1.224745,x = 1.581139,x = 1.511839,x = 1,242745,

y = 1.561139y = 1.224745y = 1.227445y = 1.561139

19. Starting with x0 = 1.5 , y0 = 1.2 , the values x3 and y3 obtained using the pair

formulae xn+1 = 4 − yn2 , yn+1 = xn

2 −1 n = 0,1,2,... are

(a)(b)(c)(d)

⎧

⎨⎪⎪

⎩⎪⎪

x3 = 1.562050,x3 = 1.322875,x3 = 1.526050,x3 = 1.233875,

y3 = 1.322875y3 = 1.562050y3 = 1.322875y3 = 1.562050


Solutions

1. (c) 11. (b)

2. (b) 12. (b)

3. (d) 13. (c)

4. (b) 14. (d)

5. (a) 15. (a)

6. (d) 16. (c)

7. (d) 17. (d)

8. (c) 18. (b)

9. (c) 19. (c)

10. (d)

Pedagogical Comment For Learners

The learner’s confidence to embark on this module will greatly be enhanced by solving these preassessment questions.

Questions 1,2 are on errors and 3 is on continuity. If you have problems in solving them you are advised to look up the definitions given in the Module Glosary.

Questions 4,5,6 and 7 are on the concepts of limits, continuity, differentiation and the anti-derivative. If you fail to solve any of them then work through the relevant Sections of the pre-requisite Module (Mathematics Module 3). The same tip holds if you experience difficulties in solving Questions 8,9,10 and 11 on integration.

Questions 12,13,16,17,18 and 19 relate to the concept of iterations and simula-tneous equations. Here, simply study carefully the given formulae and apply them faithfully.

Questions 15, 16 and 17 are on linear approximation of a function. In case one faces some problems in solving any of these, one should work through the relevant section in your Higher Level Mathmatics books.

Each correct solution has 5 marks, giving a maximum score of 90 marks. A score in the range 41 – 60 is average. A below average score (0 – 40) probably implies that you need a thorough go at the relevant prerequisite material before starting the mo-dule. An average score may mean you can start the module but with frequent cross reference to some prerequisite materials. With an above average score (61 – 90) the learner should confidently embark on the module in the knowledge that he/she has the required background knowledge to start and successfully complete it.


X. learning activities

LEARNING ACTIVITY # 1

Types and Sources of Errors

Summary

In this learning activity we discuss three important introductory topics. We begin by making a distinction between analytic and numerical solutions. This is followed by discussion of some typical mathematical problems specifically selected to convince the learner why numerical methods are needed. We conclude the activity by defining the concept of errors in computational mathematics, pointing out their main causes, types and presenting practical ways of eliminating or reducing their effect on nume-rical solutions.

Specific Learning objectives

At the end of this learning activity the learner will be able to:

• Define the concept of errors in computational mathematics• Distinguish between absolute and relative errors and appropriately relate

them to the concepts of accuracy and precision• List the chief sources of computational errors and the practical steps to be taken

to eliminate or reduce their cumulative effect on the numerical solution• Appreciate the difference between the size (accuracy) and the seriousness

(precision) of an error.• Understand and apply linear interpolation on a given table of function va-

lues• Estimate the error in linear interpolation for a known smooth function

List of required readings

Wikipedia: Numerical Methods/Errors Introduction

List of relevant useful links

Wolfram MathWorld (visited 03.04.07) http://mathworld.wolfram.com Students should search for the entry covering the unit title. Also, search for

any key words in the text. Mathworld gives a detailed reference in all cases.


Wikipedia (visited 03.04.07)

http://en.wikipedia.org/wiki As with mathworld, students should search for the entry covering the unit title.

Also, search for any key words in the text. Wikipedia generally gives shorter and less complete entries. However, they can be easier to read.

MacTutor History of Mathematics (visited 03.03.07)

http://www-history.mcs.standrews.ac.uk/Indexes The MacTutor Archive is the most comprehensive history of mathematics on

the internet. Students should search for their unit title and read the history of the subject. This gives a helpful overview of the importance and context of the topic being studied.

Key Words

[Full defintions are given in the text]

Error in an approximation: The difference between the exact and the approximate value.

Absolute error: The error without consideration of the sign (positive or negative).

Relative error: The ratio between the absolute error and the exact quantity.

Initial, discretization, truncation and rounding errors: Different types of errors caused by different sources of error.


Learning Activity 1: Types and Sources of Errors

Introduction

This first learning activity of the module is intended to provide the learner with answers to the following important questions; questions raised by many students taking a numerical methods course for the first time.

(1) What is a numerical solution and how does such a solution differ from an exact (true) or analytical solution?

(2) Why should one learn numerical methods? Are numerical methods needed?

(3) What are errors in the context of mathematics? What are the main sources of computational errors? How can one eliminate errors or reduce their effect on numerical solutions?

We begin the activity by explaining the difference between an analytic and a numerical solution. In order to establish the need for numerical methods we discuss a number of mathematical problems specifically selected with a view to convince the learner that there is a real need for learning numerical methods either because analytical solutions cannot be found or they are too complex to be of any practical use.

To answer the questions on errors we define the concept of an error and list a number of sources of errors. In the sequel we also suggest for each type and source of error practical steps to be taken to reduce the error and hence its impact on numerical solution.

Analytic Methods, Numerical Methods and Errors.

(a) Analytic versus numerical methods

What is a numerical solution and how does such a solution differ from an exact (true) or analytical solution?

An analytic method for solving a given mathematical problem is any method based on rigorous mathematical analysis and whose application leads to the true (exact) solution, also known as analytic solution.

A numerical method for solving a given mathematical problem is any method based on rigorous mathematical analysis whose application, in most cases, can only lead to an approximate (non-exact) solution, also known as numerical solution. In some very rare cases, a numerical method may result in an exact solution.


Example 1.1

The exact solutions of the nonlinear equation x2 − 5x + 3 = 0 can be obtained using the well known quadratic formula (analytic method)

x1, 2 =

−b± b2 − 4ac2a .

This gives the analytic solutions x1, 2 =5 ± 13

2 .

On the other hand, the iteration formula (numerical method)

xn+1 = 5xn − 3,n = 0,1,2,... ; x0 = 4.5

may also be applied to approximate one of the two solutions of the given quadratic equation. This method can only give an approximate numerical solution.

DO THIS

Given the numerical integration formulas

Trapezoidal rule: [ ] 0110 ;)()(2

)(1

0

xxhxfxfh

dxxfx

x

−=+=∫

Simpson’s rule: [ ] 0112210 ;)()(4)(3

)(2

0

xxxxhxfxfxfh

dxxfx

x

−=−=++=∫

Verify that:

(i) The trapezoidal rule gives exact values of the integral ∫1

0

)(x

xdxxf for any linear

function baxxf +=)( .

(ii) Simpson’s rule gives exact values of the integral ∫2

0

)(x

xdxxf for any cubic

function : dcxbxaxxf +++= 23)( .

In general, the difference between analytical and numerical solutions can be summed up by the statement: Analytical solutions are exact while numerical methods are only approximate.


(b) Need for numerical methods

Why should one learn numerical methods? Are numerical methods needed?

Because of the above distinction between analytic and numerical solutions one can easily be tempted to conclude that one should only use analytic methods in solving mathematical problems. In other words, there is no need to learn numerical methods because they can only give approximate solutions. Such a conclusion is misguided. We need to learn numerical methods for the following three main reasons.

1. For some problems the analytical solution may not be known. A typical examples is given by the following cases.

x 1.0 1.25 1.5 1.75 2.0

)(xf 0000.1 1180.1 1180.1 3229.1 4142.1

2. An integral, such as ∫1

0

2

dxe x , is perfectly defined but the anti‑derivative of the in-

tegrand 2

)( xexf = cannot be expressed using known mathematical functions.

3. In some cases, it may be possible to find a mathematical expression for the analytical solution of a given problem. However, the expression may be com-putationally too complicated to handle numerically. A typical problem is that of

finding an anti‑derivative of the function 38

1)(

xxf

−= .

After some tedious manipulations involving factorization of the denominator fol-lowed by application of the method of partial fractions, one finds the general anti‑derivative

Cxxxxx

xF +⎟⎟⎠

⎞⎜⎜⎝

⎛

−−

+++⎟⎟

⎠

⎞⎜⎜⎝

⎛ += −

44

42ln

24

1

3

1tan

12

3)(

2

21

where C is an arbitrary constant of integration. This complicated result makes the

evaluation of a typical associated definite integral ∫ −

2

1

38 x

dx almost impossible to

carry out with any meaningful degree of accuracy.


DO THIS

(i) Verify that )(xfdx

dF= (using the function F in example 3 above).

(ii) By factorizing 38 x− = )42)(2( 2 ++− xxx and applying the method of

partial fractions derive the expression for )(xF as an anti-derivative of )(xf .

(c) Errors

What are errors in the context of mathematics? What are the main sources of com-putational errors? How can one eliminate errors or reduce their effect on numerical solutions?

In the key words section you can find definitions of the key concepts and stated the key theorems and principles relevant to the topic of numerical methods. This includes the definitions of errors, absolute errors, relative errors, initial errors, discretization errors, truncation errors and rounding errors. For ease of reference we list them here.

Assumption

Let *X be an approximation to an exact (true) quantity X . Then,

The absolute error in *X is defined by *XX − .

The error in *X is defined by *XX −

The percentage error in *X is defined by X

XX *

100−

%

Since the exact (true) value X is normally not known, one replaces it with the

approximate value *X in the denominator of the expression for the relative and percentage errors.

Precision and Accuracy

Measurements and calculations can be characterized with regard to their accuracy and precision.


Precision refers to how big or how small the absolute error *XX − is. The absolute error is therefore a measure of the precision of an approximation.

Accuracy refers to how closely the approximation *X agrees with the true value X

. Here, what counts is not only the magnitude of the deviation *XX −but its size

relative to the true value X . Accuracy is therefore measured by the relative error

X

XX *−.

(d) Types and Sources of Errors

We now list the sources and types of errors and briefly discuss methods of eliminating or reducing such errors so that the numerical solution we get is not seriously affected by them to the extent of rendering it meaningless.

(i) Initial errors

Any mathematical problem meriting to be solved numerically involves some initial data. Such data may be in the form of coefficients in a mathematical expression or entries in a matrix. If this initial data is not exact, then the deviations from their respec-tive true values are called initial errors. In some problems, uncertainties in the initial data can have devastating effect on the final numerical solution to the problem.

(ii) Discretization error

Most of the literature on the subject of computational errors does not make a distinc-tion between discretization and truncation errors, the reason being that the two types of errors are almost inseparable. In this presentation we separate the two because truncation errors are special types of discretization errors.

The true (exact) solutions of some mathematical problems are continuous functions

)(xfy = of their respective independent variables. In almost all cases, numerical methods for solving such problems approximate the unknown continuous solution

)(xf by a sequence { })( nxf of approximate values of the solution at a discrete set of

points { }nx in the domain of the solution function )(xf . For example, the continuous

function xexxf −+=)( is the solution of the initial value problem xyy +=+ 1/ ,

1)0( =y


A typical numerical method for solving this problem is given by the recurrence relation

,1,0 00 == yx ,)1()1( 11 hxyhy nnn −− ++−= ,...3,2,1=n , h being a constant distance between two consecutive discrete values of the variable x . The error resulting from such a discretization process is called discretization error.

(iii) Truncation error

Truncation errors are special types of discretization errors. The term truncation error refers to the error in a method, which occurs because some infinite process is stopped prematurely (truncated) to a fewer number of terms or iterations in the process.

Such errors are essentially algorithmic errors and one can predict the extent of the error that will occur in the method.

Specifically, the solution obtained using some numerical methods may involve infinite processes. For instance, this is the case with all convergent iteration methods and convergent infinite series. Since such infinite processes cannot be carried out indefi-nitely, one is forced to stop (truncate) the process and hence accept an approximate solution. The error caused though this unavoidable termination of an infinite process is called a truncation error.

(iv) Rounding error

Rounding errors are errors introduced during numerical calculations due to the ina-bility of calculating devices to perform exact arithmetic. For example, if we multiply two numbers, each with six decimal digits, the product will have twelve decimal digits. Unfortunately some calculating devices may not be able to display all twelve decimal digits. In such cases one is forced to work with fewer digits thereby necessitating dropping some of the (less significant) digits on the right of the product. The error so introduced is called a rounding error.

(e) Methods of reducing errors

In the spirit of “prevention is better than cure” we shall attempt in this section to give practical suggestions of ways to eliminate or reduce the impact of various types of computational errors that are encountered in resorting to numerical methods.

(i) How to reduce initial error

Initial errors can have a devastating effect on numerical solutions.

We illustrate a typical case involving an example taken from Francis Sheid, Nume-rical Analysis, Shaum Outline Series, 1968 page 342 involving the solution of the following two simultaneous linear equations.

x - y = 1x -

y00001.1= 0


The true (analytical) solution is 001,100=x , 000,100=y . In this example, the

set of initial data consists of the elements of the coefficient matrix

and the right hand side vector .

However, if the entry 00001.1− in the matrix A

is changed

to 99999.0− while all other data items remain unchanged, the resulting system of equations

x - y = 1x -

y99999.0= 0

has the drastically changed exact (analytical) solution 999,99−=x , 000,100−=y

This somewhat startling result demonstrates how a small change in the initial data can cause disproportionately large changes in the solution of some problems.

Thus, the only way to reducing or if possible, to eliminate initial errors is by ensuring that all data given with or computed for use in solving a problem is as accurate as is humanly possible.

(ii) How to reduce discretization errors

Different numerical methods for approximating the solution of a given mathematical problem can result in numerical solutions with very different degrees of accuracy due to the magnitudes of their respective discretization errors. Consider the problem of evaluating the definite integral:

∫ +

1

0 21 x

dx. The exact (analytical) value of the integral, correct to six decimal places

is 785398.0)1(tan 1 =− .

Let us apply the trapezoidal method and Simpson’s rule using an interval length

25.0=h . First we evaluate the integrand 21

1)(

xxf

+= at the relevant points and

get

⎥⎦

⎤⎢⎣

⎡

−

−=

00001.11

11A

⎟⎟⎠

⎞⎜⎜⎝

⎛=

0

1b


i 0 1 2 3 4

ix 0 25.0 5.0 75.0 1

)( ixf 000000.1 941176.0 800000.0 640000.0 500000,0

The trapezoidal rule [ ]⎭⎬⎫

⎩⎨⎧

++++ )()()()(2)(2 43210 xfxfxfxfxfh

gives the numerical solution 782794.0 .

Simpson’s rule [ ]{ })()(2)()(4)(3 42310 xfxfxfxfxfh

++++ leads to the numerical solution 0.785392.

One observes that, while the solution obtained using the trapezoidal rule is correct to only two decimal places, the solution obtained using Simpson’s rule is correct to four decimal places. This significant difference in the accuracy of the two numerical solutions is caused by the differences in the discretization errors of the two nume-rical methods. Simpson’s rule has a smaller discretization error than the trapezoidal rule.

In general, discretization errors cannot be avoided. However, one can reduce them substantially by being careful in selecting a numerical method whose discretization error is known a priori to be relatively small.

(iii) How to reduce truncation errors

Truncation errors are caused by the unavoidable need to stop a convergent infinite process in efforts to get a solution. The size of the truncation error will therefore depend on the particular infinite process (numerical method) being used an on how far we are prepared to carry on with the infinite process.

The truncation error can be reduced either by

(a) Choosing a numerical method with a small truncation error or(b) Carrying out the infinite process sufficiently far.


Example 1.2

The continuous function 13)( 2 +−= xxxf has a root which lies in the interval

10 << x (Why?). Using the quadratic formula, the exact value of the root correct

to six decimal places is 381966.0=ρ . A number of iterative methods exist for approximating such a root. Here we consider two such methods:

The bisection method

xn+1 =xn − xn−1

2, provided f (xn ) f (xn−1 ) < 0 .

The Newton-Raphson method

x = x −f (xn )f / (xn ) , provided f / (xn ) ≠ 0 .

If one performs only three iterations (truncation after three iterations) with each

method using the starting values 00 =x and 11 =x for the bisection method and

01 =x for the Newton-Raphson method, one gets the following sequence of ap-proximations for each method.

Method Initial Values2x

3x 4xBisection

00 =x 11 =x 500000.0 250000.0 375000.0

Newton Raphson 01 =x 333333.0 380952.0 381966.0

These results demonstrate that in stopping the infinite process (iteration) after the third iteration, the truncation error of the Newton Raphson method is much smaller than that of the bisection method.


DO THIS

Continue applying the bisect method on the above example until the solution is correct to three decimal places. How many more iterations did this require?

(iv) How to Reduce Rounding Errors

Before we discuss this important last task in our learning activity we shall first intro-duce a few terms that will frequently be mentioned and used in the process.

•What are Figures or Digits

In computational mathematics, the words “figure” and “digit” are synonyms. They are used interchangeably to mean any one of the ten numerals in the set {0, 1, 2, 3, 4, 5, 6, 7, 8, 9}.

In the decimal system of real numbers, a number N is a string or an ordered se-quence of figures or digits A typical example is the number

00000450700.03650007392060=N

A number can be viewed as a measure of the size or magnitude of some real or ima-ginary quantity. The position of each digit in the string of digits has direct bearing on the importance or significance of that digit (figure) in the overall measure of the size or magnitude of the quantity the number represents.

Intuitively we know that the leftmost digit 7 in the number N above is more signi-ficant than the rightmost digit 7.

• Which digits in a number are Significant?

The following rules apply in deciding which digits or figures in a given number are significant.

1. Nonzero integers are always significant figures.2. Any zeros on the leftmost part of a number are not significant.3. All zero digits positioned between nonzero digits are significant.4. Zeros at the rightmost end of a number are counted as significant only if the

number contains a decimal point.

• How many Significant Figures are in a given Number

The number of significant figures in a given number is found using the following rule:

Rule 1: The number of significant figures in a purely integer number (with no decimal digits) is obtained by counting, starting with the leftmost nonzero digit and ending with the rightmost nonzero digit.


Example 1.3

The number 541500409 has 9 significant figures.

The number 002507030 has 6 significant figures

Rule 2: The number of significant figures in a number having a decimal part is ob-tained by counting all the digits, starting with the leftmost nonzero digit.

Example 1.4

The number 6.00213 has 6 significant figures.

The number 6.00213000 has 9 significant figures

NOTE: All zero digits at the end of a decimal number are significant.

(iv) How to Reduce Rounding Errors

Armed with the concepts of digits/figures and significant figures in a number we can now comfortably discuss ways of reducing rounding errors.

One obvious method of dealing with the problem of rounding errors is to work with the maximum allowable accuracy on our calculating device at each stage in our calculations.

Example 1.5

Find the sum of the numbers 2.35, 1.48, 4.24 using a calculating device that can only handle numbers with two significant figures.

The exact sum is 07.824.448.135.2 =++=S

If we neglect the second decimal digit from each term and form their sum we find the approximate sum

9.72.44.13.21 =++=S

The absolute error in 1S is: 17.01 =− SS

A better approximation of S under the same limitations is

1.82.45.14.22 =++=S


The absolute error in 2S is: 03.02 =− SS

This error is significantly smaller than that in 1S .

The immediate question expected to be raised by the learner is “How did one arrive

at the two digit terms in the sum 2S ?”

The answer to the above question is simple. Each term has been obtained from its corresponding three-digit term by rounding.

The learner will soon know how to round numbers

What it means to round a number

To round a number to a fixed number of figures or digits simply means leaving out (dropping) all digits on the right hand side of the number beyond a certain posi-tion.

If a number is rounded simply by dropping all digits beyond a certain position on the right hand side of the number without making any adjustments to the last retained digit, then one speaks of “rounding off or chopping the number”.

Example 1.6

The sum 1S has been calculated using terms obtained from the original numbers by rounding off (chopping) the third decimal digit from each term. The term 2.35 was rounded to 2.3, the term 1.48 was rounded to 1.4 and the term 4.24 was rounded to 4.2. In each case, the last retained digit (the first decimal place) has not been adjusted in the process of rounding.

Note

The sum 2S has also been obtained through rounding. However, the rounding this time is different. Here, not all the three terms have been rounded off!

The term 2.35 has been rounded to 2.4The term 1.48 has been rounded to 1.5The term 4.24 has been rounded to 4.2

We observe that in rounding each of the first two terms 2.35 and 1.48, the digit occu-pying the second decimal position has been dropped but the digit occupying the first decimal position has been adjusted by increasing it by one (unity). The third digit 4.24 has simply been rounded off.

This practice (or as yet unknown rule for rounding numbers) seems to have some significant advantage over rounding off manifested by the above example in which

2S is more accurate than 1S .


Rules for Rounding Numbers

In order to reduce the error in rounding numbers, the rejection of digits beyond some predetermined position ( n ) is accompanied by making adjustments to the digit re-tained in position (n-1). The adjustment involves either leaving the digit in position ( n ) unchanged or increasing it by one (unity). The decision to retain or increase by 1 the digit occupying position (n-1) is governed by the following rules.

(a) If the digit in position ( 1+n ) is greater than 5 then the digit in position ( n ) is increased by 1.

(b) If the digit in position ( 1+n ) is 5 and at least one other digit to its right is non zero then the digit in position ( n ) is increased by 1.

(c) If the digit in position ( 1+n ) is less than 5 then the digit in position ( n ) is left unchanged.

(d) If the digit in position ( 1+n ) is 5 and all other digits to the right of position

( 1+n ) zero, then

(i) The digit in position ( n ) is increased by 1 if it is an odd number )9,7,5,3,1( ;(ii) The digit in position ( n ) is retained unchanged if it is an even number

)8,6,4,2,0( .

Example1.7

Rounding a given number correct to two significant figures

S/N Number Rounded to 2 Significant figures

Rule Used

1 8.361 8.4 (a)

2 8.351 8.4 (b)

3 8.350 8.4 (d) (i)

4 8.450 8.4 (d) (ii)

5 8.050 8.0 (d) (ii)

6 8.349 8.3 (c)

7 2.55 2.6 (d) (i)

8 2.65 2.6 (d) (ii)

9 0.0557 0.056 (a)

10 0.0554 0.055 (b)


Formative Evaluation: Students should work through this exercise carefully writing full solutions for each problem. They should check their work thoroughly using the solutions provided.

Questions

1. (a) Using the method of substitution find the exact solution of the linear system of equations

905.16107

075.1275

=+

=+

yx

yx

(b) Round the values on the right hand side of each equation to two significant figures and then find the exact solution of the resulting system of linear equation.

(c) Use the solutions obtained from the two systems of equations to explain why initial errors need to be avoided as much as possible.

2. (a) How many significant figures are in each of the following numbers:

(i) 00001000020000 (ii) 10000200003004 (iii) 000123.0004500

(b) Round each of the following numbers correct to five significant figures.

(i) 0123.395 (ii) 0123.205 (iii) 0123.206

3. Given the quantity 20

3

11

3

3

1−⎟

⎠

⎞⎜⎝

⎛ +=X , perform the following calculations:

(a) Find the exact value of X correct to five significant figures.

(b) Approximate value of X using three digit chopping arithmetic (rounding without making any adjustments).

(c) Approximate value of X using three digit rounding arithmetic.(d) Calculate the absolute errors and percentage errors in the approximations

obtained in parts (b) and (c).


4. The truncation error )(xE in linearly interpolating a function )(xf between two

points 0x and 1x , with hxx += 01 , is given by )())((2

1)( //

10 ξfxxxxxE −−=

where ξ is some point in the interval ),(: 10 xxI = .

(a) Using the second derivative test, show that

Mh

xEMaxIx 8

)(2

=∈

, where

M = Max

x∈If / / (x)

.

(b) If )sin()( xxf = determine the value of h for which the truncation error

will always be less than 01.0 .

5. Assuming that the function )(xf has a single root ρ lying inside the closed

interval, bxa ≤≤ the bisection method for approximating ρ uses the iteration formula

212 −− +

= iii

xxx , ,....3,2,1=i ax =−1 , bx =0 , 0)()( 12 <−− ii xfxf

(a) Prove by mathematical induction that the error in the thi iterate ix is given by

ii

abE

2

−= .

(b) If 0=a and 1=b , how many bisection iterations will be needed to obtain

an approximation with an error not greater than 310 − ?

Solutions

1. (a) 415.2=x , 0=y .

(b) 1=x , 1=y . (c) Initial errors must be avoided because the solutions of some problems may

be very sensitive to relatively small initial errors.

2. (a) (i) 6 (b) (i) 123.40

(ii) 14 (ii) 123.20

(iii) 10 (iii) 123.21


3. (a) Exact value 45606.0=X

(b) Approximate value 455.0=bX

(c) Approximate value 456.0=cX

(d) Absolute error 00106.0=− bXX

Absolute error 00006.0=− cXX

Percentage errors %23.0100 =⎟⎟⎠

⎞⎜⎜⎝

⎛ −

X

XX b

Percentage errors %013.0100 =⎟⎟⎠

⎞⎜⎜⎝

⎛ −

X

XX c

4. (a) Consider the function ))(()( 10 xxxxxg −−= whose first and second deri-

vatives are: )(2)( 10/ xxxxg +−= and 2)(// =xg , respectively. )(xg has a

single critical point at 2

10 xx +=ζ , and since 0)(// >ςg , we conclude that

)(ςg = ))(( 10 xx −− ςς

= ⎥⎦

⎤⎢⎣

⎡ −+⎥⎦

⎤⎢⎣

⎡ −+ 110010 2

1

2

1

2

1

2

1xxxxxx

= ( ) ( )10012

1

2

1xxxx −−

= 4

2h−

We therefore conclude that and hence the result that

Max E xh

M| ( ) |=2

4 .

Max g xh| ( ) |=

2

4


(b) With )sin()( xxf = , we have 1=M and therefore we need to find h such

that 01.08

2

≤h

. This leads to the value 3.008.0 <≤h .

5. (a)Mathematical induction ii

abE

2

−=

Test : Formula is true for 1=i because the error in the first bisection is

2

ab − .

Assumption: Let the formula hold for 1>= ki (fixed). This means

kk

abE

2

−= .

Induction: Error in kkk EEx2

111 == ++

1222

1+

−=⎥

⎦

⎤⎢⎣

⎡ −kk

abab = . Q.e.d

(b) If 0=a and 1=b , then iiE

2

1= . This will not exceed

310 − if .

10)2ln(

)1000ln(=≥i


Learning Activity 2: Interpolation

Summary

The concept of interpolation is important in any introductory course on numerical methods. Numerical approximation addresses both the need to approximate numbers as well as approximating functions. Numerical interpolation approximates functions. We approximate functions for one or several of the following reasons:

• A large number of important mathematical functions may only be known through tables of their values.

• Some functions may be known to exist but are computationally too complex to manipulate numerically.

• Some functions may be known but the solution of the problem in which they appear may not have an obvious mathematical expression to work with.

Interpolation as it is presented in this Learning Activity will give the Learner an op-portunity to experience first hand some practical applications of numerical methods

in solving the mathematical problem of approximating a function )(xf which is known only through a set of values at a finite number of points.

This Learning Activity covers the following subtopics:

• Linear interpolation • Lagrange interpolation polynomial • Newton’s divided differences interpolation polynomial • Finite differences interpolation polynomials

We start off by defining the concept of interpolation. Linear interpolation is used to illustrate the concept. This is followed by presenting special interpolation polynomials. Specifically, we discuss interpolation polynomials based on Lagrangian interpola-tion coefficients, Newton’s divided differences and on finite differences.

Specific Learning objectives

At the end of this Learning Activity the learner will be able to:

• Explain why numerical methods are needed in solving interpolation pro-blems.


• Apply Lagrangean interpolation polynomials.• Apply Newton’s divided differences interpolation polynomial.• Define and manipulate finite difference operators (shift, forward, backward,

central and mean difference operators).• Construct tables of differences for tabulated function values.• Derive and apply Newton’s forward, Newton’s backward and Stirling’s

central difference interpolation polynomials, and finally• Use finite difference interpolation polynomials in deriving some numerical

integration methods.


Wikipedia: Interpolation


Wolfram MathWorld (visited 03.04.07)

http://mathworld.wolfram.com Students should search for the entry covering the unit title. Also, search for

any key words in the text. Mathworld gives a detailed reference in all cases.Wikipedia (visited 03.04.07)






Key Terms and Theorems

Full definitions are given in the text.

Interpolation: approximation of a function using discrete values.

Interpolation polynomial: a polynomial which interpolates a function using discrete values provided.

Divided Differences: Differences of the function values provided, related to the differences between the discrete points provided.


Finite difference operators: Mathematical operators used in constructing the diffe-rences in the function values.

Finite difference tables: A table showing different orders of the differences in the function values.

Introduction

How Many Learning Activities are in this Module?

As stated in the summary above, numerical methods are used both for approximating numbers as well as for approximating functions. The contents of this module are covered in four Learning Activities. Two of these Learning Activities are on methods of approximating numbers (the definite integral and roots of functions). Another Learning Activity is on errors and a fourth Learning Activity is on numerical methods for approximating functions.

Why are Polynomials Chosen to Approximate Functions?

Functions are approximated using other functions deemed to be simple to manipulate numerically. Specifically, one uses polynomials to approximate other complicated functions, mainly because polynomials are

• Simple to evaluate• Simple to differentiate, and• Simple to integrate.

We begin the presentation by defining the concept of interpolation and establishing the need for numerical methods for carrying out interpolation. This is followed by a detailed discussion of the simplest polynomial used in approximating functions, liner interpolation. Higher order interpolation polynomials are then introduced, including

• Lagrangian interpolation polynomials• Newton’s divided differences interpolation polynomials, and finally


• Finite difference interpolation polynomials.

At the end of the Learning Activity we briefly discuss the possible use of interpolation polynomials in deriving some numerical integration methods such as the Trapezoidal rule and Simpson’s rule.

Learning activity # 2

2.1 Meaning of Interpolation

The mathematical concept of interpolation is concerned with the problem of approxi-

mating a function )(xf defined over a closed interval ],[ ba .

The function )(xf to be approximated is usually defined through a set of its values

at 1+n distinct points lying in the interval ],[ ba . Typically one is given the pairs of values

bxaxnkxfx nkk === ,;,...,2,1,0));(,( 0

The problem is then to find values (or even derivatives) of the function at some non tabular points lying inside the interval.

These types of problems are quite common in experimental physics and chemistry, where an algebraic expression for a function may not be known but its values for different values of its independent variable may be obtained experimentally through laboratory measurements.

Typical Example (Bajpai et al 1975 pp 205)

In an experiment carried out in a physics laboratory, the length y of a wire was measured for various loads x suspended from it. The results of the experiment are as tabulated hereunder:

Table 1: Load versus extension of a wire

:x Load (kg) 0 1 2 3 4 5

:y Length (mm) 2027.1 2029.4 2031.8 2034.1 2036.5 2039.0


From the data obtained above one may wish to know the length of the wire for any

non tabulated load lying inside the range ]5,0[ . This is known as interpolating

the function )(xfy = , where the functional dependence of y on the load x is not explicitly given here. Any attempt to use the data to calculate the length of the wire for a load outside the given range is called extrapolation.

2.2 Linear interpolation

In linear interpolation we assume that we have two points ),( 00 fxA and ),( 11 fxB

on the curve of a (usually unknown) continuous function )(xfy = and that we now

wish to approximate the value of the function at a point ),( 10 xxx∈ .

DO THIS

Revisit and learn the various forms of defining a straight line and writing the equation of the straight line for each form:

• Two-points given• A point and a slope is given

Because a straight line is completely defined by giving two points which lie on it,

we approximate the function )(xf locally over the interval ],[ 10 xx by the straight line through the two given points.

The equation of the straight line can be given in different forms. Here we consider three forms:

(i) The usual y-intercept - slope or Newton’s form.(ii) The Lagrange form and(iii) The determinant form

For computational purposes, the determinant form is the best of the three forms given.


⎪⎪⎪⎪⎪⎪

⎩

⎪⎪⎪⎪⎪⎪

⎨

⎧

−

−

−

−

−+

−

−

−⎟⎟⎠

⎞⎜⎜⎝

⎛

−

−+

=

xxfxxf

xxiii

fxxxx

fxx

xxii

xxxxff

fi

xP

11

00

01

101

00

10

1

001

010

1

1)(

)(

)()(

)(

Worked Example 1

Using the data obtained from the laboratory experiment of suspending load from a wire and measuring its length, apply the determinant form of linear interpolation to approximate the length of the wire when the load is 2.7 kg

Solution

In this example we substitute the values

7.2,1.2034,3,8.2031,2 1100 ===== xfxfx

into the determinant form of the equation and obtain the value

41.20333.01.2034

7.08.2031)7.2(1 =

−=P

which looks to be quite reasonable in that as may be expected, it is closer to 1f than

it is to 0f .


DO THIS (2.1)

Apply the other two forms of the linear interpolation function )(1 xP to approximate the length of the wire when the load is 1.35 kg

2.3 Error in linear interpolation

We now attempt to answer the question: How big is the error in linear interpola-tion?

In other words, we wish to know how accurate the answers obtained through the process of linear interpolation are.

An important feature of interpolation polynomial )(xPn of any degree is that

nkxfxP kkn ,...,2,1,0)()( == .

For linear interpolation this implies that )()(),()( 1100 xfxPxfxP == .

This further implies that the error )(xE at any point ],[ 10 xxx∈ in linear interpo-lation must have the form

CxxxxxE ))(()(1 −−=

where C is a constant which is independent of x . In a subsequent section of this Lear-

ning Activity the value of C for a function )(xf which is assumed to be sufficiently dif-

ferentiable is given as )(2

1 // ξfC =where ),,(),,( 1010 xxxMaxxxxMin << ξ

With this result, the error in linear interpolation assumes the form

).(

2

1))(()( //

101 ξfxxxxxE −−=

If )(// xfMaxM = over the interval ],[ 10 xx , then one can show that

22011 8

1)(

8

1)( MhxxMxE =−≤


2.4 Interpolation polynomials

As already stated at the beginning of the Learning Activity the main problem of polynomial interpolation may be stated as follows:

Given the values kk fxf =)( of some continuous function )(xf at 1+n distinct

points nkxk ,...,2,1,0, = one is required to construct a polynomial )(xPn of degree n such that it satisfies the following co‑location criteria

nkfxP kkn ,...,2,1,0;)( ==

One can show that this so called “Interpolation polynomial” does exist and is unique. However there are different forms of representing the interpolation polynomial. In this Learning Activity we give four different forms of the interpolation polynomial.

2.5 Lagrangean interpolation polynomial

The Lagrangean form of the interpolation polynomial has the general form

∑=

=++++=n

iiinnn fxLfxLfxLfxLfxLxP

0221100 )()(....)()()()(

in which the terms nixLi ,...,2,1,0),( = are individually polynomials of degree n in x called are called the Lagrangean interpolation coefficients.

To ensure that )(xPn satisfies the co‑location criteria given above, the Lagrangean interpolation coefficients are constructed such that they satisfy the condition

⎪⎩

⎪⎨

⎧

≠

=

==

jiif

jiifxL jiji

0

1

)( ,δ

One can verify that the following definition of )(xLi meets this requirement:


))...()()...()((

))...()()...()(()(

1110

1110

niiiiiii

niii xxxxxxxxxx

xxxxxxxxxxxL

−−−−−

−−−−−=

+−

+−

With this definition of the Lagrangian interpolation coefficients one can now verify the co-location condition on the interpolation polynomial, for

njfffxLxP ji

n

iij

n

ijjijn ,...,2,1,0;)()(

00

==== ∑∑==

δ

Worked Example 2

Construct the cubic Lagrangean interpolation polynomial which interpolates the

function )(xf given by the following table of values:

Table 2

i 0 1 2 3

ix1 2 3 4

if1.54 0.58 0.01 0.35

P3 (x) = 1.54L 0 (x) + 0.58L1 (x) + 0.01L 2 (x) + 0.35L 3 (x)

where

)3)(2)(1(6

1

)34)(24)(14(

)3)(2)(1()(

)4)(2)(1(2

1

)43)(23)(13(

)4)(2)(1()(

)4)(3)(1(2

1

)42)(32)(12(

)4)(3)(1()(

)4)(3)(2(6

1

)41)(31)(21(

)4)(3)(2()(

3

2

1

0

−−−=−−−

−−−=

−−−−=−−−

−−−=

−−−=−−−

−−−=

−−−−=−−−

−−−=

xxxxxx

xL

xxxxxx

xL

xxxxxx

xL

xxxxxx

xL


DO THIS (2.2)

Make use of the above Lagrangean interpolation polynomial to interpolate the func-

tion )(xf at 6.2=x

Observation

(i) We note that the second form (ii) of the equation of a straight line given in the DO THIS (2.1) self-exercise problem is a typical example of the Lagrangean linear interpolation formula.

(ii) In changing from one degree Lagrangian interpolation polynomial to ano-ther, say from linear to quadratic, all calculations must be done afresh. For instance, none of the expressions used in the linear approximation case (the

linear Lagrangian coefficients )(),( 10 xLxL ) are useful in the construction of the quadratic Lagrangian interpolation polynomial.

Because of the second observation, efforts have been made to construct interpola-tion polynomials that are iterative in the sense that, a higher degree interpolation polynomial can be got simply by adding higher-degree terms to an existing lower degree interpolation polynomial. In what follows we shall present two interpolation polynomials in this category.

2.6 Newton’s divided differences

Let the function )(xf be given at 1+n distinct points nxxx ,...,, 10 such that

nkxff kk ,...,2,1,0,)( ==

We define what are known as Newton’s divided differences

nkxxxf k ,...,2,1,],...,,[ 10 = as follows

0

1210321210

03

2103213210

02

1021210

01

0110

],...,,,[],...,,,[]...,,,[

],,[],,[],,,[

],[],[],,[

],[

xxxxxxfxxxxf

xxxxf

xxxxxfxxxf

xxxxf

xxxxfxxf

xxxf

xxff

xxf

k

kkk −

−=

•

•

−

−=

−

−=

−

−=

−


A typical table of divided differences would look as follows

Table 3: Table of Divided Differences

Divided Differences

x )(xfFirst Second Third Fourth

0x )( 0xf

],[ 10 xxf

1x )( 1xf ],,[ 210 xxxf

],[ 21 xxf ],,,[ 3210 xxxxf

2x )( 2xf ],,[ 321 xxxf ],,,,[ 43210 xxxxxf

],[ 32 xxf ],,,[ 4321 xxxxf

3x )( 3xf ],,[ 432 xxxf ],,,,[ 54321 xxxxxf

],[ 43 xxf ],,,[ 5432 xxxxf

4x )( 4xf ],,[ 543 xxxf

],[ 54 xxf

5x )( 5xf

Using these divided differences Newton derived the following n-th degree po-

lynomial )(xPn which interpolates the function )(xf at the 1+n distinct

collocation points nkxk ,...,2,1,0, = : (Burden and Faires, 1989 p 113)

∑=

−

+

−−−+=

−−−−+

+−−−

+−−+−+=

n

kkkn

nn

n

xxxxfxxxxxxxfxP

OR

xxxxfxxxxxxxx

xxxxfxxxxxx

xxxfxxxxxxfxxxfxP

11210100

2101210

3210210

210101000

],...,,[))...()(()()(

],...,,,[))...()()((

...],,,[))()((

],,[))((],[)()()(

We shall illustrate this approach on the following example.


Worked Example 3

(i) Construct a table of divided differences for the function )(xf given by the fol-lowing data:

Table 4: Tabulated values of a function

x -3 -2 -1 0 1 2 3

f(x) -17 -25 -13 -5 -1 23 115

(ii) Use the resulting table of differences to interpolate )(xf at:

3.2−=x using Newton’s linear, quadratic and cubic interpolation polynomials

based at 30 −=x

Solution

Table 5: Table of divided differences

Divided Differences

x )(xfFirst Second Third Fourth

-3 -17

-8

-2 -25 10

12 -4

-1 -13 -2 1

8 0

0 -5 -2 1

4 4

1 -1 10 1

24 8

2 23 34

92

3 115

Interpolation based on 0.3−=x


• Linear Interpolation

Here we take: 3.2,8],[,17)(,0.3 1000 −=−=−=−= xxxfxfx and we

get 6.22],[)()()3.2( 1000 −=−+=− xxfxxxff

• Quadratic Interpolation

Because we are interpolating at the same point 3.2−=x we only need to consider

the additional term ],,[))(( 21010 xxxfxxxx −− whose value at 3.2−=x is

1.2)10)(3.0)(7.0( −=−

Adding this correction to the value obtained through linear interpolation we get

7.241.26.22)3.2( −=−−=−f

• Cubic interpolation

Again we only need to calculate a correction to be made in the value obtained using quadra-

tic interpolation. The additional term is ],,,[))()(( 3210210 xxxxfxxxxxx −−−

whose value is 092.1)4)(3.1)(3.0)(7.0( −=−−− .Making this correction in the value obtained by quadratic interpolation gives

792.25)3.2( −=−f

DO THIS (2.3)

Using the above table of divided differences

(i) Write down the value of the divided difference ],,[ 432 xxxf and show exactly how it has been arrived at.

(ii) Interpolate )(xf at 3.0−=x using a 4-th degree Newton’s divided differences

interpolation polynomial based on the point 0.1−=x .

Compare your answer with the value of the function 5432)( 234 −+−+= xxxxxf at 3.0−=x


2.7 Finite Difference Operators

The third and final set of interpolation polynomials are closely related to those based on Newton’s divided differences.

The basic assumption here is that

The function values kk fxf =:)( are given at a number of equally spaced and

distinct points having a constant interval length kk xxh −= +1 between consecutive collocation points.

A systematic introduction of the method of approach involves the introduction of some difference operators. There are four basic difference operators: The Shift, Forward, Backward and Central difference operators. These are defined as follows.

The Shift Operator E

The shift operator E is defined through the relation

)()( hxfxEf += in the case of a continuous variable x , and through the relation

1+= kk fEf in the case of a discrete variable kx .

Powers of the operator (positive or negative) are defined in a similar manner:

pkkpp ffEphxfxfE +=+= );()(

The Forward Difference Operator Δ

The forward difference operator Δ is defined through the relation

)()()( xfhxfxf −+=Δ in the case of a continuous variable x , and through the relation

kkk fff −=Δ +1 in the case of a discrete variable kx . (1)

Powers of the forward difference operator Δ can also be defined

rrkp

kp

kp ffpfff =Δ=Δ−Δ=Δ ++ 0

11 ,...;12,0 for any value of r .


The Backward Difference Operator ∇

The backward difference operator ∇ is defined through the relation

)()()( hxfxfxf −−=∇ in the case of a continuous variable x , and through the relation

1−−=∇ kkk fff in the case of a discrete variable kx . (2)

Powers of the backward difference operator can also be defined

rrkp

kp

kp ffpfff =∇=∇−∇=∇ −+ 0

11 ,...;12,0 for any value of r .

The Central Difference Operator δ

The central difference operator δ is defined through the relation

)2

1()

2

1()( hxfhxfxf −−+=δ in the case of a continuous variable x , and

through the relation

∇fk = fk − fk −1

in the case of a discrete variable kx . (3)

Powers of the central difference operator can also be defined

δ fk +

12

− δ fk −

12

for any value of r . (4)

Relationship between Difference Operators

The definition of powers of the Shift Operator E enables us to relate the other three difference operators with one another.

We first note that:

2

12

1

+=

kk ffE ; ;

2

12

1

−

−=

kk ffE 1

1−

− = kk ffE

Using these three results, all of which are based on the valid definition of the shift operator we now establish the following relationships:

Because kkkkkk fEfEffff )1(1 −=−=−=Δ + we conclude that


11 +Δ≡−≡Δ EorE (5)

Again, because kkkkkk fEfEffff )1( 111

−−− −=−=−=∇ we conclude that

∇−≡−≡∇ −

1

11 1 EorE

(6)

Similarly, because kkkkk

k fEEfEfEfff )( 2

1

2

1

2

12

1

2

1

2

1

−−

−+−=−=−=δ

we

conclude that

2

1

2

1−

−≡ EEδ (7)

In the case of the central difference operator δ it is not possible to express the shift

operator E in terms of the central difference operator δ alone. However, we shall do so by involving the other finite difference operators. Specifically we can relate the forward and backward difference operators with one another by involving the shift and central difference operators.

(i) Multiplying both sides of equation (7) with 2

1

E and taking note of the result in equation (5) we get

Δ≡−=−=−

1)( 2

1

2

1

2

1

2

1

EEEEE δ , therefore

Δ≡⇒≡Δ−

2

1

2

1

, EE δδ (8)

(ii) Multiplying both sides of equation (7) with 2

1�E and taking note of the result

in equation (6) we get

∇≡−=−= −−

−− 12

1

2

1

2

1

2

1

1)( EEEEE δ , therefore

∇≡⇒≡∇ − 2

1

2

1

, EE δδ (9)


Having established all these inter-relationships between the operators δ,,, ∇ΔE one

can now define any powers of the δ,,∇Δ operators.

A last useful operator, the Mean (or averaging) Operator μ is often used in connection with some interpolation formulas that are based on the central difference

operator .δ

The mean operator is defined through the relationship

⎟⎟⎠

⎞⎜⎜⎝

⎛+=

−+2

1

2

12

1kk

k fffμ

(10)

2.8 Finite Difference Interpolation Polynomials

Armed with the five difference operators ( E , Δ, ∇, δ , μ ) we are now ready to list (not derive) a number of interpolation polynomials based on their use. Exactly how they are used will be discussed after introduction of the concept of a table of dif-ferences. Instead of operating with the variable x , such polynomials are usually written in terms of a digital variable s obtained from the transformation of variables

( )0

1xx

hs −=

(1) Newton’s Forward Difference Interpolation Polynomial

00

03

02

00 ...)2)(1(6

1)1(

2

1)(

fks

fsssfssfsfsP

kn

k

n

Δ⎟⎟⎠

⎞⎜⎜⎝

⎛=

+Δ−−+Δ−+Δ+=

∑=

(2) Newton’s Backward Difference Interpolation Polynomial

nk

n

k

nnnnn

fk

ks

fsssfssfsfsP

∇⎟⎟⎠

⎞⎜⎜⎝

⎛ +=

+∇+++∇++∇+=

∑=0

32 ...)2)(1(6

1)1(

2

1)(


(3) Stirling’s Central Difference Interpolation Polynomial

Pn (s) = f0 + sμδ f0 +12

s2δ 2 f0 +

16

s(s2 −1)μδ 3 f0 +1

24s2 (s2 −1)δ 4 f0 + ..

(4) Bessel’s Central Difference Interpolation Polynomial

...)2()1(24

1

)2

1)(1(

6

1)1(

2

1)

2

1()(

2

142

2

13

2

12

2

1

2

1

+−−

+−−+−+−+=

fsss

fsssfssfsfsPn

μδ

δμδδμ

(5) Everett’s Central Difference Interpolation Polynomials

....5

2

3

1

....5

2

3

1)(

04

02

0

14

12

1

+⎟⎟⎠

⎞⎜⎜⎝

⎛ ++⎟⎟

⎠

⎞⎜⎜⎝

⎛ +++

+⎟⎟⎠

⎞⎜⎜⎝

⎛ ++⎟⎟

⎠

⎞⎜⎜⎝

⎛ ++=

ft

ft

tf

fs

fs

sfsPn

δδ

δδ

where st −= 1

2.9 Tables of Differences

All five finite difference interpolation polynomials given above contain forward or backward or central differences of powers exceeding one. It is therefore important to know how these quantities can be got. The answer is not difficult. These quantities are obtained from tables of differences constructed using the given values of the

function )(xf at the equidistant collocation points kx .


The tables of values are obtained in a process similar to that taken in constructing the Newton’s divided differences. The only difference here is that only differences of function values are involved. Such differences are not divided by any quantity. The following are typical examples of tables of differences.

Table 6: Forward Difference Table

FORWARD DIFFERENCES

x )(xfFirst Second Third Fourth Fifth

0x 0f∆f

0

1x1f

∆2f0

∆f1

∆3f0

2x 2f∆2f

1∆4f

0

∆f2

∆3f1

∆5f0

3x 3f∆2f

2∆4f

1

∆f3

∆3f2

4x 4f∆2f

3

∆f4

5x 5f

Observation

Note the forward slopping nature of the entries based at any point kx in the table of forward differences. The Newton Forward Interpolation polynomial uses value along such a path


Table 7: Backward Difference Table

BACKWARD DIFFERENCES


0x 0f∇f

1

1x 1f∇2f

2

∇f2

∇3f3

2x 2f∇2f

3∇4f

4

∇f3

∇2f4 ∇5f

5

3x 3f∇2f

4∇5f

4

∇f4

∇3f5

4x 4f∇2f

5

∇f5

5x 5f

Observation

Note also the backward slopping nature of the entries based at any point kx in the table of backward differences. The Newton Backward Difference Interpolation Po-lynomial uses values along such a path.


Table 8: Central Difference Table

CENTRAL DIFFERENCES


x-3

f-3

δf-2 ½

f-2

f-2 δ2f

-2

δf-1 ½

δ3f-1 ½

f-1

f-1 δ2f

-1δ4f

-1

δf - ½

δ3f - ½

δf 5- ½

0x 0fδ2f

0δ4f

0

δf½

δ3f ½

δ5f½

1x 1fδ2f

1δ4f

1

δf1 ½

δ3f1 ½

2x 2fδ2f

2

δf2 ½

3x 3f

Observation

Note further the horizontal slopping nature of the entries based at any point kx in the table of central differences. Stirling’s, Bessel’s and Everret’s formulas use table entries on and in the neighborhood of such a path.


DO THIS (2.4)

(i) In reference to Table 9 (Page 57) and taking 0.00 =x give the values of the following finite differences:

254

22

2

11

12

3 ,,,,, ffffff μδδ ∇Δ∇Δ

(ii) Using appropriate equivalences express 86 fδ in terms of forward and backward

differences.

2.10 Application Of Difference Tables In Interpolation

We are now ready to illustrate both how to construct a table of differences and also how to use it for interpolation purposes.

Worked Example 4

(i) Construct a table of differences for the function tabulated bellow.

x 0 0.2 0.4 0.6 0.8 1.0

f(x) 1.0000 0.9801 0.9211 0.8253 0.6967 0.5403

(ii) Use the resulting table to interpolate )(xf

• At 1.0=x using Newton’s forward difference formula

• At 9.0=x using Newton’s backward difference formula and

• At 5.0=x using Stirling’s central difference formula.


Solution

(i)

Table 9: Table of differences

Differences

x f(x) First Second Third Fourth Fifth

0.0 1.0000

-0.0199

0.2 0.9801 -0.0391

-0.0590 0.0023

0.4 0.9211 -0.0368 0.0017

-0.0958 0.0040 -0.0007

0.6 0.8253 -0.0328 0.0010

-0.1286 0.0050

0.8 0.6967 -0.0278

-0.1564

1.0 0.5403

(ii) Interpolation

Application of Newton’s forward difference formula

We take 0.00 =x . Since the interval length being used is 2.0=h the value of the

parameter s is calculated and found to be 5.02.0

0.01.0=

−=s

Newton’s forward difference formula is given by

..)3)(2)(1(24

1)2)(1(

6

1)1(

2

1)( 0

40

30

200 +Δ−−−+Δ−−+Δ−+Δ+= fssssfsssfssfsfxf

The table entries needed for this calculation are highlighted in the above table of

differences based on 0.00 =x Substitution of these values and the calculated value of s gives the following result

9950.050000664062.000014375.00048875.000995.00000.1)1.0( =−++−≈f

Application of Newton’s backward difference formula

We take 0.1=nx . Again since the interval length is 2.0=h the value of the para-

meter s is calculated and found to be

5.02.0

0.19.0−=

−=s


Newton’s backward difference formula is given by

..)3)(2)(1(24

1)2)(1(

6

1)1(

2

1)( 0

40

30

200 +∇++++∇+++∇++∇+= fssssfsssfssfsfxf

The table entries needed for this calculation are highlighted in the above table of

differences based at 0.1=nx . Direct substitution of these values and the calculated value of s gives

6216.00000390625.00003125.0003475.00782.05403.0)9.0( =−−++≈f

rounded correct to four significant figures.

Application of Stirling’s central difference formula

We take 4.00 =x . Since the interval is 2.0=h the value of the parameter s is

calculated and found to be 5.02.0

4.05.0=

−=s

Stirling’s central difference formula is given b

..)1(24

1)1(

6

1

2

1)( 0

4220

320

2200 +∇−+−+++= fssfssfsfsfxf μδδμδ

The table entries needed for this calculation are not highlighted in the above table but are relatively easy to pick up. For the inexperienced learner the teething problem may lie in finding the averaged values. To maximize transparency in the calculations we point out the following:

⎥⎦

⎤⎢⎣

⎡+=⎥

⎦

⎤⎢⎣

⎡+=

−−2

13

2

13

03

2

1

2

10 2

1;

2

1ffffff δδμδδδμδ

With this clarification we now perform the calculations and find

8776.050000132812.0000196875.00046.00387.09211.0)5.0( =−−−−=f


DO THIS (2.5)

Using the entries in Table 9 apply Everret’s central difference formula based on

4.00 =x to interpolate the function )(xf at 45.0=x .

References for the Learning Activity

A.C. Bajpai, I.M. Calus and J.A. Fairley, Numerical Methods for Engineers and Scientists, Taylor & Francis Ltd., London 1975.

R.L. Burden and D. Faires, Numerical Analysis, PWS-Kent Publishing Co. Boston, Fifth Edition 1989.

Solutions To Formative Evaluation Questions

DO THIS (2.1)

The load 1.35 kg lies between the loads 1 kg and 2 kg. We therefore take and subs-titute into the two linear interpolation formulas:

(i) 24.2030)35.0)(4.2(4.2029)()( 001

0101 =+=−⎥

⎦

⎤⎢⎣

⎡

−

−+= xx

xxff

fxP

(ii)

24.2030)8.2031)(35.0()4.2029)(65.0()( 101

00

10

11 =+=⎥

⎦

⎤⎢⎣

⎡

−

−+⎥

⎦

⎤⎢⎣

⎡

−

−= f

xxxx

fxx

xxxP


DO THIS (2.2)

)(35.0)(01.0)(58.0)(54.1)( 32103 xLxLxLxLxP +++=

At 6.2=x the values of the Lagrangean coefficients are found to be

064.0)36.2)(26.2)(16.2(6

1)6.2(

672.0)46.2)(26.2)(16.2(2

1)6.2(

448.0)46.2)(36.2)(16.2(2

1)6.2(

056.0)46.2)(36.2)(26.2(6

1)6.2(

−=−−−=

=−−−−=

=−−−=

−=−−−−=

L

L

L

L

Substitution into the expression for )6.2(3P gives the result 15792.0)6.2(3 =P .

DO THIS (2.3)

(i) 2],,[ 432 −=xxxf

This result is obtained from the definition of divided differences as follows:

2)128(2

1

)()()()(1],[],[],,[

23

23

34

34

2424

3243432

−=−=

⎭⎬⎫

⎩⎨⎧

−

−−

−

−

−=

−

−=

xxxfxf

xxxfxf

xxxxxxfxxf

xxxf

(i) Interpolation of )(xf at 3.0−=x using Newton’s 4th degree divided differences interpolation polynomial.

Here 0.10 −=x


6279.0],,,,[))()()((

092.1],,,[))()((

42.0],,[))((

6.5],[)(

13)(

],,,,[))()()((

],,,[))()((

],,[))((

],[)(

)()(

432103210

3210210

21010

100

0

432103210

3210210

21010

100

04

−=−−−−

=−−−

=−−

=−

−=

−−−−

+−−−

+−−

+−

+=

xxxxxfxxxxxxxx

xxxxfxxxxxx

xxxfxxxx

xxfxx

xf

xxxxxfxxxxxxxx

xxxxfxxxxxx

xxxfxxxx

xxfxx

xfxP

We then get the answer

5159.66279.0092.142.06.50.13)3.0(4 −=−+++−=−P

On the other hand, the function 5432)( 234 −+−+= xxxxxf is found to have

the same value (-6.5159) at 3.0−=x .

DO THIS (2.4)

(i) One finds

0774.02

1,0007.0

,0391.0,0590.0,:,1286.0

2

11

2

12

254

22

2

11

12

3

−=⎥⎦

⎤⎢⎣

⎡+=−=∇

−=Δ−=−∇−=Δ

ffff

ffexistentNonff

δδμδ

δ


(ii) This is solved using the equivalence relations ∇≡Δ≡−

2

1

2

1

, EE δδ . One finds:

116

386

863

8

6

2

1

56

386

863

8

6

2

1

86

fffEfE

fffEfEf

∇=∇=∇=⎥⎦

⎤⎢⎣

⎡∇=

Δ=Δ=Δ=⎥⎦

⎤⎢⎣

⎡Δ=

+

−−

−δ

DO THIS (2.5)

Using the entries in Table 9 apply Everret’s central difference formula based on

4.00 =x to interpolate the function )(xf at 45.0=x .

We are given the quantities 45.0,2.0;4.00 === xhx and therefore the two pa-rameters required in the application of the Everret method are

75.01,25.02.0

4.045.00 =−==−

=−

= sth

xxs

9004.0...)0368.0)(25.0)(75.0)(75.1(6

1

)0328.0)(75.0)(25.0)(25.1(6

1)9211.0)(75.0()8253.0)(25.0()45.0(

=+−−

+−−++=f

African Virtual University 63

Learning Activity # 3

Numerical Integration

Summary

In this learning activity we focus on numerical integration methods. A major portion of the presentation is devoted to the Newton-Cotes category of formulae. However, we also dwell on the more accurate family of numerical integration methods clas-sified as Gaussian integration methods. Specific attention is given to the two best known Newton Cotes methods: Trapezoidal rule, Simpson’s rule and the application of Richardson’s extrapolation technique on both methods in deriving the Romberg integration method. As for Gaussian integration methods we limit our discussion to the Gaussian method based on Legendre polynomials. The Learning Activity is presented under the following headings:

• The need for numerical integration• Classification of methods• Newton-Cotes formulae• The Trapezoidal Rule• Simpson’s rule• Error terms in the Trapezoidal and Simpson’s rules• Romberg integration• Gaussian integration methods

At the end of this learning activity the learner will be able to:

1. Classify numerical integration methods2. Derive and apply the Trapezoidal rule and Simpson’s rules. 3. Derive and apply Romberg’s integration method based on either the Tra-

pezoidal or Simpson’s rule4. Apply the Gaussian integration method based on Legendre polynomials.

List of relevant readings

Fundamental Numerical Methods and Data Analysis, George W. Collins, chapter 4.

Wikipedia: Numerical Methods/Numerical Integration









MacTutor History of Mathematics (visited 03.03.07) http://www-history.mcs.standrews.ac.uk/Indexes The MacTutor Archive is the most comprehensive history of mathematics on


Key Words

• Anti-derivative of f (x)

A function F (x) is called an anti-derivative of another function f (x) if

dFdx

= f (x).

• Order of an integration method

A numerical integration method is said to be of order p if it gives exact values of the

integral for all polynomial functions f (x) of degree m≤ p .


Learning Activity: Numerical Integration

Introduction

In this Learning Activity we give the features that distinguish the family of New-ton-Cotes methods from the Gaussian integration methods. We use a geometrical approach to derive the Trapezoidal (Trapezium) rule and Simpson’s rule. Because at this juncture the Learner has not been exposed to the concept of orthogonal polyno-mials, we have presented without deriving, the Gaussian integration method based on the Legendre orthogonal polynomials. Illustrative examples are given and solved to assist the learner experience the applications of the methods presented.

3.1 Need for Numerical Integration Methods

In the first learning activity (Section 1.6 (b)) we gave two examples to demonstrate why we need to learn and be able to apply numerical methods in solving some ma-thematical problems. Part (ii) of the first Example 1 and the whole of the second example were purposely chosen to illustrate the need to learn numerical methods for evaluating some definite integrals.

3.2 Classification of Numerical Integration Methods.

Numerical methods for approximating the definite integral f (x)dxa

b

∫ have the ge-neral form

f (x)dx ≅ Wkk = 0

n

∑a

b

∫ f (xk ) .

The coefficients Wk are called weighting coefficients and xk are the abscissas or nodes

taken from the range [a,b] of integration at which the integrand is to be evaluated.

3.3 Order of an integration method

A numerical integration method is said to be of order p if it gives exact values of the

integral for all polynomial functions f (x) of degree m≤ p .


The general numerical integration formula given above has n+1 nodes xk and n+1 corresponding weights, giving a total of 2n+ 2 unknown parameters. Since a polynomial of degree m is completely determined by giving its values at m+1distinct points, it follows that the order of the general numerical integration formula given above is at most 2n+1 .

3.4 Newton-Cotes Numerical Integration Methods

The Newton-Cotes integration methods are derived from the general formula by:

(i) Requiring the n+1 nodes xk to be uniformly distributed over the range of in-

tegration [a,b] such that xk = x0 + kh , k = 0,1,2,...n , where x0 = a , xn = b

and h =

b − an

.

(ii) Determining the n+1 weights Wk such that the formula gives exact values of

the integral for all polynomial functions f (x) of degree at most n .

The Trapezoidal rule and Simpson’s rule fall in this category of methods.

(a) The Trapezoidal Rule

DO THIS (3.1)

(i) What kind of a geometrical figure is a trapezium?(ii) What sides of a trapezium determine its area?(iii) How is the area of a trapezium determined?

The Trapezoidal rule (sometimes referred to simply as the Trapezium rule) is the sim-plest practical numerical integration method. It is based on the principle of finding the area of a trapezium. The principle behind the method is to replace the curve

y = f (x) by a straight line (linear approximation) as shown in the figure 3.1.


Typically, we approximate the area A under the curve )(xfy = between the or-

dinates at 0x and 1x by ( )102

ffh

A +≅ , where )( 00 xff = , )( 11 xff = and

h is the distance between 0x and 1x .

Figure 3.1: Trapezoidal Rule

For the integral ∫b

a

dxxf )( the trapezoidal rule can also be applied by subdividing

the interval ],[ ba into n subintervals ],[ 1 kk xx − , nk ,...3,2,1= , of equal length

1−−= kk xxh , with 0xa = and nxb = , followed by applying the trapezoidal rule

over each subinterval. The area A under the curve )(xfy = between the ordinates

at 0xa = and nxb = can then be approximated by the generalized

Trapezoidal rule

∫ ≅=b

a

dxxfA )( ( ) ( ) ( ) ( )nn ff

hff

hff

hff

h++++++++ −1322110 2

...222

≅ ( )[ ]nn ffffffh

++++++ −13210 ...22

≅( ) ⎥

⎦

⎤⎢⎣

⎡++ ∑

−

=

1

10 2

2

n

kin fff

h

f (x0 )

f (x1 )


Observation

In this generalized Trapezoidal Rule, the nodes are chosen to be the equidistant points

nkhkxxk ,...,3,2,1,)1(0 =−+= while the weighting coefficients kW have been determined such that the formula gives exact value of the integral for all linear

functions of the form baxxfy +== )( . These turn out to be:

hWWWWh

WW nn ======= −13210 ...;2

Example 3.1

Approximate ∫2

1x

dx by the Trapezoidal Rule with 10=n ( 1.0=h ).

Solution

In this example: x

xf1

)( = and 1.0=h

We evaluate the function at the points, xi = 1+ (i −1)0.1, i = 1,2,3,...10 and obtain the pairs of values:

Table 3.1: Value of the function x

xf1

)( =

x )(xf x )(xf

0.1 0.1 6.1 625.0

1.1 0.9091 7.1 5882.0

2.1 8333.0 8.1 5556.0

3.1 7692.0 9.1 5263.0

4.1 7143.0 0.2 0.5

5.1 6667.0

Direct application of the generalized trapezoidal rule gives the approximate value

[ ] 69377.05.0)1877.6(20.1

2

1.02

1

=++≅∫ xdx


(b) Simpson’s Rule

To obtain Simpson’s rule we subdivide the interval ],[ ba into only two equal

subintervals using the points 210 ,, xxx , where x2 − x1 = x1 − x0 = h , and replace the

curve of the general function )(xfy = over the interval ],[ 20 xx by the quadratic

Lagrangean interpolation polynomial

2

1202

101

2101

200

2010

21

))((

))((

))((

))((

))((

))((f

xxxxxxxx

fxxxx

xxxxf

xxxxxxxx

y−−

−−+

−−

−−+

−−

−−=

⎥⎦

⎤⎢⎣

⎡ −−+−−−−−= 2101200212))((

2

1))(())((

2

11fxxxxfxxxxfxxxx

h

The general definite integral f (x)dxx0

x2

∫ is then approximated by the integral

ydxx0

x2

∫ . Without loss of generality, one takes x0 = 0 and gets the formula

ydx = ydx =h30

2 h

∫x0

x2

∫ f0 + 4 f1 + f2[ ]

Because the formula uses values at three points (two equal subintervals), a generalized Simpson’s Rule is only possible when the number n of subintervals is even:

[x0 , x2 ], [x2 , x4 ], [x4 , x6 ], ..., [xn− 2 , xn ] .

One applies the formula over each subinterval and adds the results to get the gene-ralized Simpson’s Rule formula

f (x)dx =h3

a

b

∫ f0 + 4 f1 + f2[ ] + f2 + 4 f3 + f4[ ] + ...+ fn− 2 + 4 fn−1 + fn[ ]

=h3

f0 + 4( f1 + f3 + ...+ fn−1 ) + 2( f2 + f4 + ...+ fn− 2 ) + fn[ ]

=h3

( f0 + fn ) + 4 f2 k −1k =1

n / 2

∑ + 2 f2 kk =1

( n− 2 ) / 2

∑⎡

⎣⎢

⎤

⎦⎥


Example 3.1

Approximate dxx1

2

∫ using Simpson’s rule with n = 10 ( h = 0.1 ).

Solution

This is the same problem solved above using the Trapezoidal Rule. We certainly can apply Simpson’s Rule because the number of subintervals is even (n = 10). Using the values given in the Table 3.1 one obtains

dxx1

2

∫ ≅0.13

1.0 + 4(3.4595) + 2(2.7282) + 0.5[ ] = 0.693147 .

DO THIS (3.2)

(i) What is the exact value of the integral dxx1

2

∫ ?

(ii) Which of the two approximations of the integral obtained using the Trapezoidal rule and using Simpson’s rule is more accurate?

(iii) How is the accuracy of the approximations of either the Trapezoidal Rule or Simpson’s Rule affected by taking a smaller interval h (increasing the number of subintervals / refining the partitioning of the range of integration)?

3.5 Error Terms in the Trapezoidal Rule and in Simpson’s Rule

With an exact value of 0.69314718 which is the value of ln2 , your answers to the other two questions are expected to have been that Simpson’s Rule is more accurate than the Trapezoidal Rule, and that the accuracy of both methods increases with decreasing interval length h . These results are made more vivid by the error terms of the respective methods.


(i) Error Term in the Trapezoidal Rule

For reasonably behaved functions f (x) it can be shown (Fox and Mayers, 1958) that

the exact (true) value I of the integral f (x)dxa

b

∫ is related to the value T (h) obtained by the Trapezoidal Rule with an interval of length h through the expression

I − T (h) = AT h2 + BT h4 + CT h6 + ...

where AT , BT , CT ,... are constants whose values are independent of the interval length h.

(ii) Error Term in Simpson’s Rule

Similarly, the error in Simpson’s approximation S(h) of the integral I is given by

I − S(h) = AS h4 + BS h6 + C S h8 + ...

where AS , BS , C S ,... are again constants whose values do not depend on the interval length h.

The leading error term in the Trapezoidal Rule is AT h2 while that of the Simpson

Rule is AS h4 . This explains why Simpson’s Rule is appreciably more accurate than the Trapezoidal Rule for the same interval length h .

3.6 Romberg Integration

Strictly speaking, Romberg integration is not a Newton Cotes method. Romberg integration is a post-processing technique. It is a method which uses previously computed values to produce more accurate results.

The method takes advantage of the knowledge of the error terms in either the Tra-pezoidal Rule or the Simpson’s Rule to produce a much more accurate approximation of the integral using previously calculated approximate values.


(a) Romberg Integration Using the Trapezoidal Rule

Let h1 and h2 be two different intervals used with the Trapezoidal Rule. From the above form of the error term for the method we can write

I − T (h1 ) = AT h12 + BT h1

4 + CT h16 + ...

I − T (h2 ) = AT h22 + BT h2

4 + CT h26 + ...

Eliminating the constant AT from these two formulae and solving for I we get

I = T (h2 ) + h2

2

h12 − h2

2 T (h2 ) − T (h1 ){ } − BT h14 h2

4 + ...

Specifically, if we choose h2 =12

h1 (halving the interval) we get

I = T (1

2h1 ) + 1

3T (1

2h1 ) − T (h1 )⎧

⎨⎩

⎫⎬⎭−

14

BT h14 + ...

The quantity

T (h1 ,12

h1 ) = T (12

h1 ) + 13

T (12

h1 ) − T (h1 )⎧⎨⎩

⎫⎬⎭

is an approximation of the integral I with a much smaller error than either of the

two approximations T (h1 ) andT (h2 ) . Its value is called the Romberg value of the integral with respect to the Trapezoidal Rule.

(b) Romberg Integration Using Simpson’s Rule

Derivation of a Romberg integration formula based on Simpson’s Rule follows the path as that used in connection with the Trapezoidal Rule.

Let h1 and h2 be two different intervals used with Simpson’s Rule. From the above form of the error term for the method we can write


I − S(h1 ) = AS h12 + BS h1

4 + C S h16 + ...

I − S(h2 ) = AS h22 + BS h2

4 + C S h26 + ...

Eliminating the constant AS from these two formulae and solving for I we get

I = S(h2 ) + h2

4

h14 − h2

4 S(h2 ) − S(h1 ){ } − h14 h2

4 (h22 − h1

2 )h1

4 − h24 BS + ...

I = S(h2 ) + h2

4

h14 − h2

4 S(h2 ) − S(h1 ){ } − h14 h2

4 (h22 − h1

2 )h1

4 − h24 BS + ...

Specifically, if we choose h2 =12

h1 (halving the interval) we get

I = S(1

2h1 ) + 1

15S(1

2h1 ) − S(h1 )⎧

⎨⎩

⎫⎬⎭−

h6

20BS + ...

The quantity S(h1 ,12

h1 ) = S(12

h1 ) + 115

S(12

h1 ) − S(h1 )⎧⎨⎩

⎫⎬⎭

is an approximation of the integral I with a much smaller error than either of the

two approximations S(h1 ) and S(h2 ) . Its value is called the Romberg value of the integral with respect to Simpson’s Rule.


Example 3.3

Approximate ∫2

1x

dx

by Romberg integration based on the Trapezoidal Rule with

2.01 =h and 1.02 =h

Solution

All the values required for the calculations are given in Table 3.1 One finds

[ ] 69564.05.0)7282.2(20.1

2

2.0)2.0( =++=T

[ ] 69377.05.0)1877.6(20.1

2

1.0)1.0( =++=T .

Therefore

{ } 693147.0)2.0()1.0(3

1)1.0()1.0,2.0( =−+= TTTT

The Romberg value we have obtained by combining two fairly inaccurate values using the Trapezoidal rule has the same high accuracy as the one obtained using

Simpson’s Rule with 1.0=h

DO THIS (3.3)

(i) Is it possible to apply Simpson’s Rule on the integral ∫2

1x

dx using the interval

2.0=h ?

(ii) If your answer is YES, apply the method. If your answer is NO explain carefully why not.


Example 3.4

Construct a table of values of the function x

xf1

)( = at the equidistant points

9,...,3,2,1);1(125.01 =−+= kkxk . Then approximate the integral ∫2

1x

dx by

applying Romberg integration based on Simpson’s Rule with interval lengths

25.01 =h and 125.02 =h .

Solution

Table 3.2: Value of the function x

xf1

)( =

x )(xf x )(xf1.0 0.1 1.625 0.615385

1.125 0.888889 1.75 0.571429

1.25 0.8 1.875 0.533333

1.375 0.727273 2.0 0.5

1.5 0.666667

With these values one finds:

[ ] 693254.05.0)666667.0(2)371429.1(40.1

3

25.0)25.0( =+++=S

[ ] 693155.05.0)038096.2(2)764880.2(40.13

125.0)125.0( =+++=S

Using these two Simpson approximations of the integral, Romberg integration leads to the much more accurate value

. [ ] 6931484.0)25.0()125.0(15

1)125.0()125.0,25.0( =−+= SSSS


3.7 Gaussian Integration Methods

Gaussian integration methods are derived from the general integration formula

)()(0

k

b

a

n

kk xfWdxxf∫ ∑

=

≅

by assuming that all the 22 +n parameters in the

formula (the 1+n nodes kx and their corresponding 1+n weights kW ) are to be determined so that the resulting integration formula gives exact values for all poly-

nomial functions )(xf of degree at most 12 +n .

For functions )(xf defined over the range ]1,1[− Gauss developed a numerical integration formula

∫ ∑−

=

=1

11

)()(n

kkk xfWdxxf

where the nodes are the roots of some special polynomial (Legendre orthogonal po-lynomials) which are symmetrically positioned about the origin and all the weighting coefficients are positive.

The above mentioned restriction of the range of integration for the Gaussian integra-

tion methods is not a serious one because we can transform any finite interval ],[ ba

to ]1,1[− using the transformation

2

1+=

−

− t

ab

ax

or )1(

2+

−+= t

abax

and hence obtain the transformed integral on the right hand side of the equation

dtt

abaf

abdxxf

b

a ∫∫ −⎟⎠

⎞⎜⎝

⎛ +−

+−

=1

1)1(

22)(

on which the Gaussian integration method can be applied.


Extensive tables giving values of the Gaussian nodes and their corresponding weight for different values of n have been constructed ready for use in solving any given definite integral over a finite range.

We reproduce here a table of the weights and nodes for the Gaussian integration

method based on Legendre polynomials for .5,4,3,2,1=n

Nodes and Weights for Gauss-Legendre Integration Method

n Nodes

kt

Weights

kW

1 ± 0.577 350 1.000 000

2 ±

0.000 000

0.774 597

0.888 889

0.555 556

3

±

±

0.339 981

0.861 136

0.652 145

0.347 855

4 ±

±

0.000 000

0.538 469

0.906 180

0.568 889

0.478 629

0.236 927

5

±

±

±

0.238 619

0.661 209

0.932 470

0.467 914

0.360 762

0.171 324

Example 3.5

Approximate the integral ∫ +=

4

2 21 x

dxI by the Gaussian integration method using

the Gauss-Legendre nodes and weights for .4=n

Solution

With 2=a and 4=b the transformation of variables from ]4,2[∈x to ]1,1[−∈t

gives: 3+= tx and dtdx =


∫∫∫ −−=

++=

+

1

1

1

1 2

4

2 2)(

)3(11dttF

tdt

xdx

Direct calculations yield the following values:

568889.00 =W ; 1.0)0( =F

0568889.0)0(0 =FW

;478629.01 =W 073960.0)538469.0( =+F ; 141660.0)538469.0( =−F

[ ] 1032020.0)538469.0()538469.0(1 =−+ FFW

;236927.02 =W 061507.0)906180.0( =+F ; 185733.0)906180.0( =−F

[ ] 0585778.0)906180.0()906180.0(2 =−+ FFW

Adding the three products together gives the value

218667.0=I

This is an astonishingly accurate result for, the true value of the integral is

218669.0107149.1325818.1)2(tan)4(tan 11 =−=− −−

DO THIS (3.4)

Approximate the integral dxxe x∫−

−2

1

)cos( by the Gaussian integration method using the Gauss-Legendre nodes and weights for 4n = .


References for the Learning Activity

Fox, L. and Mayers, D.F., Computing Methods for Scientists and Engineers, - Oxford University Press, London (1958).


DO THIS (3.1)

(i) A trapezium is a quadrilateral which has a pair of parallel sides.

(ii) The sides of a trapezium that determine its area are the pair of parallel sides.

(iii) The area of a trapezium is given by: [ ]212LL

hAArea += , where 21 , LL

are the lengths of the parallel sides and h is the perpendicular distance between them.

DO THIS (3.2)

(i) The exact value of the integral ∫2

1x

dx is 693147.0)2ln( = .

(ii) The approximation obtained using Simpson’s rule is more accurate (has a smaller error) than that obtained using the trapezoidal (trapezium) rule.

(iii) The accuracy of both the trapezoidal rule and Simpson’s rule become more

accurate as the interval length h gets smaller.

DO THIS (3.3)

(i) It is not possible to apply Simpson’s rule on the integral ∫2

1x

dx using the interval

2.0=h .

(ii) The reason why it is not possible is because with 2.0=h the number of subin-

tervals over the range ]2,1[ of integration would be odd (5) while Simpson’s rule can only be applied when the number of subintervals is even.


DO THIS (3.4)

The integral dxxe x )cos(2

1

∫−

− is first transfor-

med into ∫−

1

1

)( dttF using the change of variables:

dtdxtttFtx2

3,1)1(

2

3cos)1(

2

31exp)(;1)1(

2

3=⇒⎟

⎠

⎞⎜⎝

⎛ −+⎟⎠

⎞⎜⎝

⎛ +−=⇒−+=

One uses the following nodes and weighting coefficients

861136.0,861136.0,347855.0

339981.0,339981.0,652145.0

222

111

−=−==

−=−==

ttW

ttW

One finds

( ) ( )[ ] 967614.1)()()()(2

3)cos( 222111

2

1

=−++−+=∫−

− WtFtFWtFtFdxxe x


Learning Activity # 4

Roots of Functions

Summary

This is the fourth and final learning activity for this module. In this learning activity we shall discuss a frequently occurring problem in mathematics: the root finding problem for either a nonlinear equation

0)( =xf

involving a single independent variable x or for a coupled system of two nonlinear equations

0),( =yxf , 0),( =yxg

in two independent variables ),( yx .

At the end of this learning activity the learner is expected to be able to derive, apply and prove that the bisection method always converges; derive and apply both the Secant method and the Regula Falsi method; derive and apply the Newton Raphson method and also derive and apply Newton’s method for a coupled system of nonlinear equations. At the end of our presentation we discuss briefly the concept of fixed points of a function and the theorems which guarantee their existence and uniqueness and relate these results to the root finding problem in a manner which allows the learner to derive her/his own convergent iteration methods.


Wikipedia: Numerical Methods/Equation Solving












Key Words, Theorems

[Full definitions are given in the text]

Root or zero of a function: A value of x at which the function has a value of zero.

Fixed-points of a function: A value of x for which the function has the same value of x (e.g. f(2)=2)

Intermediate value theorem for continuous functions: A continuous function takes all values lying between two values of the function.


Learning Activity : Roots of Functions

4.1 Introduction

Five numerical methods for finding roots of functions will be presented in this lear-

ning activity. The first four methods are for solving the nonlinear equation 0)( =xfThe fifth method is for solving a coupled system of two nonlinear equations in two

variables 0),(,0),( == yxgyxf .

The bisection method will be derived. We prove that the method is always convergent. This will be followed by the method of Regula Falsi, which has a significant simi-larity with the bisection method but converges slightly faster. The Secant method is immediately presented after the Regula Falsi method because the two methods share a common mathematical formula. However, the Secant method is computationally more efficient. The Newton-Raphson method is presented last and is shown to be a generalized form of the Secant and Regula Falsi methods.

4.2 Roots or Zeros of a Function

For a function of a single independent variable )(xfy = , a point ρ=x is called

a root or a zero of )(xf if the value of the function is zero at the point, meaning.

0)( =ρf In Figure 4.1 the points 321 ,, xxxxxx === are all roots of

the function ).(xf

x1 x2 x3

0)(

0)(

0)(

3

2

1

=

=

=

xf

xf

xf

)(xfy =

y

x

Figure 4.1: Roots of a Function )(xf


4.3 Numerical Methods

(a) The Bisection Method

The bisection method for approximating roots of functions is a typical example of an iteration method.

In its simplest form, an iteration method may be defined as a repetitive process of

applying a function )(xg on one or several previous approximate values 1−nx … to

produce a new and hopefully more accurate approximation nx of the specific quantity being sought. In mathematical symbolism we write

,...),,( 321 −−−= nnnn xxxgx

The bisection method is based on the intermediate value theorem for continuous

functions. If )(xf is continuous on an interval ],[ ba and if the values )(af and

)(bf differ in sign, ( 0)()( <bfaf ), then the equation 0)( =xf has at least one

real root ],[ ba∈ρ .

Assuming that the points a and b were chosen to contain only one root we can

bisect the interval ],[ ba into two halves at the point )(2

1bac += and conclude

that the root lies either in the interval ),( ca or in the interval ),( bc provided that

0)( ≠cf , in which case c is the required root.

The Bisection method repeats the process of bisecting the interval which contains the root ρ until we are satisfied that we are close enough to the root.

The above process can be summed up by the following algorithm (Kendal E. Atkin-son, 1989 pp 56).


A lgor ithm “ B isect ( ερ,,,),( baxf )” Steps: 1. Set ax =:1 and bx =:2

2. Define [ ]213 2

1xxx +=

3. If ε≤− 32 xx , then accept 3x=ρ and exit.

4. If 0)()( 32 ≤xfxf then 31 : xx = ; otherwise 32 : xx = .

5. Go back to step 2.

Convergence of the Bisection Method

Convergence of any iteration method implies that the error in the approximation is tending to zero as the number of iteration increases.

For the Bisection method, the absolute value of the error is bounded by the length of the interval in which the root lies at that particular stage.

Error bound after 1st bisection [ ]abx −≤=−2

113 ερ .

Error after 2nd bisection 224 222

1 ababx

−=⎟

⎠

⎞⎜⎝

⎛ −≤=− ερ

Error after 3rd bisection 3235 222

1 ababx

−=⎟

⎠

⎞⎜⎝

⎛ −≤=− ερ

Error after thn bisection nnn

ababx

222

116

−=⎟

⎠

⎞⎜⎝

⎛ −≤=−

−ερ

Because 02

limlim =⎟⎠

⎞⎜⎝

⎛ −=∈

∞→∞→ nnnn

ab we conclude that the bisection method always

converges.

Example 4.1

Verify that the function 104)( 2 −+= xxxf has a root inside the interval )2,1( and use the limits of the interval as starting values of the bisection method to approximate

the root in 10 bisections.

limn→ ∞

εn


Solution

We evaluate the function at the two end points of the given interval and find 5)1( −=f

and 2)2( =f . Since )(xf is a continuous function and 0)2()1( <ff , the interme-

diate value theorem asserts that f has at least one root in the interval 21 ≤≤ x . We can therefore carry out the bisection method to find the results tabulated hereunder.

Table 4.1: Bisection Method for the function 104)( 2 −+= xxxf

n a )(af b )(bf c )(cf1 -5 2 2 1.5 -1.75

1 1.5 -1.75 2 1.5 1.75 0.0625

2 1.5 -1.75 1.75 0.0625 1.625 -0.859375

3 1.625 -0.859375 1.75 0.0625 1.6875 -0.402344

4 1.6875 -0.402344 1.75 0.0625 1.71875 -0.170898

5 1.71875 -.170898 1.75 0.0625 1.734375 -0.054443

6 1.734375 -0.054443 1.75 0.0625 1.742188 0.003971

7 1.734375 -0.054443 1.742188 0.003971 1.738282 -0.025248

8 1.738282 -0.025248 1.742188 0.003971 1.740235 -0.010642

9 1.740235 -0.010642 1.742188 0.003971 1.741212 -0.003333

10 1.741212 -0.003333 1.742188 0.003971 1.741700 0.000319

The sequence of values under the c-column in the table is convergent. The true value of the root being approached by this sequence (use the quadratic formula) is

...741657.1=ρ

DO THIS (4.1)

Verify that the function )cos()( xxxf −= has a root in the interval ]1,0[ and hence apply the bisection method, in only five iterations, to approximate the root.

(b) The Regula Falsi Method

The Bisection method we have just presented was a bit wasteful. In it one spends

great efforts at calculating values of the function f at two points which are only used for deciding in which subinterval the root lies but are never used in the actual calculation of the approximate value of the root.


The method known as Regula Falsi (or rule of false position) rectifies this anomaly. The method retains the principle of enclosure characteristic of the Bisection method but also makes direct use of the function values at the two points which enclose the root of the function.

The general approach for this and subsequent methods (Secant method and the Newton

Raphson method) is to replace the curve of )(xfy = in the interval where the root lies by the straight line joining the two points.

Specifically, let the root ρ lie in the interval ],[ 21 xx . The equation of the secant line

joining the two points ),( 11 fxA and ),( 22 fxB is

. )( 112

121 xx

xxff

fy −−

−+=

Figure 4.2 Regula Falsi Method

y

x

1x

2x

Secant

3x

A

B

This line intersects the x-axis at the point with coordinates )0,( 3x where


12

12211

12

1213 ff

fxfxf

ffxx

xx−

−=⎥

⎦

⎤⎢⎣

⎡

−

−−=

Unlike the Bisection method, the Regula Falsi assumes that the two values 1x and

2x enclose the root being sought by requiring that 021 <ff and at the same time involves the same function values in calculating a new approximation to the root.

The above process can be repeated several times in an iterative process using the following algorithm.

Algorithm “Regula Falsi ( ερ ,,,),( baxf )”

Steps:

1. Deine 112

1213 f

ffxx

xx ⎥⎦

⎤⎢⎣

⎡

−

−−=

2. If ε≤− 31 xx and ε≤− 32 xx then accept 3x=ρ and exit.

3. If 032 ≤ff then 31 : xx = ; otherwise 32 : xx = .4. Go back to step 1.

Example 4.2

Starting with the values 2,1 21 == xx apply the Regula Falsi method on the function

104)( 2 −+= xxxf to obtain an approximate value of the root enclosed in the

interval ),( 21 xx in only four (4) iterations.

Table 4.2: The Regula Falsi Method for the function 104)( 2 −+= xxxf

1 -5 2 2 1.714286 -0.204080

1 1.714286 -0.204080 2 2 1.740741 -0.006857

2 1.740741 -0.006857 2 2 1.741627 -0.000227

3 1.741627 -0.000227 2 2 1.741656 -0.000010

4 1.741656 -0.000010 2 2 1.741657 -0.000003

n1x )( 1xf 2x )( 2xf

12

12213 ff

fxfxx

−

−= )( 3xf


Examination of the values of 3x against the background of the exact value of the

root ...741657387.1=ρ demonstrates the faster convergence of the Regula Falsi method compared to the Bisection method.

Observation

The requirement in both the Bisection and the Regula Falsi methods that the two

values involved in the calculations, 21 , xx must enclose the root is computationally very restrictive and significantly reduce the efficiency of both methods. In a pro-

grammed algorithm, the process of checking whether or not 0)()( 21 <xfxf is time consuming. As a result, efforts have been made to do without it. Our next method achieves just that.

(c) The Secant Method

The Secant method is essentially the same as the Regula Falsi method. The only dif-

ference is that in the Secant method the requirement that the two values 1x and 2x must enclose the root is dropped. All one needs is to require the two values used in the computation be close enough to the required root. The algorithm for the Secant method is as given hereunder.

Algorithm “Secant Method ( ερ ,,,),( baxf )”

Steps:

1. Define 112

1213 f

ffxx

xx ⎥⎦

⎤⎢⎣

⎡

−

−−=

5. If ε≤− 31 xx and ε≤− 32 xx then accept 3x=ρ and exit.

6. Otherwise set 21 : xx = and 32 : xx =

7. Go back to step 1.


DO THIS (4.2)

Starting with 1,0 10 == xx , perform 5 iterations using the secant method to ap-

proximate the root of the function )cos()( xxxf −= .

(d) The Newton-Raphson Method

The Newton Raphson method is by far the most popular numerical method for

approximating roots of functions. The method assumes that the function )(xfis differentiable in the neighborhood of the root and that the derivative is not zero anywhere in that neighborhood.

Assuming that 0x is a point which is sufficiently close to the root of the function,

the graph of the function )(xfy = is approximated by the tangent to the curve at the point.

y

x 1x 2x

Tangent

Fig ure 4.3: Newton -Rap hson M ethod

The equation of the tangent through point ))(,( 00 xfx on the curve )(xfy = is

)()( 0/

00 xfxxfy −+=

This tangent intersects with the x - axis at the point 1x whose value is

)(

)(

0/

001 xf

xfxx −=


The value 1x is then accepted as a new approximation to the root. The point

))(,( 11 xfx can be taken as a new point to draw a tangent through. Its intersection

with the x - axis, given by )(

)(

1/

112 xf

xfxx −= is also accepted as a new approximation

to the root. This process can be repeated over and over, leading to the iteration method given by the

Newton-Raphson formula ,...2,1,0,)(

)(/1 =−=+ n

xfxf

xxn

nnn

Each iteration using the Newton Raphson method requires one function evaluation and one first derive evaluation. Compared to the previous three numerical methods, the Newton Raphson method converges very rapidly to the root.

Example 4.3

Starting with 10 =x approximate a root of the function 104)( 2 −+= xxxf correct to 6 decimal places.

Solution

10)4(104)( 2 −+=−+= xxxxxf

42)(/ += xxf

The requirement that we obtain an answer correct to 6 decimal places simply means that we carry out the iteration until the 7th decimal digit in the values being calculated is no longer changing.


Table 4.3: The Newton-Raphson Method for the function 104)( 2 −+= xxxf

nnx )( nxf

10)4( −+ nn xx

)(/nxf

42 +nx )(

)(/1

n

nnn xf

xfxx −=+

0 1 -5 6 1.833 333 3

1 1.833 333 3 0.6944442 7.6666666 1.742 753 6

2 1.742 753 6 0.0082045 7.4855072 1.741 657 5

3 1.741 657 5 0 7.483315 1.741 657 5

Because the value of the function at 3x is for all purposes zero, we can conclude that

the required approximation is 741657.1=x rounded to six decimal places. The true

value of the root to the same degree of accuracy is 741657.1=ρ

DO THIS (4.3)

Starting with 00 =x apply the Newton-Raphson method in only four iterations to

approximate the root of the function )cos()( xxxf −= .

4.4 Newton’s Method for a Coupled System

Our fifth and last numerical method will focus on solving a system of two simulta-neous nonlinear equations of the general form

0),(,0),( == yxgyxf .

A typical example of such a system is to find the coordinates of the point in the first

quadrant where the parabola 72 2 −= xy intersects with the circle 622 =+ yx .

Here, we are looking for the pairs of values ),( yx which satisfy the two nonlinear equations

06),(,072),( 222 =−+≡=−−≡ yxyxgyxyxf


To obtain a Newton-Raphson type method for approximating the solution of the ge-

neral problem stated above, we let ),( 00 yx be an approximation to the exact solution

),( βα of the coupled system. To obtain an improved solution ),( 11 yx we assume

that the exact coordinates of the point are obtained by making adjustments h and k to our initial values, such that

hx += 0α ; ky += 0β .

and hence, 0),( 00 =++ kyhxf 0),( 00 =++ kyhxg .

Expanding f and g in a Taylor Series about the point ),( 00 yx one gets

...),(),(),(),(0

...),(),(),(),(0

00000000

00000000

+∂

∂+

∂

∂+=++=

+∂

∂+

∂

∂+=++=

yxyg

kyxxg

hyxgkyhxg

yxyf

kyxxf

hyxfkyhxf

where we have deliberately left out all higher order terms and retained only the terms

linear in the increments h and k .

If we truncate the series on the right hand side of each equation after the linear terms we still get

0),(),(),(),(

0),(),(),(),(

00000000

00000000

=∂

∂+

∂

∂+=++

=∂

∂+

∂

∂+=++

yxyg

kyxxg

hyxgkyhxg

yxyf

kyxxf

hyxfkyhxf

but α≠+ hx0 and β≠+ ky0 . We denote the values of x and y which satisfy the above pair of equations with

hxx += 01 ; kyy += 01 .


Solving the resulting system of two linear equations:

0),(),(),(

0),(),(),(

000000

000000

=∂

∂+

∂

∂+

=∂

∂+

∂

∂+

yxyg

kyxxg

hyxg

yxyf

kyxxf

hyxf

for the unknown value h and k

The values of h and k are found to be

D

Dk

D

Dh kh == ,

where kh DDD ,, are the three determinants

gxg

fxf

D

yg

g

yf

fD

yg

xg

yf

xf

D hh

−∂

∂

−∂

∂

=

∂

∂−

∂

∂−

=

∂

∂

∂

∂∂

∂

∂

∂

= ;;

in which all the quantities (function values and partial derivatives) which appear

therein are evaluated at the point ).,( 00 yx

Once these quantities have been calculated, the values of the new approximate solu-tion can be calculated .

The above analysis carried out using the initial approximate solution (x1 , y1 )can now

be repeated using the new pair ),( 21 yx to lead to a new approximate solution ),( 22 yx , and so on, and so on in an obvious iterative manner,

),( 11 yx


The method described above is known as Newton’s method for a system of simulta-neous nonlinear equations. Its speed of convergence is the same as its counterpart in

solving the single nonlinear equation 0)( =xf .

Example 4.4

(a) Using analytical methods find the true solutions of the coupled system of equa-tions

06),(,072),( 222 =−+≡=−−≡ yxyxgyxyxf

(b) Perform two iterations with Newton’s method to approximate a solution of the

coupled pair of equations which is believed to lie close to the point )1,2( .

Solution

(a) Solving the equation 0),( =xg for 2x one gets 22 6 yx −= , and substituting this

expression of 2x into the equation 0),( =yxf one gets [ ] 0762 2 =−−− yy This leads to the quadratic equation 052 2 =−+ yy

Whose roots are

850781.1,350781.1 )2()1( −== yy

The corresponding values for x are found to be

604559.1;043377.2 )2()1( ±=±= xx

(b) 6),(;72),( 222 −+=−−= yxyxgyxyxf

.2;2;1;4 yyg

xxg

yf

xxf

=∂

∂=

∂

∂−=

∂

∂=

∂

∂


First Iteration

1,2 00 == yx

1),(,8),(,0),( 000000 −=∂

∂=

∂

∂= yx

yf

yxxf

yxf

2),(,4),(,1),( 000000 =∂

∂=

∂

∂−= yx

yg

yxxg

yxg

8,1,20 === kh DDD

0.05, 0.4h kD D

h kD D

= = = =

4.1,05.2 0101 =+==+= kyyhxx

Second Iteration

4.1,05.2 11 == yx

1),(,2.8),(,005.0),( 111111 −=∂

∂=

∂

∂= yx

yf

yxxf

yxf

8.2),(,1.4),(,1625.0),( 111111 =

∂

∂=

∂

∂= yx

yg

yxxg

yxg

312.1,1765.0,06.27 −=−== kh DDD

0.006523, 0.048485h kD Dh k

D D= = − = = −

351515.1,043477.2 1212 =+==+= kyyhxx


DO THIS (4.4)

Perform 2 iterations with Newton’s method to approximate the solution of the coupled system of equations

0

022

22

=+−

=−−

yxy

yxxthat lies near the point with coordinates )4.0,8.0( .

(4.5) Fixed-Point Iterations

Definition

A number ρ is called a fixed point of a function )(xg if ρρ =)(g .

The mathematical problem of finding values of x which satisfy the equation

)(xgx = is called the fixed point problem.

(a) Existence Theorem

If )(xg is continuous over ],[ ba and ],[],[)( baxbaxg ∈∀∈ , then )(xg has

at least one fixed point in ],[ ba

To prove this theorem we need the Intermediate Value Theorem for continuous functions.

Proof

If aag =)( or bbg =)( then the proof is complete because then, a or b , or both are

fixed points of )(xg . However, if aag ≠)( and bbg ≠)( then from the assumption

],[)( baxg ∈ made above, the function )(xg satisfies bxga << )( . We define

a function xxgxh −= )()( for ],[ bax ∈ . Like )(xg , the function )(xh is also

continuous over ],[ ba . By evaluating )(xh at ax = and bx = we find

0)()( >−= aagah , 0)()( <−= bbgbh .

The intermediate value theorem asserts that the function )(xh must vanish (have

zero value) at an intermediate point ),( ba∈ρ . At such a point ρ=x we have,

0)()( =−= ρρρ gh which implies ρρ =)(g a result which implies that )(xg

has a fixed point ),( ba∈ρ .


(b) Uniqueness Theorem

The above theorem establishes the existence of at least one fixed point. There could therefore be several fixed points. To guarantee that there is only one fixed point we have the following theorem.

If in addition to the assumptions made above, the function )(xg is differentiable in

),( ba and its derivative satisfies the condition

1),(

<= Kdxdg

Maxba

Then, )(xg has a unique fixed point in ),( ba .

Proof

Assume that )(xg has two different fixed points: 1ρ , 2ρ with 21 ρρ ≠ . Then,

11 )( ρρ =g and 22 )( ρρ =g .

Through subtraction and using the Mean Value Theorem for differentiation we get

))(()()( 12/

1212 ρρξρρρρ −=−=− ggg , where ξ ∈(ρ1 ,ρ2 )

By taking absolute values on both sides of this equation and observing the above

condition on )(/ xg we find

121212/

12 )(( ρρρρρρζρρ −<−≤−=− Kg

This is a contradiction which can only result from the assumption we made that

21 ρρ ≠ . We therefore conclude that under the stated assumptions, the function

)(xg has a unique (only one) fixed point.

(c) Relationship between Root -finding and Fixed -point Problems

We introduced the concept of a fixed point for the purpose of using it in solving our root-finding problem. The relationship between the two problems is simple.


In the root-finding problem we want to find all values of x which satisfy the equa-

tion 0)( =xf .

Let us assume that we are able to express (split) the function )(xf in the form

)()( xgxxf −= .

It is obvious that this splitting can be done in many ways. The particular choice of

)(xg will be critical in converting the root-finding problem into a useful fixed-point problem.

If ρ is a root (zero) of )(xf then

ρρρρρρρ =⇒=−⇒=−= )(0)(0)()( gggf

This result implies that the roots of )(xf are the fixed points of )(xg .

What we now need to do is to find the best splitting of the function )(xf .

Such a splitting must result in a function )(xg which has a unique fixed point in a given interval. We shall demonstrate the splitting process on a typical example.

Example 4.5

Consider the root-finding problem 0104)( 2 =−+≡ xxxf . Using the Intermediate

Value Theorem we can show that )(xf has roots in the intervals ]2,1[,]5,6[ −−

For our purposes we shall be interested in finding the root that lies in ]2,1[ .

The equation 01042 =−+ xx can be written in many different ways. The following are but a few alternate ways of expressing the equation.

2

2

104)(

10)4()(

410)(

xxiii

xxiixxi

−=

=+

−=


We solve each of these three equations for x and get

)(:4

10)(

)(:4

10)(

)(:410)(

3

2

2

1

xgx

xiii

xgx

xii

xgxxi

=−

=

=+

=

=−=

These are three possible fixed-point problems for the given root-finding problem given above. The question is which one of these fixed-point problems is most suited for solving the root-finding problem?

To answer this question we find which of the three fixed-point functions

)(),(),( 321 xgxgxg satisfies the criteria stated by the uniqueness theorem with

respect to the root in ]2,1[ . Using direct substitution we find that

50.1)2(,25.2)1(

67.1)2(,00.2)1(

41.1)2(,45.2)1(

33

22

11

==

==

==

gg

gg

gg

From the above results we conclude that, only )(2 xg satisfies the existence criteria.

Questions

Why do the other two functions fail the test?

Does )(2 xg satisfy the uniqueness criteria?

The Learner is expected to answer the first question by checking whether or not

)(,)( 31 xgxg meet the existence criterion for possessing a fixed point in ]2,1[ . To answer the second question we note that

28.0)2(,40.0)1(;)4(

10)( /

2/

22

/2 −=−=⇒⎥

⎦

⎤⎢⎣

⎡

+−= gg

xxg


Since 14.02

)2,1(<=

dxdg

Max we conclude that )(2 xg has a unique fixed

point )2,1(∈ρ which is automatically a root of the function )(xf .

Approximation of the root

The root ρ can be approximated iteratively using the iteration

nnx

xgxn

nn ...,,2,1,0;4

10)(1 =

+==+

for any starting value 0x taken

from the interval ).2,1( The following ten iterates were obtained using the starting

value 0.10 =x

nnx )(2 nxg

0 1.000 000 2.000 000

1 2.000 000 1.666 667

2 1.666 667 1.764 706

3 1.764 706 1.734 694

4 1.734 694 1.743 772

5 1.743 772 1.741 016

6 1.741 016 1.741 852

7 1.741 852 1.741 598

8 1.741 598 1.741 675

9 1.741 675 1.741 652

10 1.741 652 1.741659

We note that the last value 741659.1)( 102 =xg is very close to the true value of the

root of )(xf which is 741657.1=ρ rounded to 6 decimal places.

xn+1 = g2 (xn ) = 104 + xn


DO THIS (4.5)

(i) Show that the function )cos()( xxg = satisfies the conditions for the existence

of a unique fixed point inside the interval ]1,0[ .

(ii) Use the result obtained in part (a) to find in only ten iterations, the point of in-

tersection of the two curves )cos(, xyxy == . Start the iteration process

with 10 =x .

Reference for the Learning Activity

Kendall E. Atkinson, An Introduction to Numerical Analysis, - John Wiley & Sons, Second Edition (1989).



DO THIS (4.1)

Application of the bisection method in 5 iterations on )cos()( xxxf −=

459698.0)1(,1)0( =−= ff . Because )(xf is continuous over ]1,0[ and

0)1()0( <ff , then by the Intermediate Value Theorem for continuous functions it

follows that )(xf has at least one root in the interval ).1,0( The bisection method gives the following iterates:

00 =x 0)( 0 <xf

11 =x 0)( 1 >xf

5.02 =x 0)( 2 <xf

75.03 =x 0)( 3 >xf

625.04 =x 0)( 4 <xf

6875.05 =x 5( ) 0<f x

71875.06 =x 0)( 6 <xf

DO THIS (4.2)

Application of the secant method in 5 iterations on ).cos()( xxxf −=

Formula to be used: 11

111 −

−

−−+ ⎟⎟

⎠

⎞⎜⎜⎝

⎛

−

−−= n

nn

nnnn f

ffxx

xx

Starting values given: 459698.0,1;1,0 1010 =−=⇒== ffxx

One obtains the following iterates:


000000.0,739085.0

000683,0,738677.0

007979.0,743847.0

505751.0,410979.0

089300.0,685073.0

66

55

44

33

22

+==

−==

+==

−==

−==

fx

fx

fx

fx

fx

DO THIS (4.3)

Application of Newton-Raphson’s method in three iterations on )cos()( xxxf −=

Formula to be used is )(

)(/1

n

nnn xf

xfxx −=+ .

Starting value given 00 =x ; )sin(1)(/ xxf +=

One gets the iterates

000047.0)(,739113.0

681905.1)(,018923.0)(,750364.0

841471.1)(,459698.0)(,000000.1

000000.1)(,000000.1)(,000000.0

34

2/

22

1/

11

0/

00

+==

=+==

=+==

=−==

xfx

xfxfx

xfxfx

xfxfx

DO THIS (4.4)

ygxgyxyyxg

yfxfyxxyxf

yx

yx

21,2,),(

2,21,),(22

22

+=−=⇒+−=

−=−=⇒−−=

Starting values

4.0,8.0 00 == yx


First iteration

420339.0,772881.0 11 == yx

Second iteration

419644.0,771846.0 22 == yx

DO THIS (4.5)

Given: ]1,0[,)cos()( ∈= xxxg

Checking existence of a fixed point

)(xg is continuous over ]1,0[ .

]1,0[]1,0[)(;459698.0)1(,1)0( ∈∀∈⇒=−= xxggg

Therefore the existence criterion is fulfilled.

Checking uniqueness of a fixed point

)sin()(/ xxg −=

| g/ (x) |< 1 ∀x ∈[0,1] Therefore the uniqueness criterion is fulfilled.

Computations

)cos()(1 nnn xxgx ==+

Starting value: 0.10 =x The following iterates are obtained


744237.0)cos(

731404.0)cos(

750418.0)cos(

722102.0)cos(

763960.0)cos(

701369.0)cos(

793480.0)cos(

654290.0)cos(

857553.0)cos(

540302.0)cos(

910

89

78

67

56

45

34

23

12

01

==

==

==

==

==

==

==

==

==

==

xx

xx

xx

xx

xx

xx

xx

xx

xx

xx


XI. Compiled List of all Key Concepts (Glossary)

Key Concepts

Each of the (four) learning activities has key concepts, theorems and principles specific to its contents. However, to give the learner an opportunity to have a glimpse at what lies ahead in terms of the general framework and contents of the module we list here, a priori, all the key concepts, theorems and principles relevant to the module.

Reproduction of these concepts under their respective learning activities is meant to enhance the learner’s understanding and appreciation of their importance in the overall understanding of the course.

1. Error in an approximation

Let X be the exact (true) value of some quantity Q and *X be an approximate value

of Q obtained through some numerical process. The difference (deviation) between

the exact value X and its approximation *X is called the error in *X , and is denoted

by Error ( *X ) = *XX −

2. Absolute error

The error in an approximation may be positive or negative, depending on whether

one has underestimated or overestimated the true value of the quantity Q being ap-proximated. In practice, what matters more is the size of the error rather than its sign. In order to ignore the sign and concentrate on the size of the error, one introduces the

concept of absolute error, which is defined by: Absolute error in *X = *XX − .

3. Relative error

The absolute error gives the size of the error and thereby serves as a measure of the

accuracy of the approximation *X . However, by not relating the error to the true value being approximated one may not be able to gauge the seriousness of the error. To achieve this one introduces the concept of relative error or percentage error. This is

defined by: Relative error in ( *X ) = X

XX *− provided that 0≠X . Percentage


error in ( *X ) = X

XX *

100−

% provided that 0≠X .

Since the true value X is normally not known, one replaces it with the approximate

value *X in the denominator of the relative (percentage) error.

4. Arithmetic progression

An arithmetic progression (AP) is a special sequence { } ,...2,1,0: =nan of numbers.

The essential elements of such a sequence are its first element 0a and a constant dif-

ference d . Except for the first element, the thk − element of the sequence is given

as dkaak )1(0 −+= , ,...3,2,1=k

The sum of the first n terms of an AP is easily shown to be

Sn = na0 +12

n(n−1)

5. Geometric progression.

A geometric progression (GP) is a special sequence { } ,...2,1,0: =nan of numbers.

The essential elements of such a sequence are its first element 0a and a constant

multiplication factor 1≠r . Except for the first element, the thk − element of the

sequence is given by 10

−= kk raa , ,...3,2,1=k

The sum of the first n terms of a GP is ⎥⎦

⎤⎢⎣

⎡

−

−== −

=

∑1

10

1

10 r

raraS

nk

n

kn

6. Limit of a function

A function )(xf is said to approach the value (have the limit) L as x approaches

a value c in the domain of f , and we write limx→ c

f (x) = L if for every choice of a

small positive number ε one can find a corresponding small positive number )(εδ

such that whenever x − c < δ (ε) then f (x) − L < ε . Essentially this statement


implies that values of the function )(xf will be arbitrarily close to the number L provided the corresponding values of x are close enough to point c .

7. Continuity

A function )(xf is said to be continuous at a point cx = if the limit of the values

)(xf is the value )(cf , that is if limx→ c

f (x) = f (a) . Essentially what this definition

of continuity implies is that, for )(xf to be continuous at point cx = the following three conditions must hold:

(i) The function must be defined at the point: )(cf must exist.

(ii) The function must have a limit at the point:limx→ c

f (x) = L

(iii) The limit must be equal to the function value: Lcf =)(

8. Derivative

The derivative of a function )(xf at an arbitrary point x in its domain is the limit

of the difference quotient x

xfxxfxf

Δ

−Δ+=

Δ

Δ )()( as the change xΔ in x tends

to zero, that is ⎥⎦

⎤⎢⎣

⎡Δ

−Δ+→Δ x

xfxxfx

)()(lim

0 . If the limit exists one denotes it by

dx

df or )(/ xf and calls it the first derivative of f .

9. Anti-derivative

An anti-derivative of a function )(xf is itself a function ).(xF The function

F has the property that its derivative must be the function )(xf , meaning that

)()(/ xfxF = .

Essentially, the process of finding an anti-derivative of )(xf is the reverse process to that of differentiating a function. For this reason, one also speaks of anti-differen-

tiation when finding )(xF . Symbolically one writes ∫= dxxfxF )()( .


10. Convergence of a sequence

A sequence of numbers, written { }na is a well defined and ordered finite or infinite set of numbers. The individual members (elements) of the set must be unambiguously defined. Essentially a set is a special function defined over the set of natural (counting)

numbers N or over a subset of N . A set { }na is said to converge to a limit L if

for every 0>ε one can find a corresponding positive integer number )(εN such

that ε<− La n if only )(εNn > .

11. Fixed point of a function

Given a continuous function )(xg over a closed interval ],[ ba any value ρ=x

which satisfies the relation ρρ =)(g is called a fixed point of the function )(xg. The process of finding fixed points of a function is closely related to that of deter-mining roots of functions.

12. Secant line

A secant is a line which passes through two given points ),( 11 yxP and ),( 22 yxQ

and is given by the equation )( 112

121 xx

xxyy

yy −−

−−= . The secant equation is

used in deriving the Regula Falsi and the Secant methods for approximating roots of functions.


Key Theorems and Principles

1. Intermediate Value Theorem

If f is continuous on a closed interval ],[ ba and if k is any number lying between

the two values )(af and )(bf , then there is at least one number c in ),( ba such

that .)( kcf = This theorem is the basis upon which two important numerical

methods for finding roots of a function )(xf , the bisection method and the Regula Falsi (Rule of false position).

2. Mean Value Theorem of Differentiation (MVT)

If f is a continuous function on the closed interval ],[ ba and is differentiable on

the open interval ),( ba , then there exists at least one value of ),( bacx ∈= such

that ab

afbfcf

−

−=

)()()(/ .

3. Taylor’s Theorem

Let f be a function whose first n derivatives are continuous on the closed interval

],[ hcc + (or ],[ chc + if h is negative) and assume that )1( +nf exists in ],[ hcc +

(or ],[ chc + if h is negative). Then, there exists a number ,ϑ with 10 << ϑ such that

),()(!

)(1

)(

0

ϑhcfkh

hcf R nk

n

k

k

+=

+=+ ∑ , )()!1(

),( )1(1

1hcf

nh

c nn

nR ϑϑ ++

= ++

+

4. Fundamental Theorem of Calculus

If f is continuous on [ , ]a b and ( ) ( )x

aF x f t dt= ∫ for each x in [ , ]a b , then ( )F x

is continuous on [ , ]a b and differentiable on ( , )a b and


( )dF

f xdx

= . In other words, ( )F x is an anti-derivative of ( )f x .

Corollary: If ( )f x is continuous on a closed and bounded interval[ , ]a b , and

' ( )g x = ( )f x , then

( ) ( ) ( )b

af x dx g b g a= −∫ .

5. Fixed Point Theorem

If the function )(xg is continuous over bxa ≤≤ and ],[)( baxg ∈ ],[ bax ∈∀ then there exists at least one fixed point ],[ bax ∈= ρ such that ρρ =)(g .

If, in addition to the above assumptions, )(xg is differentiable in bxa << and

1<≤ Ldxdg

for all ),( bax ∈ , then )(xg has a unique (only one) fixed point

).,( ba∈ρ


XII. Compiled List Of Compulsory Readings

Wikipedia: Numerical Analysis

Wikipedia: Interpolation

Fundamental Numerical Methods and Data Analysis, George W. Collins, II, chapter 4. (see: http://bifrost.cwru.edu/personal/collins/numbk/)

Wikipedia: Numerical Methods/Numerical Integration

Wikipedia: Numerical Methods/Equation Solving


XIII. Compiled List of (Optional) Multimedia Resources

Reading # 1: Wolfram MathWorld (visited 03.11.06)

Complete reference : http://mathworld.wolfram.comAbstract : Wolfram MathWorld is a specialised on-line mathematical encyclopedia. Rationale: It provides the most detailed references to any mathematical topic. Stu-dents should start by using the search facility for the module title. This will find a major article. At any point students should search for key words that they need to understand. The entry should be studied carefully and thoroughly.

Reading # 2: Wikipedia (visited 03.11.06)

Complete reference : http://en.wikipedia.org/wikiAbstract : Wikipedia is an on-line encyclopedia. It is written by its own readers. It is extremely up-to-date as entries are contunally revised. Also, it has proved to be extremely accurate. The mathematics entries are very detailed.Rationale: Students should use wikipedia in the same way as MathWorld. However, the entries may be shorter and a little easier to use in the first instance. Thy will, however, not be so detailed.

Reading # 3: MacTutor History of Mathematics (visited 03.11.06)

Complete reference :http://www-history.mcs.standrews.ac.uk/IndexesAbstract : The MacTutor Archive is the most comprehensive history of mathematics on the internet. The resources are organsied by historical characters and by historical themes.Rationale: Students should search the MacTutor archive for key words in the topics they are studying (or by the module title itself). It is important to get an overview of where the mathematics being studied fits in to the hostory of mathematics. When the student completes the course and is teaching high school mathematics, the cha-racters in the history of mathematics will bring the subject to life for their students. Particularly, the role of women in the history of mathematics should be studied to help students understand the difficulties women have faced while still making an important contribution.. Equally, the role of the African continent should be studied to share with students in schools: notably the earliest number counting devices (e.g. the Ishango bone) and the role of Egyptian mathematics should be studied.


XIV. Synthesis of the ModuleCompletion of the fourth learning activity marks the end of this module. At this juncture, and before presenting oneself for the summative evaluation, it is desirable and appropriate that the learner reflects back on the mission, objectives, activities and attempts to form a global picture of what he/she is expected to have achieved as a result of the collective time and efforts invested in the learning process.

As laid out in the module overview in Section 6, the learner is expected to have acquired knowledge of the basic concepts relating to numerical approximation of numbers and functions.

The learner is now expected to be able to comfortably define the concept of an er-ror in mathematics, identify the sources of errors, the different types of errors and methods of reducing the impact of errors on our final numerical approximation to a mathematical problem.

After the opening learning activity on errors three substantive learning activities on the core issue of approximation were presented. One of these three core learning ac-tivity was on numerical approximation of functions using interpolation polynomials. Specifically the learner is now expected to be reasonably comfortable in using linear interpolation in its various forms and derive an error bound for linear interpolation. The activity also presents the important topic of interpolation polynomials, concentrating on Lagrangean, Newton’s divided differences and the all important finite difference interpolation polynomials.

The remaining two core learning activities (the third and fourth) are on numerical methods for approximating numbers (roots of functions and values of definite inte-grals). In this regard the learner is expected to have appreciated the need for resorting to numerical methods, but also be able to apply the methods learnt, including the Bisection, Regula-Falsi, Secant and the Newton-Raphson methods for approximating roots of a non-linear function of a single variable. One is also expected to solve a evaluate limits of functions, sequences and infinite series. One is also expected to be able to apply Newton’s method to approximate solutions of a couples system of two nonlinear equations in two variables.

The module has been structured with a view to escorting and guiding the learner through the material with carefully selected examples and references to the core references. A reasonable number of worked out examples have been included in every learning activity to serve as beacons of reference both in understanding the text before them as well as serving as points of reference while solving related formative evaluation problems that appear in the text in the form of DO THIS. The degree of mastery of the module contents will largely depend on the learner’s deliberate and planned efforts to monitor his/her progress by solving the DO THIS problems.


XV. Summative Evaluation

Questions

1. (a) Define the concept of an error in mathematics.

(b) Give the main types of mathematical errors and mention their sources.

(c) Distinguish between absolute and relative errors and attempt to relate them to the concepts of accuracy and precision.

2. (a) The error )(xE in linear interpolation of a function )(xf using

the values )(,)( 1100 xffxff == where hxx =− 01 is given by

)(2

1))(()( //

10 ξfxxxxxE −−= , where ),( 10 xx∈ξ . Derive an upper bound for E .

(b) Use the error bound obtained in part (a) to determine the smallest interval

size h for which linear interpolation of )cos(x will give approximate values

with errors not exceeding 01.0 .

3. The following table gives values of a function )(xf at a number of equally

spaced points kx .

x 0.0 0.125 0.25 0.375 0.5

f(x) 1.000000 0.984615 0.941176 0.876712 0.800000

x 0.625 0.75 0.875 1.0

f(x) 0.719101 0.640000 0.566372 0.500000

Approximate )(xf by a quadratic Lagrangean interpolation polynomial using the

points )876712.0,375.0(,)941176.0,25.0(,)984615.0,125.0( and hence,

interpolate )(xf at 3.0=x .


4. (a) Construct a table of finite differences for the function )(xf tabulated in Question 3 (a).

(b) With the aid of the table of finite differences interpolate the function at 4.0=x

using Bessel’s interpolation formula centered at 375.00 =x

...)2)(1(

24

1

)2

1)(1(

6

1)1(

2

1)

2

1()(

2

142

2

13

2

12

2

1

2

1

+−−=

+−−+−+−+=

fsss

fsssfssfsfsf

μδ

δμδδμ

where )(1

0xxh

s −=

5. Apply Newton’s interpolation based on divided differences to interpolate the

function tabulated in Question 4 at the point 4.0=x

6. (a ) Derive a Newton-Raphson iteration formula for approximating the thr root of a positive real number A .

(b) If 7=A apply the formula you have derived to approximate the square root

of 7 correct to six decimals. Start the iteration with .20 =x

7. (a) Define the concept of a fixed point for a function )(xg which is known to be

continuous over ],[ ba and differentiable in ).,( ba

(b) State without proof, the existence and uniqueness theorem for fixed points

of ).(xg

(c) Discuss whether or not the function g(x) = 4

6 − x has a unique fixed

point in the interval ]1,0[ .


8. (a) Using an analytical method calculate all the solutions of the coupled system

22,52 2222 =−=+ yxyx correct to 6 decimal places.

(b) Starting with 5.0,5.1 00 == yx approximate one of the solutions by carrying out two iterations with Newton’s method for systems of nonlinear equations.

9. (a) Find the anti-derivative )(xF of the function 23

)(2 ++

=xx

xxf and use

it to evaluate the integral dxxx

x∫ ++

1

0 2 23 correct to 6 decimals.

(b) Evaluate the above integral using the Newton-Cotes formula

[ ])()(3)(3)(8

3)( 3210

3

0

xfxfxfxfh

dxxfx

x+++=∫

with

6

1=h .

10. (a) Evaluate the integral ( )dxxex∫ −1

0)cos( analytically correct to 6 decimal

places accuracy.

(b) Tabulate values of the integrand in (a) at the points kxk 125.0=

8,...2,1,0=k and use the values obtained with the Trapezoidal rule to

approximate the integral, taking 25.0=h and 125.0=h , respectively.

(c ) Apply Romberg integration on the two Trapezoidal values obtained in (b) to obtain a better approximation of the integral in (a).


SOLUTIONS

1. (a) If *X is an approximation to an exact (true) quantity X , then the deviation *XX − is called the error in the approximation *X .

(b) The main types of mathematical errors are:

• Initial errors. These are errors in the initial data supplied with a mathematical problem.

• Discretization errors. These are errors caused by the process of converting a mathematical problem having a continuous solution to a numerical model whose solution is a discrete function in the form of a sequence of numbers.

• Truncation errors. These are errors introduced through unavoidable termina-tion (truncation) of an otherwise infinite process such an infinite series or a convergent iteration.

• Rounding errors. These are errors introduced because of limitations on the part of the instruments we use in performing arithmetic operations (addition, subtraction, multiplication or division).

(c) The absolute error is the magnitude or size of an error. The absolute error, denoted

by *XX − is always non-negative.

The relative error is the ratio between the absolute error and the absolute value of the true (exact) quantity being approximated. The relative error is denoted by

X

XX *−.

The absolute error is a measure of the accuracy in the approximation, while the relative error is a measure of the precision and relates to the seriousness of the error.

2. (a) To obtain a bound on the error )(xE in linear interpolation one needs to

find the maximum value of the functions ))(()( 10 xxxxxg −−= and )(// ξf

over the interval 10 xxx << . With )(2)( 00/ xxxxg +−= we find that )(xg

has a critical point at )(2

110

* xxx += , and because 2)(// =xg is always

positive we conclude that 4

)(4

1)(

22

01* h

xxxg −=−−= Assuming that


MxfMax =)(// over the interval in question we can then write

[ ]8

)()(2 Mh

xgMaxMxE =≤

(b) The function being approximated is ).cos()( xxf = In this case, )cos()(// xxf −=

and in this case 1=M . We now must determine the step length h such that

01.088

22

≤=hMh

. This gives the result 3.0≤h

3. The quadratic Lagrangean interpolation polynomial passing through three points

),(),,(),,( 221100 fxfxfx is given by the expression

2211002 )()()()( fxLfxLfxLxP ++= where )(),(),( 210 xLxLxL are theLa-grangean coefficients of degree 2. In this case we have the following values:

))((

))(()(,876712.0,375.0

))((

))(()(,941176.0,250.0

))((

))(()(,984615.0,125.0

1202

10223

2101

20111

2010

21000

xxxxxxxx

xLfx

xxxxxxxx

xLfx

xxxxxxxx

xLfx

−−

−−===

−−

−−===

−−

−−===

With 3.0=x , direct substitution of the various values of involved into the expressions for the Lagrangean coefficients gives:

28.0)3.0(,84.0)3.0(,12.0)3.0( 210 ==−= LLL , results which lead to the solution

917913.0)876712.0)(28.0()941176.0)(84.0()984615.0)(12.0(

)3.0()3.0()3.0()3.0( 2211002

=++−=

++= fLfLfLP


4. (a) Table of Differences

Differences

x f(x) First Second Third Fourth

0.000 1.000000

-0.015385

0.125 0.984615 -0.028054

-0.043439 0.007029

0.250 0.941176 -0.021025 0.001748

-0.064464 0.008777

0.375 0.876712 -0.012248 -0.000716

-0.076712 0.008061

0.500 0.800000 -0.004187 -0.005672

-0.080899 0.002389

0.625 0.719101 -0.001798 0.001286

-0.079101 0.003675

0.750 0.640000 -0.005473 -0.005458

-0.073628 -0.001783

0.875 0.566372 -0.007256

-0.066372

1.000 0.500000

(b) Since 0 0.375, 0.4, 0.125= = =x x h , then value of the parameter

1

(0.4 0.375) 0.20.125

= − =s


With the aid of the table of finite differences interpolate the function at 4.0=x using Bessel’s interpolation formula:

0000459936.0)006388.0(2

1)0144.0()2)(1(

24

1

000064488.0)008061.0)(008.0()2

1)(1(

6

1

0006574.0)012248.0004187.0(2

1)08.00()1(

2

1

0230136.0)076712.0)(3.0()2

1(

838356.0)(2

1

2

142

2

13

2

12

2

1

01

2

1

==−−

==−−

=−−−=−

=−−=−

=+=

fsss

fsss

fss

fs

fff

μδ

δ

μδ

δ

μ

Addition of all these terms leads to the answer 862137.0)4.0( =P .

5. First one constructs the table of Newton’s divided differences. Then one applies the interpolation polynomial

...],,[))((],[)()( 210101000 +−−+−+= xxxfxxxxxxfxxfxPn

Table of Newton’s Divided Differences

Divided Differencesx f(x) First Second Third Fourth0.0 1.000000

-0.123080.125 0.984615 -0.897728

-0.347512 0.5998080.25 0.941176 -0.672800 0.298326

-0.515712 0.7489710.375 0.876712 -0.391936 -0.122198

-0.613696 0.6878720.5 0.800000 -0.133984 -0.354304

-0.647192 0.5107200.625 0.719101 0.057536 -0.394240

-0.632808 0.0.3136000.75 0.640000 0.175136 -1.251670

-0.589024 -0.3122350.875 0.566372 0.058048

-0.5309761.0 0.500000


Direct application of the formula based on 375.00 =x and using the values of va-

rious divided differences highlighted in the table together with the value 4.0=x gives the following results:

000077616.0],,,,[))()()((

00028728.0],,,[))()((

00033496.0],,[))((

0153424.0],[)(

876712.0)(

432103210

3210210

21010

100

=−−−−

=−−−

=−−

−=−

=

xxxxxfxxxxxxxx

xxxxfxxxxxx

xxxfxxxx

xxfxxxf

Therefore

=+++−≈ 000078.0000287.0000335.0015342.0876712.0)4.0(4P , 0.862070

6. (a) If r Ax = then, 0=− Axr .

One may therefore take the function whose roots are being sought to

be Axxf r −=)( .

The derivative of )(xf is 1/ )( −= rrxxf .

Substituting these quantities into the Newton-Raphson formula

)(

)(/1

n

nnn xf

xfxx −=+

we obtain the general Newton-Raphson formula

⎥⎥⎦

⎤

⎢⎢⎣

⎡+−=

⎥⎥⎦

⎤

⎢⎢⎣

⎡ +−=

+−=

−−=

−

−−−+

1

1111

)1(1

)1(1

rn

n

rn

rn

rn

rn

rn

rn

rn

nn

x

Axr

r

x

Axrrrx

Axrx

rx

Axxx


(b) The Newton-Raphson formula for finding the square root of a number 7=A and 2=r . This gives the special formula

⎥⎦

⎤⎢⎣

⎡+=+

n

nn xxx

7

2

11 .

Taking 0.20 =x the following iterates are obtained from the formula:

6457513.2

6457513.2

6457520.2

6477273.2

75.2

0.2

6

5

4

3

1

0

=

=

=

=

=

=

x

x

x

x

x

x

Since the 7th decimal digit remains constant in the last two iterates we accept

645751.2=x as the desired approximation of 7 correct to 6 decimals.

7. (a) A number ρ is said to be a fixed point of a function )(xg if ρρ =)(g .

(b) If )(xg is a function which satisfies the conditions

• )(xg is continuous over a closed interval ],[ ba

• ],[],[)( baxbaxg ∈∀∈

• )(xg is differentiable in ),( ba

• ),(1)(/ baxLxgMax ∈∀<=

Then )(xg has a unique fixed point ),( ba∈ρ .

(c) Consider the function g(x) = 4

6 − x over the interval ]1,0[ .

Since the only point of discontinuity for the function is 6=x , the function is

definitely continuous over the interval ]1,0[ .


With both 6

4)0( =g and

5

4)1( =g lying in )1,0( we also note that all the function

values lie inside the given interval.

The function is also differentiable and g(x) = −4

(6 − x)2 . The maximum value of .

g/ (x) = g/ (0) =436

=19< 1

This result proves that the function has a unique fixed

point in the interval 10 << x .

8. (a) Analytical Solution

To obtain the true (analytical) solution one eliminates one of the variables and solves the resulting quadratic equation in the retained variable.

From the first equation one finds

22 25 xy −=

We use this result to eliminate y from the second equation.

⎪⎩

⎪⎨

⎧

−=

+=

==⇒=⇒=−−

549193.1

549183.1

4.21252)25(2

2

1222

x

xxxxx

The corresponding values of y is obtained from the relation

⎪⎩

⎪⎨

⎧

−=

+=

==−=

447213.0

447213.0

2.025

2

12

y

yxy

The four possible solutions to the coupled system are

)447213.0,549183.1(,)447213.0,549163.1(

)447213.0,549183.1(,)447213.0,549183.1(

−

−−−


(b) Approximation of the roots using Newton’s method

Let

022),(

052),(22

22

=−−≡

=−+≡

yxyxg

yxyxf

The partial derivatives of ),( yxf and ),( yxg are

ygyg

xgxg

yfyf

xfxf

yx

yx

4;2

2;4

−==∂

∂==

∂

∂

==∂

∂==

∂

∂

Newton’s method states t that, starting from an approximate solution ),( nn yx an

improved approximation ),( 11 ++ nn yx can be obtained by setting

nnnnnn kyyhxx +=+= ++ 11 , .

The increments nn kh , are obtained by solving the linear system of equations

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

−

−=

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

⎟⎟⎟

⎠

⎞

⎜⎜⎜

⎝

⎛

g

f

k

h

gf

gf

n

n

yy

xx

Application of Newton’s method:

First Iteration

0.2,0.3,25.0

0.1,0.6,25.0

5.0,5.1

0

0

00

−==−=

==−=

==

yx

yx

ggg

fff

yx


Substituting these quantities into the linear system and solving the resulting system

one gets 083333.0,083333.0 00 −== kh and therefore

416667.0,583333.1 001001 =+==+= kyyhxx

Second Iteration

666668.1,166666.3,.159721.0

833334.0,333333.6,.187498.0

416667.0,583333.1

1

1

11

−===

===

==

yx

yx

ggg

fff

yx

Substituting these quantities into the linear system and solving the resulting system

one gets 031667.0,033772.0 11 =−= kh and therefore

448334.0,549561.1 112112 =+==+= kyyhxx

9. (a) The anti-derivative of the function 23

)(2 ++

=xx

xxf

tConsxx

xx

dxxx

xx

xxdx

xxxxd

xxdx

dxxx

xdx

xxx

xxxdx

tan2

1ln

2

3)23ln(

2

1

2

1

1

1

2

3)23ln(

2

1

)2)(1(2

3

23

)23(

2

1232

3

23

32

2

1

23

332

2

1

23

2

2

2

2

2222

+⎟⎠

⎞⎜⎝

⎛+

+−++=

⎟⎠

⎞⎜⎝

⎛+

−+

−++=

++−

++

++=

++−

++

+=

++

−+=

++

∫

∫∫

∫∫∫∫

Therefore, dxxx

x∫ ++

1

0 2 23 =

[ ] 117783.0431523.0549306.02

1ln

3

2ln

2

3)2ln()6ln(

2

1=−=⎥⎦

⎤⎢⎣

⎡ −−−


(b) To approximate the same integral dxxx

xI ∫ ++=

1

0 2 23 using the special

Newton-Cotes formula [ ])()(3)(3)(8

3)( 3210

3

0

xfxfxfxfh

dxxfx

x+++=∫

One first tabulates the function:

kx 00 =x

6

11 =x

6

22 =x

6

33 =x

6

44 =x 6

55 =x 0.16 =x

kf 0.0 065934.0 107143.0 133333.0 15.0 160428.0 166667.0

While the given formula involves values of the function over three consecutive inter-

vals, the specified interval length 6

1=h results in a pair of such consecutive intervals,

],,,[ 3210 xxxx and ],,,[ 6543 xxxx . Therefore, the integral will be approximated by the expression

( ) ( ){ } 117741.033338

365433210 =+++++++= ffffffff

hI

10. (a) The value of the integral ( )dxxex∫ −1

0)cos( is

[ ] [ ] 876811.0)0sin()1sin( 01 =−−− ee

(b) Table of values of the integrand )cos()( xexf x −= for values at the

equidistant points: 0.1)125.0(0.0=x :

)177980.2,0.1(,)757878.1,875.0(,)385311.1,75.0(

)057283.1,625.0(,)771139.0,5.0(

)524484.0,375.0(),315113.0,25.0(,)140951.0,125.0(,)0.0,0.0(


With the aid of the extended trapezoidal rule one finds:

880144.0)125.0(,890138.0)25.0( == TT

(c) Application of Romberg integration on the two Trapezoidal values obtained above gives

[ ] 876813.0)25.0()125.0(3

1)125.0(]125.0,25.0[ =−+= TTTR


XVI. References

G. Stephenson, 1973, Mathematical Methods for Science Students – Second Edition, Pearson Education Ltd.

Brian Bradie, A friendly Introduction to Numerical Analysis, Pearson Education International.

A.C. Bajpai, I.M. Calus and J.A. Fairley, 1975, Numerical Methods for Engineers and Scientists, Taylor & Francis Ltd., London.

Burden and Faires, 1985, Numerical Analysis – Fifth Edition, PWS – KENT Pu-blishing Company.

Fox, L. and Mayers, D.F., 1958, Computing Methods for Scientists and Engineers, - Oxford University Press, London.

Kendall E. Atkinson, 1987, An Introduction to Numerical Analysis, John Wiley & Sons.

M.K. Jain, S.R.K. Iyengar & R.K. Jain, 1993, Numerical Methods – Problems and Solutions, Wiley Eastern Ltd.

Walter Jennings, 1964. First Course in Numerical Methods, Collier – Macmillan Ltd.

Lay, 2002, Linear Algebra and its Applications – Third Edition, Addison Wesley.


XVII. Author of the Module

Dr. Ralph W.P. Masenge is Professor of Mathematics in the Faculty of Science, Technology and Environmental Studies at the Open University of Tanzania. A graduate in Mathematics, Physics and Astronomy from the Bavarian University of Wuerzburg in the Federal Republic of Germany in 1968, Professor Masenge obtained a Masters degree in Mathematics from Oxford University in the United Kingdom in 1972 and a Ph. D in Mathematics in 1986 from the University of Dar Es Salaam in Tanzania through a sandwich program carried out at the Catholic University of Nijmegen in the Netherlands.

For almost 33 years (May 1968 – October 2000) Professor Masenge was an acade-mic member of Staff in the Department of Mathematics at the University of Dar Es Salaam during which time he climbed up the academic ladder from Tutorial Fellow in 1968 to full Professor in 1990. From 1976 to 1982, Professor Masenge headed the Department of Mathematics at the University of Dar Es Salaam. He also served as Associate Dean in the Faculty of Science and was Acting Chief Administrative Officer of the University of Dar e salaam for one year (1995).

Born in 1940 at Maharo Village situated on the slopes of Africa’s tallest mountain, The Killimanjaro, Professor Masenge retired in 2000 and left the services of the University of Dar Es Salaam to join the Open University of Tanzania where he now heads the Directorate of Research and Postgraduate Studies as well as being in charge of a number of Mathematics Courses in Calculus, Mathematical Logic and Numerical Analysis.

Professor Masenge is married and has four children. His main hobby is working on his small banana and coconut farm at Mlalakuwa Village, situated some 13 km from the City Centre on the outskirts of Dar Es Salaam, the commercial capital of the United Republic of Tanzania.

Documents

Methods - MacHighwayfrink.machighway.com/~dynamicm/Numerical-Methods.pdf · In all such cases, numerical methods have to be resorted to. The mathematics or science student is therefore