Exercise # 7: Apply the modified false position method to y = x2 − 2 anddraw a complete flow diagram for this problem.

4.5 Newton-Raphson Method

One of the most widely used methods of solving equations is Newton-RaphsonMethod. The idea behind the method is starting from an initial estimate whichis not too far from a root, x1, we extrapolate along the tangent to its intersectionwith the x−axis, and take that as the next approximation. This is continued untileither the successive x−values are sufficiently close, or the value of the function issufficiently near zero.

Assume we have a preliminary estimate x1 of a root of

f(x) = 0

Assume f(x) differentiable in the region of interest.The line through (x2, 0) is the tangent to the curve at (x1, f(x1)). Thus from

the geometry we have

f ′(x1) = limx2→x1

f(x2) − f(x1)

x2 − x1

f(x2) = f(x1) + f ′(x1)(x2 − x1)

and the value where f(x2) = 0 is

x2 = x1 −f(x1)

f ′(x1)

We continue the calculation scheme by comupting

x3 = x2 −f(x2)

f ′(x2)

or, in more general terms,

xk+1 = xk −f(xk)

f ′(xk)

This formula provides a method of going from one guess xk to the next guessxk+1. This may also be viewed as an application of Taylor’s series, using only thefirst two terms.

Newton’s method when it works is fine. The method does not always converge;it may jump to another root or oscillate around the desired root. The three sketchesshow some of the troubles that can occur when the method is used carelessly.

Thus, in practice, unless the local structure of the function is well understood,Newton’s method is to be avoided.

63

¶

xk+1 = xk −f(xk)

f ′(xk)

= xk −f(xk)

f(xk+1)−f(xk)xk+1−xk

= xk −f(xk)(xk+1 − xk)

f(xk+1) − f(xk)

=xkf(xk+1) − xk+1f(xk)

f(xk+1) − f(xk)

∼ Secant method

∼ False-position method

¶ Newton’s method also works for complex roots. If we give it a complex valuefor the starting value.

¶ Newton’s method is widely used because; at least in the near neighborhoodof a root, it is more rapidly convergent than any of the methods so far discussed.This method is quadratically convergent, it tends on each step almost to double thenumber of decimal places that are accurate; thus, from two accurate figures we getin one step almost four figures accurate, in the next step almost eight figures, andso on. This is why, when it can be made to work, it is a good method. However,offsetting this is the need for two function evaluations at each step, f(xn) and f ′(xn).

Example # 3: Find a formula for Newton’s method for the function

y = xex − 1.

Solution:

y′ = ex + xex = (x + 1)ex

xk+1 = xk −(xke

xk − 1)

(xk + 1)exk

= xk −(xk − e−xk)

xk + 1

=(x2

k + e−xk)

xk + 1

Example # Apply Newton’s method to

f(x) = 3x + sin x − ex = 0.

64

If we begin with x1 = 0.0, we have:

x2 = x1 −f(x1)

f ′(x1)= 0.0 − −1.0

3.0= 0.33333;

x3 = x2 −f(x2)

f ′(x2)= 0.33333 − −0.068418

2.54934= 0.36017;

x4 = x3 −f(x3)

f ′(x3)= 0.36017 − −6.279 × 10−4

2.50226= 0.3604217.

After three iterations, the root is correct to seven significant digits. The rule ofdoubling the number of correct digits would indicate six correct digits in the lastresult because three are repeated from the previous one.

4.6 Muller’s Method

Most of the roots finding methods that we have considered so far have approximatedthe function in the neighborhood of the root by a straight line. Obviously, this isnever true; if the function were linear, finding the root would take practically noeffort.

Muller’s method is based on approximating the function in the neighborhood ofthe root by a quadratic polynomial. This gives a much closer match to the actualcurve.

A second degree polynomial is made to fit three points near a root,

(x◦, f(x◦)), (x1, f(x1)), (x2, f(x2)),

and the proper zero of this quadratic, using the quadratic formula, is used as theimproved estimate of the root. The process is then repeated using the three pointsnearest the root being evaluated.

The procedure for Muller’s method is developed by writing a quadratic equationthat fits through three points in the vicinity of the root, in the form

P2(v) = av2 + bv + c.

The development is simplified if we transform axes to pass through the middle point,by letting v = x − x◦.

Leth1 = x1 − x◦, h2 = x◦ − x2

We evaluate the coefficients by evaluating P2(v) at the three points

v = 0 ⇒ a(0)2 + b(0) + c = f◦ ⇒ c = f◦v = h1 ⇒ a(h1)

2 + b(h1) + c = f1 ⇒ ah21 + bh1 + c = f1

v = h2 ⇒ a(−h2)2 + b(−h2) + c = f2 ⇒ ah2

2 − bh2 + c = f2

65

ah21h2 + bh1h2 + ch2 = f1h2

ah22h1 − bh2h1 + ch1 = f2h1

}⇒ a(h2

1h2 + h1h22) + c(h1 + h2) = f1h2 + f2h1

⇒ a =f1h2 + f2h1 − f◦(h2 + h1)

h21h2 + h1h

22

Leth2

h1

= η, then

⇒ a =f1η + f2 − f◦(η + 1)

h21η(1 + η

Similarly, we can find

b =f1 − f◦ − ah2

1

h1

, c = f◦

Nowav2 + bv + c = 0 ⇒

v =−b ±

√b2 − 4ac

2a

x − x◦ =−b ±

√b2 − 4ac

2a

x = x◦ +−b ±

√b2 − 4ac

2aSince

−b ±√

b2 − 4ac

2a=

(−b ±

√b2 − 4ac

2a

)(+b ±

√b2 − 4ac

+b ±√

b2 − 4ac

)

=−2c

b ±√

b2 − 4ac

Therefore,

x = x◦ −2c

b ±√

b2 − 4ac

with the sign in the denominator taken to give the largest absolute value of thedenominator (i.e., if b > 0, choose plus; if b < 0, choose minus; if b = 0, chooseeither).

To take the root of the polynomial as one of the set of three points for the nextapproximation, taking three points that are most closely spaced (i.e., if the root isto the right of x◦, take x◦, x1, and the root; if to the left, take x◦, x2, and the root).Always reset the subscripts to make x◦ be the middle of the three values.

Example # Find a root between 0 and 1 of the transcendental function

f(x) = 3x + sin x − ex = 0.

66

Let

x◦ = 0.5, x1 = 1.0, x2 = 0.0then h1 = 0.5, h2 = 0.5, γ = 1.0,and f(x◦) = 0.330704, f(x1) = 1.123189, f(x2) = −1.0

a =(1.0)(1.123189)− 0.330704(2.0) + (−1)

1.0(0.5)2(2.0)= −1.07644,

b =1.123189− 0.330704 − (−1.07644)(0.5)2

0.5= 2.12319,

c = 0.330704,

root = 0.5 − 2(0.330704)

2.12319−√

(2.12319)2 − 4(−1.07644)(0.330704)

= 0.330704,

For the next iteration, we have

x◦ = 0.354914, x1 = 0.5, x2 = 0.0then h1 = 0.145086, h2 = 0.354914, γ = 2.44623,and f(x◦) = −0.0138066, f(x1) = 0.330704, f(x2) = −1.0

a =(2.44623)(0.330704)− (−0.0138066)(3.44623) + (−1)

2.44623(0.145086)2(3.44623)= −0.808314,

b =0.330704 − (−0.0138066) − (−0.808314)(0.145086)2

0.145086= 2.49180,

c = −0.0138066,

root = 0.354914− 2(−0.0138066)

2.49180 −√

(2.49180)2 − 4(−0.808314)(−0.0138066)

= 0.360465,

After third iteration, we get root = 0.3604217, which is identical to that from New-ton’s method after three iterations, but it does not require the evaluation of deriva-tives, however, needs only one function evaluation per iteration after obtaing thestarting values.

4.7 Fixed-Position Iteration

Fixed point iteration is a possible method for obtaining a root of the equation

f(x) = 0. (4.1)

67

In this method, we rearrange equation (??) of the form

x = g(x) (4.2)

so that any solution of (??) i.e., any fixed point of g(x), is a solution of (??). Thismay be accomplished in many ways. If, e.g.,

f(x) = x2 − x − 2 (4.3)

Then among the possible choices for g(x) are the following:-

1. g(x) = x2 − 2

2. g(x) =√

x + 2

3. g(x) = 1 +2

x

4. g(x) = x − x2 − x − 2

mfor some non-zero constant m.

Each such g(x) is called an iteration function for solving (??) (with f(x) givenby (??)). Once an iteration function g(x) for solving (??) is chosen, one carries outthe following algorithm.

AlgorithmGiven an iteration function g(x) and a starting point x◦.

Iterate for n = 1, 2, 3, · · · until satisfied.Calculate xn+1 = g(xn)For this algorithm to be useful, we must prove,

1. For the given starting point x◦, we can calculate successively x1, x2, x3, · · ·

2. The sequence x1, x2, x3, · · · converges to some point ξ.

3. The limit ξ is a fixed point of g(x) i.e., ξ = g(ξ).

Some iterative formulations may not converge, and if the equation has multiple roots,different iterate formulations may converge to different roots. Thus the definition ofthe conditions for convergence is of fundamental importance. Let x1, x2, x3, · · · offixed point iteration method all lying in I(interval), let Ean, is the error in the nth

iteration, thenen = ξ − xn n = 0, 1, 2, 3, · · ·

then since ξ = g(ξ) and xn = g(xn−1), we have

en = ξ − xn = g(ξ) − g(xn−1) = g′(ηn)en−1

68

for some ηn between ξ and xn−1 by the mean value theorem for derivatives. Hence

|en| ≤ k|en−1|

It follows from induction on n

|en| ≤ k|en−1| ≤ k2|en−2| ≤ k3|en−3| · · · ≤ kn|e◦|

since0 ≤ k < 1 lim

n→∞kn = 0 ⇒ lim

n→∞|en| = lim

n→∞kn|e◦| = 0

regardless of the initial error e◦. But this says that x1, x2, x3, · · · converges to ξ.It also proves that ξ is the only fixed point of g(x) in I. For if, also η is a fixed pointof g(x) in I, then with x◦ = η, we should have x1 = g(x◦) = η, hence

|e◦| = |e1| ≤ k|e◦|.

Since k < 1, then this implies |e◦| = 0 or ξ = η.

?Corollary If g(x) is continuously differentiable in some open interval contain-ing the fixed point ξ and if |g′(ξ)| < 1, then there exists an ε > 0 so that fixed pointiteration with g(x) converges whenever |x◦ − ξ| ≤ ε.

Example # 4:x2 − xex + c = 0.

Choices for g(x) are

1. g(x) = ex − c

x

2. g(x) =x2 + c

ex

3. g(x) = x2 + x(1 − ex) + c etc.

Example #Consider an example

f(x) = x2 − 2x − 3 = 0.

Roots of this equation are x = −1 and x = 3.Suppose we rearrange to give this equivalent form:

x = g1(x) =√

2x + 3

orxk+1 =

√2xk + 3

69

If we start with x◦ = 4 and iterate with the fixed-point algorithm, successive valuesof x are

x◦ = 4,

x1 =√

11 = 3.31662,

x2 =√

9.63325 = 3.10375,

x3 =√

9.20750 = 3.03439,

x4 =√

9.06877 = 3.01144,

and it appears that the values are converging on the root at x = 3.Other RearrangementsAnother rearrangement of f(x) is

x = g2(x) =3

(x − 2).

Let us start the ieration again with x◦ = 4. Successive values then are

x◦ = 4,

x1 = 1.5,

x2 = −6,

x3 = −0.375,

x4 = −1.263158,

x5 = −0.919355,

x6 = −1.02762,

x7 = −0.990876,

x8 = −1.00305,

and it seems that we now converge to the other root, at x = −1. We note that theconvergence is oscillatory rather than monotonic.

Consider a third rearrangement:

x = g3(x) =x2 − 3

2.

Starting again with x◦ = 4, we get

x◦ = 4,

x1 = 6.5,

x2 = 19.625,

x3 = 191.070,

70

and the iterates are obviously diverging.This difference in three arrangements is because, the fixed point of x = g(x) is

the intersection of the line y = x and the curve y = g(x) plotted against x.insert figObserve that with the successive iterates, the points on the curve converge to

a fixed point or else diverge. It appears that the different behaviors depend onwhether the slope of the curve is greater, less, or of opposite sign to the slope of theline (which equal +1).

71