1

Newton’s Method

1

Sold - Christie’s Lot 269 Sale 2013

2

NEWTON, Sir Isaac. Arithmetica Universalis; sive de Compositione et Resolutione Arithmetica Liber. Ciu accessit Helleiana Aequationum Radices Arithmetice Inveniendi Methodus. Edited by William Whiston. Cambridge: Typis Academicus; London: Benjamin Tooke, 1707.

Actually written by Joseph Raphson

Remarks

One of the common problems in mathematics is to solve equations and find zeros

We have methods to solve simple equations like quadratics, but must resort to numerical solutions to solve more complicated equations like sin(x) +ex = x

3

2

RemarksA common numerical approach is bisection which can be applied on a closed interval

If we know f(a) < 0 and f(b) > 0, we know there is a number c where a < c < b and f(c) = 0 by the Intermediate Value Theorem

We can then test f(x) at the midpoint (a + b)/2 and if greater than zero we set a = midpoint and if less than zero we set b = midpoint

We continue this process until we achieve the desired degree of accuracy

4

Remarks

The disadvantage of the bisection method is that it converges slowly

The advantage of the bisection method is that it the function is continuous we expect it to converge, because we are promised there is a zero by the Intermediate Value Theorem

The bisection method is less efficient than Newton's Method but it is less prone to odd behavior

5

Remarks

The error when using the bisection method is after n iterations

This section is about Newton’s Method which can converge quickly even though it does not always work

12n

b a

6

3

Remarks

►Around 1669, Isaac Newton (1643-1727) gave a new algorithm to solve a polynomial equations

►He applied his method to polynomial equations, viewed his method as purely algebraic and apparently did not notice the connection with calculus

►In 1690, Joseph Raphson (1678-1715) proposed a method which avoided the substitutions used by Newton

7

Newton probably

derived his method from a similar but less precise method by Francois Viète

8

Heron of Alexandria essentially used the same method to determine the square root of 72

For polynomials Newton's method is essentially the same as Heron's method

9

4

Joseph RaphsonBorn in Middlesex in 1648, very little is known about his life, not even an obituary has been found

By the records of the University of Cambridge, all that is known that he attended Jesus College and graduated with a Master of Arts degree in 1692

Raphson used ideas of the Calculus to generalize this ancient method to find the zeros of an arbitrary equation ( ) 0f x

Their underlying idea is the approximation of the graph of the function f (x) by the tangent lines

10

Joseph Raphson

• Had an important relationship to Newton, although it is not well understood by historians

• Was one of the very few people of whom Newton would allow to see his papers

• Wrote up some of Newton's work into books

• As well as his work on calculus and analysis, he also had ideas on space and philosophy, based on Cabala, which is a Jewish Mysticism

11

Joseph Raphson

Raphson's ideas of space and philosophy were based on Cabalist ideas

Cabala was a Jewish mysticism which was influential from the 12th century on

The doctrines included the withdrawal of the divine light, thereby creating primordial space, the sinking of luminous particles into matter and a "cosmic restoration" that is achieved by Jews through living a

mystical life

12

5

0f x

0x1x

1 0

0

1 0

to find zero, let 0

y f xm f x

x x

y

The Idea

We use the tangent line to estimate the zero13

We defined the linear approximation of at asf x x a

L x f a f a x a

Since we trying the find the zero we let 0L x

0

0 0 0 1

We let our first quess be and have

0 and let the next guess be

a x

f x f x x x x

0 0 1 0then 0 f x f x x x

01 1 0

0

( )solving for we have

( )

f xx x x

f x

Discussion

14

The Method

We are trying to find the zeroes of ( ) 0f x

We make a reasonable first guess called our seed x0

01 0

0

( )

( )

f xx x

f x

We obtain our second guess by

15

6

The Method

We repeat this procedure recursively using the formula

1

( )

( )

nn n

n

f xx x

f x

As our guess come closer and closer to the root r, we

say

We then say it converges

lim nx

x r

16

Example

01 0

0

( ) 1.29Then 2.3 2.0196

( ) 4.6

f xx x

f x

We take the derivative 2f x x

0We begin with an intial guess Let 2.3x

2Find zero of 4f x x

2

We evaluate 2.3 2.3 4 1.29f

We evaluate 2.3 2 2.3 4.6f

We can find the next by repeating this process

until we achieve our desired accuracy

nx

17

Example 2Find zero of 4f x x

2

1

We can create a recrusive function to obtain the next guess

( ) 4

( ) 2

n nn n n

n n

f x xx x x

f x x

We can then find the next xn

18

7

Example 2Find zero of 4f x x

n xn Error

0 2.3000000000 1.29000000

1 2.0195652174 0.07878416

2 2.0000951079 0.00038044

3 2.0000000023 0.00000001

Our result after three iterations

19

Example 2Find zero of 4f x x

The third iteration using the TI-89

20

n=0

xn

xn+1 = xn – f(xn)/f’(xn)

|xn-xn+1|<En+1n

YesNo

Flowchart

21

8

Example2

2

We want to solve 5

We begin by creating a function which is

zero when 5

x

x

Find the square root of 5

2

We let LHS RHS

In our example,

we let 5

f x

f x x

22

2Let ( ) 5f x x ( ) 2f x x

0Take 2 (2) 1 and (2) 4x f f

1

1 92 2.25

4 4x

9 1 9 9 and

4 16 4 2f f

2

9 1 16 1612.236

4 9 2 72x

Example

23

Find the square root of 5

TI-84 Solution

Graph

x1

x2

or fromHome

Example

24

Find the square root of 5

9

Example 3Find roots of 1f x x x

The sketch of the graph will tell us that this polynomial has exactly one real root

We first need a good guess as to where it is

We see f(-2) < 0 and f (-1) > 0. By the IVT we know there is a zero in the interval (-2,-1)

We could choose x0 = -125

3 1f x x x We can take the derivative

2 3 1f x x

1

n

n

n n

f xx x

f x

To perform the iteration

3

2

1

3 1

n nn

n

x xx

x

Example 3Find roots of 1f x x x

26

x0 -1

x1 -1.500000

x2 -1.347826

x3 -1.325200

x4 -1.324718

x5 -1.324717

x6 -1.324717

x7 -1.324717

Notice the values become closer and closer to the same value.

We have found the approximate solution to six decimal places

obtained after only five relatively painless steps.

Example 3Find roots of 1f x x x

27

10

ExampleFind zero of 2 2 4 f x x x

using graphing calculator

With trigonometric functions, use nDeriv on the TI to force a numerical solution

28

Remarks

With trigonometric functions

• Set your calculator angle mode is in radians, notdegrees

• Use radians for you calculations, not degrees

• If using a calculator to assist with taking thederivative, numerical derivatives are recommended

29

Example Find a solution to cos 0x

Let cosy x

Then siny x

1n n

f xx x

f x

cos

sin

nn

n

xx

x

Define our function

Take the derivative

Find the integrative formula

Make an initial guess0 1x

cotn nx x

30

11

ExampleOur first guess of 1

cos(x) = 0

31

Our Tangent Line

Our second guess is x = 1.64209

Example cos(x) = 0

32

Example cos(x) = 0

And the Next Tangent Line

33

12

Our estimate is 1.5708

x x + cot x

1 1.64209

1.64209 1.57068

1.57068 1.5708

1.5708 1.5708

1.5708 1.5708

Example cos(x) = 0

34

Example sinEstimate zero of 0xe x

sinLet xf x e x

sin cos 1xf x e x

sin

1 sin cos 1

x

n n x

e xx x

e x

35

Example sinEstimate zero of 0xe x

0.5386

0Let us begin with a seed of 1x

sin 1

1 sin 1

11

cos 1 1

ex

e

36

13

Example sinEstimate zero of 0xe x

sin 0.5386

2 sin 0.5386

10.5386

cos 0.5386 1

ex

e

0.5783

37

sin 0.5783

3 sin 0.5783

10.5783

cos 0.5783 1

ex

e

0.5787

We then have 0.5787 as our

estimate of the zero

Example sinEstimate zero of 0xe x

38

Advantages

Applies to an equation of any degree

Applies to non-polynomial equations

Gives answers in numerical form

Continue computation until desired degree of accuracy

achieved

Converges quickly; we often need only two or iterations

to attain desired accuracy39

14

Remarks

Newton's method will usually converge provided the initial guess is close enough to the unknown zero

Furthermore, for a zero of multiplicity 1, the convergence is quadratic, which intuitively means that the number of correct digits roughly doubles in every step

If the root being sought has multiplicity greater than

one, the convergence rate is reduced to linear

40

Newton’s Method Fails If

1. If our first guess (or any guesses thereafter) is a point at which there is a horizontal tangent line, then this line will never hit the x-axis, and Newton's Method will fail to locate a root. If there is a horizontal tangent line then the derivative is zero, and we cannot divide by f '(x) as the formula requires.

2. If our guesses oscillate back and forth then Newton's method will not work.

41

Newton’s Method Fails If

3. If there are two roots, we must have a first guess near the root that we are interested in, otherwise Newton's method will find the wrong root.

4. If there are no roots, then Newton's method will fail to find it.

5. It may fail for piecewise functions

6. If f(x) = 0 and f'(x) = 0 have a common root

42

15

Example

0Let 1x

n xn

0 1

1 -2

2 4

3 -8

4 16

1 2n nx x

3Find zero of ( )f x x

1

1 3

2 32

1 3

n

n n

n

nn

n

f xx x

f x

xx x

x

We see our process does not converge43

Other Root Finding AlgorithmsBinary Search Method

Bairstow's Method

Brent's Method

Durand-Kerner Method

False Position

Inverse Quadratic Interpolation

Laguerre's Method

Lehmer-Schur Algorithm

Müller's Method

Principal nth Root

Ruffini's Rule

Secant Method

44

Secant Method

1

1

1

Then the Secant Method is

( )n n n

n n

n n

f x x xx x

f x f x

We do not have to take the derivative

1

( )From Newton's Method,

( )

nn n

n

f xx x

f x

1

1

but we have ( )n n

n

n n

f x f xf x

x x

45

16

Remarks

46

TI-83/84 and TI-89 program listings for Newton’s Methodhttp://www.tc3.edu/instruct/sbrown/ti83/newton.htm

Programs for the several calculators – download – or enter directly into calculatorhttp://math.arizona.edu/~krawczyk/calcul.html

Animations of Newton’s Methodhttp://www.ecs.fullerton.edu/~mathews/a2001/Animations/RootFinding/NewtonMethod/NewtonMethod.html

Fractal ImagesFractal images can be created by applying the Newton-Raphson Method to various real-valued polynomials with complex solutions

47http://www.youtube.com/watch?v=7RZn3B709wo

-x 3 + 9x 2 - 18x + 6 Example

48

17

-x5 + 25x4 - 200x3 + 600x2 - 600x + 120Example

49

x 5 - 1 Example

50

Remarks

51

There are many fractal programs available today

For example see fractalarts.com/ASF/download.html

18

52

http://www.youtube.com/watch?feature=player_embedded&v=qB8m85p7GsU

Arthur Clarke - Fractals The Colors Of Infinity 1 of 6