Order and Rates of Convergence - Boise State Universitymath.boisestate.edu/.../lectures/Order_Rate/Order_Rate.pdf · Order and Rates of Convergence 1 Saturday, September 14, 13

Numerical AnalysisMath 465/565

Order and Rates of Convergence

1

Saturday, September 14, 13

“Speed of convergence”

2

We now have two algorithms which we can compare - bisection and the fixed-point method. Which is faster?

Hard to answer : Depends on what interval we start with, how close to a root we start with, etc. So for any two particular instances one method might converge in fewer iterations than the other.

How can we qualify more generally which method might be better in some sense?


Review of bisection

3

We showed that bisection has the property that

Or, bisection ”behaves” like the sequence

�k = 2�k

If we define the error as ek = |xk � x̄|, we can write

ek |b� a| 2�k

|xk � x̄| |b� a|2k


How fast does an algorithm converge?

4

Suppose that an algorithm produces iterates that converge as

limk!1

xk = x̄

If there exists a sequence �k that converges to zero and

a positive constant C, such that

ek = |xk � x̄| C|�k|

then xk is said to converge with rate �k.


Convergence rate - Bisection

5

rate 2�k

|xk � x̄| |b� a| 12

k

So the bisection method has a convergence rate of

12k

with |b� a| as the asymptotic convergence constant.


Approximate rate of convergence

6

Using the Matlab command polyfit, we can get a best fit line to the data

ak + b = log(ek)

log(ek) = � log(2)k + log(C)



6


ak + b = log(ek)

>> k = 1:49;




6


ak + b = log(ek)

>> k = 1:49; >> ab = polyfit(k,log(e),1)




6


ak + b = log(ek)


ab =




6


ak + b = log(ek)


ab =

-0.692013993991203 0.046941635188405




6


ak + b = log(ek)


ab =

-0.692013993991203 0.046941635188405

>> log(2)




6


ak + b = log(ek)


ab =

-0.692013993991203 0.046941635188405

>> log(2)

ans =




6


ak + b = log(ek)


ab =

-0.692013993991203 0.046941635188405

>> log(2)

ans =

0.693147180559945



Fixed point method

7

Suppose over the interval [x0 � ⇢, x0 + ⇢], a function

g(x) satisfies a Lipschitz condition, and furthermore,

x0 is su�ciently close so that a sequence

xk+1 = g(xk), k = 0, 1, 2, . . .

converges to a root x̄ in [x0 � ⇢, x0 + ⇢]. Then

ek = |xk � x̄| �

k⇢

where |g0(x)| � < 1.


Convergence of fixed point iteration

8

So the convergence rate is �

kwhere |g0(x)| � < 1.


Estimated rate for fixed point

9

log(ek) = log(�)k + log (⇢)

Convergence rate

⇡ 1

20k



9

>> ab = polyfit(kvec,log(e),1)


Convergence rate

⇡ 1

20k



9


ab =


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))

ans =


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))

ans =


Convergence rate

⇡ 1

20k



9


ab =

-2.9148 0.3012

>> exp(ab(1))

ans =

0.0542

>> exp(ab(2))

ans =

1.3514


Convergence rate

⇡ 1

20k


Comparison of convergence rates

10

Bisection ⇡ 2

�k

Fixed point ⇡ �k

Which is better?


Order of convergence

11

Suppose we have that

Then the convergence of the sequence xk to x̄ is said

to be of order ↵.

limk!1

|xk+1 � x̄||xk � x̄|↵ = lim

k!1

ek+1

e

↵k

= µ


Estimating order of convergence

12

and use a best-fit-line approach to finding ↵, given a

sequence of errors ek.

Both bisection and the fixed point method are linear methods

Again, we can write

log(ek+1) = ↵ log(ek) + log(µ)



12



>> ab = polyfit(log(e(1:end-1)),log(e(2:end)),1)


Again, we can write




12




ab =


Again, we can write




12




ab =

0.9885 -0.8910


Again, we can write



Documents

Order and Rates of Convergence - Boise State Universitymath.boisestate.edu/.../lectures/Order_Rate/Order_Rate.pdf · Order and Rates of Convergence 1 Saturday, September 14, 13