7/27/2019 Lecture05 Complete
1/12
Lecture05
Lecture 05: Rootfinding cont'dsolving non-linear equations f(x)=0
Outline
1) Classic Algorithms for solving f(x)=0
Bisection: review and finding a bracket
Newton's Method
Examples: f(x)=x-exp(-x)
Convergence of Newton's Method
Failure of Newton's Method
Secant Methods
2) Comparison of convergence
f(x)=x-exp(-x)
f(x)=A-mP/x*[(1+x/m)*mn-1]
3) Fail-safe hybrid methods
NewtSafe (Newton + Bisection: Numerical Recipes)
Brent's method (Secant+IQI+Bisection)
Matlab's fzero (Brent's method)
7/27/2019 Lecture05 Complete
2/12
Lecture05
Rootfinding: solve f(x)=0
f(r) = A - m*P/r [ (1-r/m)mn
-1] =0
Classical Methods:
Bisection (linear convergence)Newton's Method (Quadratic)Secant Method (super-linear)
Combined Methods
RootSafe (Newton + Bisection)Brent's method (fzero in matlab)
Bisection:
Given a bracket [a,b]:cut the interval in half and find the newbracket. Repeat until the bracket width is < tol. Given an initialbracket, bisection is guaranteed to converge.
7/27/2019 Lecture05 Complete
3/12
Lecture05
Finding a bracket:
1) Plot your function!2) Guess and Grow:
given x0, GROWTH_FACTOR:set dx
a=x0, b=x0, fa=f(a), fb=f(b)
while (not bracket(a,b))a=a-dxtestbracket(a,b)b=b+dx;testbracket(a,b)dx=(b-a)*GROWTH_FACTOR
end
Faster Methods: Newton's method
Points:
1) Given a bracket, Bisection is guaranteed to converge linearly to a root2) However: Bisection uses almost no information about f(x) beyond itssign3) Can we do better?
Newton's method: (Newton-Raphson)
Simple idea: given f(x) and f'(x), pretend f is locally linear and predictthe zero crossing:
Algebraically:
7/27/2019 Lecture05 Complete
4/12
Lecture05
Newton's method: the geometric picture
Newton's method:
Example f(x) = x-exp(-x)
7/27/2019 Lecture05 Complete
5/12
7/27/2019 Lecture05 Complete
6/12
Lecture05
Asymptotic Convergence of Newton's method:
2) multiple root e.g. f(x)=(x-1)2
(this is a mess...I need to clean these notes up)
Fragility and Failure of Newton's method:
Example: f(x)=sin(2x)
7/27/2019 Lecture05 Complete
7/12
Lecture05
Fragility and Failure of Newton's method:
Example: f(x)=sin(2x)
xk+1
Other Issues with Newton's method:
Need to supply both f(x) and f'(x)
Example: FTV equation f(r) = A - m*P/r*[(1+r/m)mn
-1]
(Cheat: symbolic differentiation, matlab symbolic toolbox (Maple),Mathematica)
syms A m P r nf = A-m*P/r*((1+r/m)^(m*n)-1)diff(f,r)
Point: would like rootfinding method with convergence behavior ofNewton but without derivatives
7/27/2019 Lecture05 Complete
8/12
Lecture05
Secant Methods:
Idea: replace f'(x) in Newton with finite difference approximation
Given xk
and xk-1
let f'(x) ~
or choose the next point as
Secant Methods:Geometric Picture
7/27/2019 Lecture05 Complete
9/12
Lecture05
Secant Methods:The Algorithm
Given f(x) and bracket [a,b]
initialize x1
=a, x2=b f
1=f(x
1), f
2=f(x
2)
Equivalence of Secant method and
linear interpolation
7/27/2019 Lecture05 Complete
10/12
Lecture05
Higher order Secant Methods:Inverse Quadratic Interpolation
Idea: Rather than use linear interpolation, use quadratic interpolation
Secant Methods: Comments
Similar to Newton's Method
1) Convergence is Superlinear (but not quite quadratic)
2) Secant methods are not guaranteed to converge
3) Secant methods do not preserve brackets
But with good guess can be almost as good as Newton's method withoutderivatives
7/27/2019 Lecture05 Complete
11/12
Lecture05
Hybrid Methods:
Design Goals:
1) Robustness: Given a bracket [a,b], maintain the bracket
2) Efficiency: Use superlinear convergent methods when possible
Some Options:
Have derivatives:NewtSafe (or RootSafe, Numerical Recipes)
Newton's method within a bracket, Bisection otherwise
No Derivatives:
Brent's Algorithm (zbrent Numerical Recipes, fzero Matlab)Returns minimum bracket using combination ofBisectionSecant methodInverse Quadratic Interpolation
NewtSafe Algorithm:
7/27/2019 Lecture05 Complete
12/12
Lecture05
NewtSafe Algorithm: