Embed Size (px)
Citation preview
1
Lecture 2
Solution of Nonlinear Equations
( Root Finding Problems )
Definitions Classification of Methods
Analytical Solutions Graphical Methods Numerical Methods
Bracketing Methods Open Methods
Convergence Notations
2
Root Finding Problems
Many problems in Science and Engineering are expressed as:
0)(such that value thefind
,function continuous aGiven
rfr
f(x)
These problems are called root finding problems.
3
Roots of Equations
A number r that satisfies an equation is called a root of the equation.
2.tymultipliciwith (3)root repeated a and
2) and 1( roots simple twohas equation The
)1()3)(2(181573 i.e.,
.1,3,3,2:rootsfourhas
181573 :equation The
2234
234
xxxxxxx
and
xxxx
4
Zeros of a Function
Let f(x) be a real-valued function of a real variable. Any number r for which f(r)=0 is called a zero of the function.
Examples:
2 and 3 are zeros of the function f(x) =
(x-2)(x-3).
5
Graphical Interpretation of Zeros
The real zeros of a function f(x) are the values of x at which the graph of the function crosses (or touches) the x-axis. Real zeros of f(x)
f(x)
6
Simple Zeros
)2(1)( xxxf
)1at x one and 2at x (one zeros simple twohas
22)1()( 2
xxxxxf
7
Multiple Zeros
21)( xxf
1at x 2)y muliplicit with (zero zeros double has
121)( 22
xxxxf
8
Multiple Zeros3)( xxf
0at x 3y muliplicit with zero a has
)( 3
xxf
9
Facts
Any nth order polynomial has exactly n zeros (counting real and complex zeros with their multiplicities).
Any polynomial with an odd order has at least one real zero.
If a function has a zero at x=r with multiplicity m then the function and its first (m-1) derivatives are zero at x=r and the mth derivative at r is not zero.
10
Roots of Equations & Zeros of Function
)1and,3,3,2are(Which
0 )(equation theofroots theas same theare )( of zeros The
181573)(
:as)(Define
0181573
:equation theof side one to termsall Move
181573
:equationGiven the
234
234
234
xfxf
xxxxxf
xf
xxxx
xxxx
11
Solution Methods
Several ways to solve nonlinear equations are possible:
Analytical Solutions
Possible for special equations only
Graphical Solutions
Useful for providing initial guesses for other methods
Numerical Solutions
Open methods
Bracketing methods
12
Analytical Methods
Analytical Solutions are available for special equations only.
a
acbbroots
cxbxa
2
4
0:ofsolution Analytical
2
2
0 :for available issolution analytical No xex
13
Graphical Methods
Graphical methods are useful to provide an initial guess to be used by other methods.
0.6
]1,0[
root
rootThe
ex
Solve
x
xe
x
Root
1 2
2
1
14
Numerical MethodsMany methods are available to solve nonlinear
equations:
Bisection Method
False position Method
Newton’s Method
Secant Method
Muller’s Method
Bairstow’s Method
Fixed point iterations
……….
These will be
covered here
15
Bracketing Methods
In bracketing methods, the method starts with an interval that contains the root and a procedure is used to obtain a smaller interval containing the root.
Examples of bracketing methods:
Bisection method
False position method
16
Open Methods
In the open methods, the method starts with one or more initial guess points. In each iteration, a new guess of the root is obtained.
Open methods are usually more efficient than bracketing methods.
They may not converge to a root.
17
Convergence Notation
Nnxx
N
xxxx
n
n
:thatsuchexiststhere0everyto
if to tosaid is,...,...,, sequenceA 21 converge
18
Convergence Notation
Cxx
xxP
Cxx
xx
Cxx
xx
xxx
p
n
n
n
n
n
n
1
2
1
1
21
:order of eConvergenc
:eConvergenc Quadratic
:eConvergencLinear
. toconverge,....,,Let
19
Speed of Convergence
We can compare different methods in terms of their convergence rate.
Quadratic convergence is faster than linear convergence.
A method with convergence order qconverges faster than a method with convergence order p if q>p.
Methods of convergence order p>1 are said to have super linear convergence.
20
Bisection Method
The Bisection method is one of the simplest methods to find a zero of a nonlinear function.
It is also called interval halving method.
To use the Bisection method, one needs an initial interval that is known to contain a zero of the function.
The method systematically reduces the interval. It does this by dividing the interval into two equal parts, performs a simple test and based on the result of the test, half of the interval is thrown away.
The procedure is repeated until the desired interval size is obtained.
21
Intermediate Value Theorem
Let f(x) be defined on the interval [a,b].
Intermediate value theorem:
if a function is continuousand f(a) and f(b) have different signs then the function has at least one zero in the interval [a,b].
a b
f(a)
f(b)
22
Examples
If f(a) and f(b) have the same sign, the function may have an even number of real zeros or no real zeros in the interval [a, b].
Bisection method can not be used in these cases.
a b
a b
The function has four real zeros
The function has no real zeros
23
Two More Examples
a b
a b
If f(a) and f(b) have different signs, the function has at least one real zero.
Bisection method can be used to find one of the zeros.
The function has one real zero
The function has three real zeros
24
Bisection Method
If the function is continuous on [a,b] and f(a) and f(b) have different signs, Bisection method obtains a new interval that is half of the current interval and the sign of the function at the end points of the interval are different.
This allows us to repeat the Bisection procedure to further reduce the size of the interval.
25
Bisection Method
Assumptions:
Given an interval [a,b]
f(x) is continuous on [a,b]
f(a) and f(b) have opposite signs.
These assumptions ensure the existence of at least one zero in the interval [a,b]and the bisection method can be used to obtain a smaller interval that contains the zero.
26
Bisection AlgorithmAssumptions: f(x) is continuous on [a,b] f(a) f(b) < 0
Algorithm:Loop1. Compute the mid point c=(a+b)/22. Evaluate f(c)3. If f(a) f(c) < 0 then new interval [a, c]
If f(a) f(c) > 0 then new interval [c, b]End loop
a
b
f(a)
f(b)
c
a0
b0
a1 a2
27
Example
+ + -
+ - -
+ + -
28
Flow Chart of Bisection Method
Start: Given a,b and ε
u = f(a) ; v = f(b)
c = (a+b) /2 ; w = f(c)
is
u w <0
a=c; u= wb=c; v= w
is
(b-a) /2<εyes
yes
no Stop
no
29
Example
Answer:
[0,2]? interval in the13)(
:of zero a find tomethodBisection useyou Can
3 xxxf
used benot can method Bisection
satisfiednot are sAssumption
03(1)(3)f(2)*f(0) and
[0,2]on continuous is)(
xf
30
Example
Answer:
[0,1]? interval in the13)(
:of zero a find tomethodBisection useyou Can
3 xxxf
used becan method Bisection
satisfied are sAssumption
01(1)(-1)f(1)*f(0) and
[0,1]on continuous is)(
xf
31
Best Estimate and Error Level
Bisection method obtains an interval that is guaranteed to contain a zero of the function.
The best estimate of the zero of the function f(x) after the first iteration of the Bisection method is the mid point of the initial interval:
2
2:
abError
abrzerotheofEstimate
32
Stopping Criteria
Two common stopping criteria
1. Stop after a fixed number of iterations
2. Stop when the absolute error is less than a specified value
How are these criteria related?
33
Stopping Criteria
nn
n
an
n
n
xabEc-rerror
n
c
c
22
:iterations After
function. theof zero theis :r
root). theof estimate theas usedusually is (
iteration n at the interval theofmidpoint theis:
0
th
34
Convergence Analysis
?) (i.e., estimate bisection
theis and of zero theiswhere
:such that needed are iterationsmany How
,,),(
kcx
xf(x)r
r-x
andbaxfGiven
)2log(
)log()log(
abn
35
Convergence Analysis – Alternative Form
abn 22 log
error desired
interval initial ofwidth log
)2log(
)log()log(
abn
36
Example
11
9658.10)2log(
)0005.0log()1log(
)2log(
)log()log(
? :such that needed are iterationsmany How
0005.0,7,6
n
abn
r-x
ba
37
Example
Use Bisection method to find a root of the equation x = cos (x) with absolute error <0.02
(assume the initial interval [0.5, 0.9])
Question 1: What is f (x) ?
Question 2: Are the assumptions satisfied ?
Question 3: How many iterations are needed ?
Question 4: How to compute the new estimate ?
CISE301_Topic2 38
39
Bisection MethodInitial Interval
a =0.5 c= 0.7 b= 0.9
f(a)=-0.3776 f(b) =0.2784Error < 0.2
0.5 0.7 0.9
-0.3776 -0.0648 0.2784Error < 0.1
0.7 0.8 0.9
-0.0648 0.1033 0.2784Error < 0.05
40
Bisection Method
0.7 0.75 0.8
-0.0648 0.0183 0.1033Error < 0.025
0.70 0.725 0.75
-0.0648 -0.0235 0.0183Error < .0125
Initial interval containing the root: [0.5,0.9]
After 5 iterations:
Interval containing the root: [0.725, 0.75]
Best estimate of the root is 0.7375
| Error | < 0.0125
41
A Matlab Program of Bisection Method
a=.5; b=.9;
u=a-cos(a);
v=b-cos(b);
for k=1:5
c=(a+b)/2
fc=c-cos(c)
if u*fc<0
b=c ; v=fc;
else
a=c; u=fc;
end
end
c =
0.7000
fc =
-0.0648
c =
0.8000
fc =
0.1033
c =
0.7500
fc =
0.0183
c =
0.7250
fc =
-0.0235
42
Example
Find the root of:
root thefind toused becan method Bisection
0)()(1)1(,10 *
continuous is *
[0,1] :interval in the13)( 3
bfaff)f(
f(x)
xxxf
43
Example
Iteration a bc= (a+b)
2f(c)
(b-a)
2
1 0 1 0.5 -0.375 0.5
2 0 0.5 0.25 0.266 0.25
3 0.25 0.5 .375 -7.23E-3 0.125
4 0.25 0.375 0.3125 9.30E-2 0.0625
5 0.3125 0.375 0.34375 9.37E-3 0.03125
44
Bisection MethodAdvantages Simple and easy to implement One function evaluation per iteration The size of the interval containing the zero is
reduced by 50% after each iteration The number of iterations can be determined a
priori No knowledge of the derivative is needed The function does not have to be differentiable
Disadvantage Slow to converge Good intermediate approximations may be
discarded
45
The False-Position Method (Regula-Falsi)
We can approximate the solution by doing a linear interpolation between f(xu) and f(xl)
Find xr such that l(xr)=0, where l(x) is the linear approximation of f(x) between xl and xu
Derive xr using similar triangles
lu
luulr
ff
fxfxx
xl
xu
f(xl)
f(xu)
next estimate,
xr
)()(
)()(
afbf
abfbafc
xfxf
xfxxfx
xfxf
xxxfxx
xx
xf
xx
xf
bx
ax
ul
ullu
ul
uluur
ur
u
lr
l
l
u
Based on
similar
triangles
47
Example
0 ,1 ,0
0sin3)(
1010
ffxx
exxxf x
k xk (Bisection) fk xk (False Position) fk
1 0.5 0.471
2 0.25 0.372
3 0.375 0.362
4 0.3125 0.360
5 0.34315 -0.042 0.360 2.93×10-5
Pitfalls of False Position Method
f(x)=x10-1
-5
0
5
10
15
20
25
30
0 0.5 1 1 .5
x
f(x)
49
Iteration xl xu xr εa (%) εt (%)
1 0 1.3 0.65 35
2 0.65 1.3 0.975 33.3 25
3 0.975 1.3 1.1375 14.3 13.8
4 0.975 1.1375 1.05625 7.7 5.6
5 0.975 1.05625 1.015625 4.0 1.6
Iteration xl xu xr εa (%) εt (%)
1 0 1.3 0.09430 90.6
2 0.09430 1.3 0.18176 48.1 81.8
3 0.18176 1.3 0.26287 30.9 73.7
4 0.26287 1.3 0.33811 22.3 66.2
5 0.33811 1.3 0.40788 17.1 59.2
Bisection Method (Converge quicker)
False-position Method
50
Newton-Raphson Method (Also known as Newton’s Method)
Given an initial guess of the root x0, Newton-Raphson method uses information about the function and its derivative at that point to find a better guess of the root.
Assumptions: f(x) is continuous and the first derivative
is known
An initial guess x0 such that f’(x0)≠0 is given
51
Newton Raphson Method- Graphical Depiction -
If the initial guess at the root is xi, then a tangent to the function of xi that is f’(xi) is extrapolated down to the x-axis to provide an estimate of the root at xi+1.
52
Derivation of Newton’s Method
)('
)( :root theof guess newA
)('
)(
.0)(such that Find
)(')()(:TheroremTaylor
____________________________________
? estimatebetter aobtain wedo How:
0)( ofroot theof guess initialan :
1
1
i
iii
i
i
xf
xfxx
xf
xfh
hxfh
hxfxfhxf
xQuestion
xfxGiven
FormulaRaphson Newton
53
Newton’s Method
end
xf
xfxx
nifor
xfnAssumputio
xxfxfGiven
i
iii
)('
)(
:0
______________________
0)('
),('),(
1
0
0
end
XFPXFXX
kfor
X
PROGRAMMATLAB
)(/)(
5:1
4
%
XXFP
XFPFPfunction
XXF
XFFfunction
*62^*3
)(][
12^*33^
)(][
F.m
FP.m
54
Example
2130.20742.9
0369.24375.2
)('
)( :3Iteration
4375.216
93
)('
)( :2Iteration
333
334
)('
)( :1Iteration
143'
4 , 32function theof zero a Find
2
223
1
112
0
001
2
023
xf
xfxx
xf
xfxx
xf
xfxx
xx(x) f
xxxxf(x)
55
Example
k (Iteration) xk f(xk) f’(xk) xk+1 |xk+1 –xk|
0 4 33 33 3 1
1 3 9 16 2.4375 0.5625
2 2.4375 2.0369 9.0742 2.2130 0.2245
3 2.2130 0.2564 6.8404 2.1756 0.0384
4 2.1756 0.0065 6.4969 2.1746 0.0010
56
Convergence Analysis
)('min
)(''max
2
1
such that
0 exists then there0)('.0)( where
rat x continuous be)('')('),(Let
:Theorem
0
0
2
1
0
xf
xf
C
C-rx
-rx-rx
rfIfrf
xf andxfxf
-rx
-rx
k
k
Proof
57
2
'
''
1
2''
1
'
1
'
2''
'
)()(2
)(
0)(!2
)()()(
)()()(0 :Raphson-Newton
;0)(!2
)()()()()(
],[:about )( ofexpansion seriesTaylor The
iii
iii
iiii
iiii
ii
xrrf
rfxrrx
xrf
xrxf
xxxfxf
xrf
xrxfxfrf
rxxrf
58
Convergence AnalysisRemarks
When the guess is close enough to a simple root of the function then Newton’s method is guaranteed to converge quadratically.
Quadratic convergence means that the number of correct digits is nearly doubled at each iteration.
59
Problems with Newton’s Method
• If the initial guess of the root is far from
the root the method may not converge.
• Newton’s method converges linearly near
multiple zeros { f(r) = f’(r) =0 }. In such a
case, modified algorithms can be used to
regain the quadratic convergence.
60
Multiple Roots
1at zeros0at x zeros
twohas )( threehas)(
-x
xfxf
21)( xxf
3)( xxf
61
Problems with Newton’s Method- Runaway -
The estimates of the root is going away from
the root.
x0 x1
62
Problems with Newton’s Method- Flat Spot -
The value of f’(x) is zero, the algorithm fails.
If f ’(x) is very small then x1 will be very far from x0.
x0
63
Problems with Newton’s Method- Cycle -
The algorithm cycles between two values x0 and x1
x0=x2=x4
x1=x3=x5
64
Newton’s Method for Systems of
Non Linear Equations
M
M
M2
2
1
2
2
1
1
1
212
211
1
1
0
)(',,...),(
,...),(
)(
)()('
'
0)( ofroot theof guess initialan :
x
f
x
f
x
f
x
f
XFxxf
xxf
XF
XFXFXX
IterationsNewton
xFXGiven
kkkk
65
Example
Solve the following system of equations:
0,1 guess Initial
05
050
2
2
yx
yxyx
x.xy
0
1,
1552
112',
5
5002
2
Xxyx
xF
yxyx
x.xyF
66
Solution Using Newton’s Method
2126.0
2332.1
0.25-
0.0625
25.725.1
1 1.5
25.0
25.1
25.725.1
1 1.5',
0.25-
0.0625
:2Iteration
25.0
25.1
1
50
62
11
0
1
62
11
1552
112',
1
50
5
50
:1Iteration
1
2
1
1
2
2
X
FF
.X
xyx
xF
.
yxyx
x.xyF
67
ExampleTry this
Solve the following system of equations:
0,0 guess Initial
02
01
22
2
yx
yyx
xxy
0
0 ,
142
112',
2
1022
2
Xyx
xF
yyx
xxyF
68
ExampleSolution
0.1980
0.5257
0.1980
0.5257
0.1969
0.5287
2.0
6.0
0
1
0
0
_____________________________________________________________
543210
kX
Iteration
69
Newton’s Method (Review)
ly.analyticalobtain todifficult or
available,not is )('
:Problem
)('
)(
: '
0)('
,),('),(:
1
0
0
i
i
iii
xf
xf
xfxx
estimatenewMethodsNewton
xf
availablearexxfxfsAssumption
70
Secant Method
)()(
)()(
)(
)()(
)(
)(
)()()('
:points initial twoare if
)()()('
1
1
1
11
1
1
1
ii
iiii
ii
ii
iii
ii
iii
ii
xfxf
xxxfx
xx
xfxf
xfxx
xx
xfxfxf
xandx
h
xfhxfxf
71
Secant Method
)()(
)()(
:Method)(Secant estimateNew
)()(
points initial Two
:sAssumption
1
11
1
1
ii
iiiii
ii
ii
xfxf
xxxfxx
xfxfthatsuch
xandx
Comparison of False Position and
Secant Method
x
f(x)
x
f(x)
1
1
2
new est.new est.
2
73
Secant Method
)()(
)()(
1
0
5.02)(
1
11
1
0
2
ii
iiiii
xfxf
xxxfxx
x
x
xxxf
74
Secant Method - Flowchart
1
;)()(
)()(
1
11
ii
xfxf
xxxfxx
ii
iiiii
1,, 10 ixx
ii xx 1Stop
NO Yes
75
Modified Secant Method
.divergemay method theproperly, selectednot If
?select to How:Problem
)()(
)(
)()(
)(
)() ()('
:needed is guess initial oneonly method,Secant modified thisIn
1
iii
iii
i
iii
iii
i
iiii
xfxxf
xfxx
x
xfxxf
xfxx
x
xfxxfxf
76
-2 -1.5 -1 -0.5 0 0.5 1 1.5 2-40
-30
-20
-10
0
10
20
30
40
50
Example
001.0
1.11
points Initial
3)(
:of roots theFind
10
35
errorwith
xandx
xxxf
77
Example
x(i) f(x(i)) x(i+1) |x(i+1)-x(i)|
-1.0000 1.0000 -1.1000 0.1000
-1.1000 0.0585 -1.1062 0. 0062
-1.1062 -0.0102 -1.1053 0.0009
-1.1053 0.0000 -1.1053 0.0000
78
Convergence Analysis
The rate of convergence of the Secant method is super linear:
It is better than Bisection method but not as good as Newton’s method.
iteration. i at theroot theof estimate::
62.1,
th
1
i
i
i
xrootr
Crx
rx
79
Summary
Method Pros Cons
Bisection - Easy, Reliable, Convergent
- One function evaluation per iteration
- No knowledge of derivative is needed
- Slow
- Needs an interval [a,b] containing the root, i.e., f(a)f(b)<0
Newton - Fast (if near the root)
- Two function evaluations per iteration
- May diverge
- Needs derivative and an initial guess x0 such that f’(x0) is nonzero
Secant - Fast (slower than Newton)
- One function evaluation per iteration
- No knowledge of derivative is needed
- May diverge
- Needs two initial points guess x0, x1 such that
f(x0)- f(x1) is nonzero
80
Example
)()(
)()(
5.11 points initial Two
1)(
:ofroot thefind tomethodSecant Use
1
11
10
6
ii
iiiii
xfxf
xxxfxx
xandx
xxxf
81
Solution
_______________________________
k xk f(xk)
_______________________________
0 1.0000 -1.0000
1 1.5000 8.8906
2 1.0506 -0.7062
3 1.0836 -0.4645
4 1.1472 0.1321
5 1.1331 -0.0165
6 1.1347 -0.0005
82
Example
.0001.0)(if
or,001.0if
or ,iterations threeafterStop
.1 :point initial theUse
1)(
:ofroot a find toMethod sNewton' Use
1
0
3
k
kk
xf
xx
x
xxxf
83
Five Iterations of the Solution
k xk f(xk) f’(xk) ERROR
______________________________________
0 1.0000 -1.0000 2.0000
1 1.5000 0.8750 5.7500 0.1522
2 1.3478 0.1007 4.4499 0.0226
3 1.3252 0.0021 4.2685 0.0005
4 1.3247 0.0000 4.2646 0.0000
5 1.3247 0.0000 4.2646 0.0000
84
Example
.0001.0)(if
or,001.0if
or ,iterations threeafterStop
.1 :point initial theUse
)(
:ofroot a find toMethod sNewton' Use
1
0
k
kk
x
xf
xx
x
xexf
85
Example
0.0000- 1.5671- 0.0000 0.5671
0.0002- 1.5672- 0.0002 0.5670
0.0291- 1.5840- 0.0461 0.5379
0.4621 1.3679- 0.6321- 1.0000
)('
)()(')(
1)(',)(
:ofroot a find toMethod sNewton' Use
k
kkkk
xx
xf
xf xf xf x
exfxexf
86
Example
Estimates of the root of: x-cos(x)=0.
0.60000000000000 Initial guess
0.74401731944598 1 correct digit
0.73909047688624 4 correct digits
0.73908513322147 10 correct digits
0.73908513321516 14 correct digits
87
Example
In estimating the root of: x-cos(x)=0, to get more than 13 correct digits:
4 iterations of Newton (x0=0.8)
43 iterations of Bisection method (initial
interval [0.6, 0.8])
5 iterations of Secant method
( x0=0.6, x1=0.8)
