1. Numerical Treatment of ODE€¦ · Numerical Treatment... Euler’s Method Discretization Error Order of Convergence Method of Runge and... Alternative: Multistep... Implicit Methods

Numerical Treatment . . .

Euler’s Method

Discretization Error

Order of Convergence

Method of Runge and . . .

Alternative: Multistep . . .

Implicit Methods

Types of Problems for . . .

Ill-Conditioned Problems

Stability

Stiffness

Boundary Value . . .


Page 1 of 13

Introduction to Scientific Computing

5. Numerical Methods forODE

Miriam Mehl

1. Numerical Treatment of ODE

• remember population dynamics: one or some ODE

– given as an initial value problem : starting point given

http://www5.in.tum.de/persons/mehl/


Euler’s Method





Implicit Methods



Stability

Stiffness



Page 1 of 13



Miriam Mehl




– sometimes also as a boundary value problem: starting andfinal point given (think of a space shuttle’s trajectory, e.g.)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 1 of 13



Miriam Mehl





• prototypes of an initial value problem (IVP):

y(t) = f(t, y(t)), y(a) = ya, t ≥ a



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 1 of 13



Miriam Mehl






y(t) = f(t, y(t)), y(a) = ya, t ≥ a

or for several unknowns yi, i = 1, . . . , n:

yi(t) = fi(t, y1(t), . . . , yn(t)), yi(a) = yi,a, t ≥ a, i = 1, . . . , n



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 1 of 13



Miriam Mehl






y(t) = f(t, y(t)), y(a) = ya, t ≥ a

or for several unknowns yi, i = 1, . . . , n:

yi(t) = fi(t, y1(t), . . . , yn(t)), yi(a) = yi,a, t ≥ a, i = 1, . . . , n

• if f depends on t only: simple integration or quadrature



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 2 of 13



Miriam Mehl

2. Euler’s Method

• standard approach for IVP: finite difference approximation (dif-ference quotient instead of derivative)

yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 2 of 13



Miriam Mehl

2. Euler’s Method


yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,

which gives an explicit formula for yk+1 once yk is given:

yk+1 = yk + δt · f(tk, yk).



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 2 of 13



Miriam Mehl

2. Euler’s Method


yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,



• this is the simplest strategy and called Euler’s method



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 2 of 13



Miriam Mehl

2. Euler’s Method


yk+1−yk

δt= f(tk, yk), tk = a + kδt, k = 0, 1, . . .,



• this is the simplest strategy and called Euler’s method

• other derivation: truncated Taylor expansion of y(t)

y(tk+1) = y(tk) + δt · y(tk) + O(δt2).= y(tk) + δt · f(tk, yk)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 3 of 13



Miriam Mehl

3. Discretization Error

• local discretization error: local influence of using differences in-stead of derivatives; here:



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 3 of 13



Miriam Mehl



l(δt) = max[a,b]{| y(t+δt)−y(t)δt

− f(t, y(t)) |}

with y(t) exact solution of the ODE.



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 3 of 13



Miriam Mehl




− f(t, y(t)) |}


• global discretization error: maximum error of all computed dis-crete approximations:

e(δt) = max[a,b]{| yk − y(tk) |}

with y(t) exact solution of the ODE, yk, k = 1, 2, . . . solution ofthe discretized equation.



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 3 of 13



Miriam Mehl




− f(t, y(t)) |}





• consistency:

l(δt) → 0 for δt → 0



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 3 of 13



Miriam Mehl




− f(t, y(t)) |}





• consistency:

l(δt) → 0 for δt → 0

• convergence (stronger):

e(δt) → 0 for δt → 0



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl

4. Order of Convergence

• Euler (some restrictions with respect to y and f ):



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl



– consistent of first order: l(δt) = O(δt)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl




– convergent of first order: e(δt) = O(δt)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl





– There are methods which are consistent but do not con-verge!



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl






• look for higher-order methods (faster convergence)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl







– start from Taylor expansion: leads to complicated formulas(higher derivatives of f )



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl








– use additional evaluations of f : Runge-Kutta-type methods



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl









– simplest representative: method of Heun

yk+1 = yk + δt2

(f(tk, yk) + f(tk+1, yk + δt · f(tk, yk))

)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 4 of 13



Miriam Mehl









– simplest representative: method of Heun

yk+1 = yk + δt2

(f(tk, yk) + f(tk+1, yk + δt · f(tk, yk))

)both consistent and convergent of second order



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 5 of 13



Miriam Mehl

5. Method of Runge and Kutta

• most famous representative: Runge-Kutta method



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 5 of 13



Miriam Mehl



yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

(tk +

δt

2, yk +

δt

2T1

),

T3 = f

(tk +

δt

2, yk +

δt

2T2

),

T4 = f(tk+1, yk + δtT3)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 5 of 13



Miriam Mehl



yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

(tk +

δt

2, yk +

δt

2T1

),

T3 = f

(tk +

δt

2, yk +

δt

2T2

),

T4 = f(tk+1, yk + δtT3)

• consistent and convergent of fourth order



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 5 of 13



Miriam Mehl



yk+1 = yk +δt

6(T1 + 2T2 + 2T3 + T4),

T1 = f(tk, yk),

T2 = f

(tk +

δt

2, yk +

δt

2T1

),

T3 = f

(tk +

δt

2, yk +

δt

2T2

),

T4 = f(tk+1, yk + δtT3)

• consistent and convergent of fourth order

• Euler/Heun/Runge-Kutta correspond to rectangle/trapezoidal/Simp-son quadrature!



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl

6. Alternative: Multistep Methods

• Runge-Kutta-type methods are expensive : many evaluationsof f (sometimes not given in closed form)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl



• different way to get higher order by profiting from history: Adams-Bashforth-type or multistep methods



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl




– prominent representative: second-order method

yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl





yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)

)– idea: use evaluations of f for previous approximations

yk−p+1, . . . , yk to find an approximation of the f -integral inthe integral form of the ODE:

yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl





yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)



yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.

– general form: take polynomial P (t) interpolating f in thediscrete points of time (tk−p+1, . . . , tk)

yk+1.= yk +

∫ tk+1

tkP (t)dt



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl





yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)



yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.


yk+1.= yk +

∫ tk+1

tkP (t)dt

– p = 1: Euler; p = 2: above method; generally: order p



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 6 of 13



Miriam Mehl





yk+1 = yk + δt2

(3f(tk, yk)− f(tk−1, yk−1)



yk+1 = yk +∫ tk+1

tkf(t, y(t))dt.


yk+1.= yk +

∫ tk+1

tkP (t)dt

– p = 1: Euler; p = 2: above method; generally: order p

– start: no/not enough predecessors available; hence modify!



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods

• All schemes mentioned so far are explicit ones: The rule showsa direct way to do another time step.



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods


• now: use the new value yk+1 on the right-hand side, too



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods



• This leads us to Adams-Moulton multistep schemes:

– use interpolation and previous values as with Adams-Bashforth(fj = f(yj, tj))



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods





– second order variant:

yk+1 = yk + δt2(fk + fk+1)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods






yk+1 = yk + δt2(fk + fk+1)

– fourth order variant:

yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods






yk+1 = yk + δt2(fk + fk+1)


yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)

• How to get fk+1 = f(yk+1, tk+1) in the implicit case?



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods






yk+1 = yk + δt2(fk + fk+1)


yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)


– straightforward way: solve the (generally nonlinear) equa-tion



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 7 of 13



Miriam Mehl

7. Implicit Methods






yk+1 = yk + δt2(fk + fk+1)


yk+1 = yk + δt24

(fk−2 − 5fk−1 + 19fk + 9fk+1)


– straightforward way: solve the (generally nonlinear) equa-tion

– easier (and widespread): predictor-correctorapproach



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 8 of 13



Miriam Mehl

8. Types of Problems for Numerical Methods for ODEs

• Ill-Conditioned Problems:

small changes in the input ⇒ big changes in the exact solutionof the ODE



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 8 of 13



Miriam Mehl




• Instabilities:

big errors in the approximate (i.e. numerical) solution comparedto the exact solution (for arbitrarily small time steps although themethod is consistent)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 8 of 13



Miriam Mehl




• Instabilities:

big errors in the approximate (i.e. numerical) solution comparedto the exact solution (for arbitrarily small time steps although themethod is consistent)

• Stiffness:

small time steps required for acceptable errors in the approxi-mate solution (although the exact solution is smooth)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl

9. Ill-Conditioned Problems

• small changes in input entail completely different results



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl



• Numerical treatment of such problems is difficult!



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl




• an example:

– consider the ODE: y(t)−N · y(t)− (N + 1) · y(t) = 0

– initial conditions: y(0) = 1, y(0) = −1

– exact solution: y(t) = e−t



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl




• an example:




• slight change in initial condition:

– new value of y in t = 0: yε(0) = 1 + ε



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl




• an example:






– resulting new solution: yε(t) =

(1 + N+1

N+2ε

)e−t + ε

N+2e(N+1)t



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl




• an example:







(1 + N+1

N+2ε

)e−t + ε

N+2e(N+1)t

– arbitrarily small change leads to completely different result:t →∞



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 9 of 13



Miriam Mehl




• an example:







(1 + N+1

N+2ε

)e−t + ε

N+2e(N+1)t

– arbitrarily small change leads to completely different result:t →∞

• risk: non-precise input, round-off errors,. . .



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability

• consider another IVP: y(t) = −2y(t) + 1, y(0) = 1



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability


• exact solution: y(t) =

(e−2t + 1

)/2



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2

• well-conditioned: yε(0) = 1 + ε ⇒ yε(t)− y(t) = εe−2t



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2


• use the midpoint rule: yk+1 = yk−1 + 2δt · fk

yk+1 = yk−1 + 2δt(−2yk + 1) = yk−1 − 4δt · yk + 2δt, y0 = 1



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2




• 2-step rule: start with initial value and the exact y(δt)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2





– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2





– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7

• midpoint rule is second-order consistent, but does not convergehere: oscillations or instable behaviour



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 10 of 13



Miriam Mehl

10. Stability



(e−2t + 1

)/2





– time step δt = 1.0 ⇒ y9 = −4945.5, y10 = 20953.9

– time step δt = 0.1 ⇒ y79 = −1725.3, y80 = 2105.7

– time step δt = 0.01 ⇒ y999 = −154.6, y1000 = 158.7

• midpoint rule is second-order consistent, but does not convergehere: oscillations or instable behaviour

• there are stability conditions; generally:

consistency + stability = convergence



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness

• for another phenomen, consider the IVP

y(t) = −1000y(t) + 1000, y(0) = y0 = 2



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2

• exact solution: y(t) = e−1000t + 1



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2


• problem well-conditioned



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2



• explicit Euler: stable (as all explicit 1-step methods or all Adams-Bashforth or all s-step Adams-Moulton methods for s > 1 are)



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2




yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2




yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1

• oscillations and divergence for δt > 0.002



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 11 of 13



Miriam Mehl

11. Stiffness


y(t) = −1000y(t) + 1000, y(0) = y0 = 2




yk+1 = yk + δt(−1000yk + 1000) = (1− 1000δt)yk + 1000δt

= (1− 1000δt)k+1 + 1

• oscillations and divergence for δt > 0.002

• Why that? Consistency and stability are asymptoticterms. Rem-edy: Implicit methods (try implicit Euler)!



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 12 of 13



Miriam Mehl

12. Boundary Value Problems: Outlook

• example:

y = f(t, y, y), ta ≤ t ≤ tb,



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 12 of 13



Miriam Mehl


• example:

y = f(t, y, y), ta ≤ t ≤ tb,

y(ta) = ya, y(tb) = yb



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 12 of 13



Miriam Mehl


• example:

y = f(t, y, y), ta ≤ t ≤ tb,


• special case:

y(t) = a(t)y(t) + b(t)y(t) + c(t), same b.c.



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 12 of 13



Miriam Mehl


• example:

y = f(t, y, y), ta ≤ t ≤ tb,


• special case:


• a vanishing and b positive: BVP has unique solution



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 12 of 13



Miriam Mehl


• example:

y = f(t, y, y), ta ≤ t ≤ tb,


• special case:


• a vanishing and b positive: BVP has unique solution

• discrete grid:

δt = (tb − ta)/n, t0 = ta, tn = tb, ti = ta + iδt



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 13 of 13



Miriam Mehl

13. Boundary Value Problem 2

• y(t) = b(t)y(t) + c(t), y(ta) = ya, y(tb) = yb



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 13 of 13



Miriam Mehl



• finite difference approximation for second derivative:

y(t).= y(t+δt)−2y(t)+y(t−δt)

δt2



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 13 of 13



Miriam Mehl





δt2

• discrete analogon to ODE in each grid point:

δt−2 · (yi+1 − 2yi + yi−1)− biyi = ci, i = 1, . . . , n− 1



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 13 of 13



Miriam Mehl





δt2



• tridiagonal system of linear equations



Euler’s Method





Implicit Methods



Stability

Stiffness



Page 13 of 13



Miriam Mehl





δt2



• tridiagonal system of linear equations

• alternative: shooting methods(reduction to IVPs)


Documents

1. Numerical Treatment of ODE€¦ · Numerical Treatment... Euler’s Method Discretization Error Order of Convergence Method of Runge and... Alternative: Multistep... Implicit Methods