3 Stability and Stiffness - TU Bergakademie Freibergernst/Lehre/Numerik_II/Folien/nade... · 3 Stability and Stiffness TU Bergakademie Freiberg, ... Finally, ode23t is an absolutely

Numerical Analysis of Differential Equations 115

3 Stability and Stiffness

Contents

3.1 Absolute Stability

3.2 Stiff Differential Equations

3.3 Further Stability Concepts

3 Stability and Stiffness TU Bergakademie Freiberg, SS 2012


3.1 Absolute Stability

In contrast with zero-stability, the property of a numerical method for solvingIVPs to generate bounded approximations on a fixed interval as h→ 0, theterm absolute stability is concerned with the behavior of such a scheme forlong integration times with fixed stepsize h > 0.

Qualitatively: given an IVP and a (convergent) numerical method, how smallmust we choose the stepsize in order for the numerical approximation tohave at least the correct “qualitative bahavior”?

Example 1: The IVP

y′(t) = − sin t, y(0) = 1 (3.1)

has the solution y(t) = cos t. The local discretization error of the explicitEuler method for (3.1) at a point t is

T (t) =h

2y′′(t) +O(h2) = −h

2cos t+O(h2).

3.1 Absolute Stability TU Bergakademie Freiberg, SS 2012


Our standard bound for the global discretization error on an interval [0, tend]

yields, for tend = 2,

|en| ≤ tend max0≤t≤T

|T (t)| ≈ h max0≤t≤tend

| cos t| = h.

To achieve an error |e| ≤ 10−3 when integrating up to tend = 2, a stepsizeof h = 10−3 should suffice, and for N = 2000 we obtain

yN = −4.166014e− 01, |y(2)− yN | = 4.547667e− 04.

Example 2: Consider the modification of (3.1)

y′(t) = − sin t+λ(y − cos t), y(0) = 1, (3.2)

with the same solution y(t) = cos t. For λ = −10, we obtain in this case

yN = −4.170721e− 01, |y(2)− yN | = 1.611611e− 05.



Example 3: Consider the previous example with λ = −2100. In this caseone obtains

yN = 1.597768e+ 76, |y(2)− yN | = 1.452516e+ 76.

Different stepsizes for this IVP

h error at T = 2

0.001 1.7e+ 76

0.000976 3.1e+ 36

0.00095 8.6e− 04

0.0008 7.3e− 04

0.0004 3.6e− 04



3.1.1 Runge-Kutta Methods

Solutions y(t) of y ′ = Ay with constant A ∈ Rn×n converge to 0 (ast → ∞) if Re(λ) < 0 for all eigenvalues λ of A. We seek conditions underwhich a RKM generates approximate solutions with the same asymptoticbehavior.

Applying an m-stage RKM to the test equation y′ = λy yields the approxi-mation sequence

yn+1 =[1 + hb>(Im − hA)−1e

]yn =: R(h) yn, h := hλ.

Therefore, for h fixed, we have limn→∞ yn = 0 for all y0 if, and only if,|R(h)| < 1.

In general: any OSM applied to the test equation yields approximations

yn+1 = R(h)yn

with a function R of h = λh depending on the method.



We define the region of absolute stability RA of a OSM by

RA :={h ∈ C : |R(h)| < 1

}.

A OSM is called absolutely stable, if RA ⊇ {h : Re(h) < 0}.

For any m-stage RKM, we have

R(h) = 1 + hb>(Im − hA)−1e = 1 +∞∑j=1

hjbTAj−1e

for h sufficiently small.

If this method has order p, then

R(h) =

p∑j=0

1

j!hj +

∞∑j=p+1

hjb>Aj−1e .

For explicit RKM: R(h) = 1 +∑mj=1 h

jbTAj−1e .



Explicit m-stage RKMs of order m (1 ≤ m ≤ 4) all have the stability functionR(h) =

∑mj=0

1j! h

j . Therefore RA is independent of the coefficients forthese methods.

Moreover, no explicit RKM has an unbounded region of absolute stability,as R is a polynomial in this case.

−4 −3 −2 −1 0 1 2−3

−2

−1

0

1

2

3Verfahren dritter Ordnung von Heun

−4 −3 −2 −1 0 1 2−3

−2

−1

0

1

2

3klassisches Rung−Kutta−Verfahren



For the implicit Euler method,

R(h) =1

1− hso that RA =

{h : |1− h| > 1

}(exterior of circle around 1 with radius 1).

For the trapezoidal rule,

R(h) =1 + 1

2 h

1− 12 h

so that in this case

RA = {h : Re h < 0} (left half-plane).

Both methods are therefore absolutely stable.



3.1.2 Linear Multistep Methods

When a LMM is applied to the test equation y′ = λy the resulting numericalapproximation satisfies the difference equation

r∑j=0

αjyn+j = h

r∑j=0

βjλyn+j

orr∑

j=0

(αj − hβj)yn+j = 0, h = hλ.

Its solutions are of the form (2.13), in which ζ1, . . . , ζr are the roots of its characte-ristic polynomial

π(ζ) = π(ζ; h) :=

r∑j=0

(αj − hβj)ζj = ρ(ζ)− hσ(ζ),

which is called the stability polynomial of the associated LMM. A LMM is thereforeabsolutely stable for a given value of h if the zeros of the stability polynomials forthis value of h satisfy the conditions for bounded solutions.



Examples:

Method π(ζ; h)

Euler (explicit) ζ + (1 + h)

Euler (implicit) (1− h)ζ − 1

trapezoidal rule (1− h/2)ζ − (1 + h/2)ζ

midpoint rule ζ2 − 2hζ − 1



20 10 0 10 20 30 4025

20

15

10

5

0

5

10

15

20

25

Re

Im

BDF1BDF2BDF3BDF4BDF5BDF6

BDF methods: The region RA of absolute stability of the BDF methods(r = 1, . . . , 6) is the exterior of the corresponding closed curve.



3.2 Stiff Differential Equations

Stiffness is a property of differential equations with strong implications fortheir practical solution using numerical methods. Unfortunately, a precisemathematical definition of stiffness covering all occurrences of this pheno-menon has not been found.

We therefore resort to a series of examples of stiff differential equations.

Example 1. The following two IVPs

y ′ =

[−2 1

1 −2

]y +

[2 sin t

2(cos t− sin t)

], y(0) =

[2

3

], (IVP1)

y ′ =

[−2 1

998 −999

]y +

[2 sin t

999(cos t− sin t)

], y(0) =

[2

3

], (IVP2)

possess the same solution

3.2 Stiff Differential Equations TU Bergakademie Freiberg, SS 2012


y(t) = exp(−t)

[2

2

]+

[sin t

cos t

].

0 2 4 6 8 10−1

−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2



We solve both on t ∈ [0, 10] using MATLAB’s ODE solver ode45a, specifyinga relative tolerance of 0.01.

0 2 4 6 8 10−1

−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2

System 1: 64 Schritte (ode45)

0 2 4 6 8 10−1

−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2

System 2: 12044 Schritte (ode45)

abased on the embedded Runge-Kutta pair of Dormand and Prince of order 4 and 5 foradaptive stepsize selection



However, with this method solving (IVP2) is roughly 200 times more costlythan for (IVP1):

IVP hmin hmax h∅ # steps

(IVP1) 2.27e-2 2.03e-1 1.56e-1 64

(IVP2) 5.98e-4 2.88e-3 8.30e-3 12044

Applying instead MATLAB’s ODE solver ode23tba, again with a relative errortolerance of 0.01, one obtains


(IVP1) 2.13e-1 5.81e-1 5.26e-1 19

(IVP2) 2.13e-1 7.36e-1 5.00e-1 20

This time both problems are solved without difficulty.abased on an implicit 2-stage RKM with the trapezoidal rule and the BDF2 method as its

first and second stages, respectively



0 2 4 6 8 10−1

−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2

System 1: 19 Schritte (ode23tb)

0 2 4 6 8 10−1

−0.5

0

0.5

1

1.5

2

2.5

3

y1

y2

System 2: 20 Schritte (ode23tb)



The reason for the different behavior of the two IVPs becomes apparentwhen we look at the general solutions of each associated ODE system.

For (IVP1) this is

y(t) = κ1 exp(−t)

[1

1

]+ κ2 exp(−3t)

[1

−1

]+

[sin t

cos t

],

whereas for (IVP2)

y(t) = κ1 exp(−t)

[1

1

]+ κ2 exp(−1000t)

[1

−998

]+

[sin t

cos t

].

Both are inhomogeneous linear systems with constant coefficients, and theeigenvalues of the coefficient matrix are

λ1 = −1, λ2 = −3 for (IVP1), λ1 = −1, λ2 = −1000 for (IVP2).

In both cases imposing the initial condition leads to κ1 = 2 and κ2 = 0 (!).



In both ODE solutions: steady-state and (decaying) transient components

[sin t, cos t]> and exp(λ1t)a1 + exp(λ2t)a2.

Fastest decaying transient (as t→∞) determined by

λ2 = −3 for (IVP1) and λ2 = −1000 for (IVP2).

Even though neither is present in the solutions (due to κ2 = 0), this ratedetermines the minimal required stepsize.

Changing the initial data to y(0) = [0, 1]> in (IVP2) results in κ1 = κ2 = 0,i.e., both transient components are then completely invisible in the resultingexact solution y(t) = [sin t, cos t]>. Even then, however, ode45 would takeexcessively small stepsizes.



For ode45, the region RA of absolute stability satisfies

RA ∩ R ∼ [−3, 0] (stability interval)

To ensure hλ ∈ RA for both eigenvalues of (IVP1) it is sufficient to requireh < 1. The observed average stepsize of h∅ ∼ 0.156 is a consequence ofthe requested accuracy.

By contrast, for (IVP2) ensuring λh ∈ RA for all λa necessitates a stepsizeof h < 0.003. In this case the observed h∅ ∼ 0.008 results from the stabilityrequirement.

Finally, ode23t is an absolutely stable method, and therefore hλ ∈ RA forany h > 0 for both (IVP1) and (IVP2) when this method is used..



For a system of ODEsy ′ = Ay + b(t)

the quantitymaxλ∈Λ(A)

|Reλ| / minλ∈Λ(A)

|Reλ|

is called its stiffness ratio.

Statement 1: A linear system of ODEs with constant coefficients is calledstiff, when all its eigenvalues have negative real part and its stiffness ratiois large.



To see that Statement 1 is not a useful definition, consider yet another IVPwith the same solution as (IVP1) and (IVP2):

y ′ =

[−2 1

−1.999 0.999

]y +

[2 sin t

−0.999(cos t− sin t)

], y(0) =

[2

3

]. (IVP3)

The eigenvalues of its coefficient matrix are λ1 = −1 and λ2 = −0.001,giving the same stiffness ratio of 1000 as for (IVP2).

However, ode45 happily solves (IVP3) with reasonable stepsizes :


(IVP3) 1.19e-1 2.50e-1 2.27e-1 44 (ode45)

(IVP3) 1.25e-1 4.92e-1 4.55e-1 22 (ode23tb)



The stiffness ratio alone can therefore not always determine whether or nota system is stiff.

This might suggest modifying Statement 1 to

a system is stiff if max |Reλ| is large, e.g., if max |Reλ| � 1.

However, the change of variables t 7→ 0.001t transforms (IVP2) to an equiva-lent problem with identical stiffness properties, but for which max |Reλ| = 1.

Pragmatic “definitions” of stiffness include

A system is stiff if stability requirements — and not accuracy requirements— determine the stepsize.

A system is stiff, if certain components decay much faster than others.

Stiff equations are problems for which explicit methods don’t work.[Hairer and Wanner, Vol. II, p. 2]



Example 2. The ODE

y′(t) = λ[y(t)− g(t)] + g′(t) (∗)

has the general solution

y(t) = γ exp(λt) + g(t), γ ∈ R,

which, for Reλ < 0, consists of a decaying transient γ exp(λt) and a steady-state component g(t).

Setting λ = −10, g(t) = arctan t and initial condition y(0) = 0, we approxi-mate the exact solution y(t) = arctan t for t ∈ [0, 5] using

• explicit Euler (RA = {h : |h+ 1| < 1}) and• implicit Euler (RA = C \ {h : |h− 1| ≤ 1}).



−5 0 5−3

−2

−1

0

1

2

3Allgemeine Loesung von (*)



−1

0

1

2h=0.3 (explizites Euler−Verfahren)

−1

0

1


0 1 2 3 4 5−1

0

1




−1

0

1

2h=0.3 (implizites Euler−Verfahren)

−1

0

1

2h=0.5 (implizites Euler−Verfahren)

0 1 2 3 4 5−1

0

1

2h=1 (implizites Euler−Verfahren)



3.3 Further Stability Concepts

Recall. A numerical scheme for solving y ′ = f (t,y), y(0) = y0, is absolu-tely stable (A-stable), when the following holds:

When applied to a linear constant-coefficient problem

y ′ = Ay , t ∈ [0,∞),

for which all eigenvalues of A lie in the left half-plane {ζ : Re ζ < 0}, theapproximate solutions yn satisfy limn→∞ ‖yn‖ = 0.(Note that limt→∞ ‖y(t)‖ = 0 for the exact solution y .)

This is equivalent with

RA ⊇ {h : Re h < 0}.

The region of absolute stability RA was defined in §3.1.

3.3 Further Stability Concepts TU Bergakademie Freiberg, SS 2012


Some weaker stability concepts:

• A(α)-stable for an α ∈ (0, π/2) :⇔ RA ⊇ {h : −α < π − arg h < α} ,

• A0-stable :⇔ RA ⊇ (−∞, 0),

• stiffly-stable :⇔ RA ⊇ R1(β) ∪R2(β, γ) for β and γ positive, where

R1(β) := {h : Re h < −β} and

R2(β, γ) := {h : −β ≤ Re h < 0, | Im h| ≤ γ}

In addition, a OSM ist called

• L-stable, if it is A-stable and, in addition: when applied to the test equa-tion y′ = λy, Re λ < 0, yields approximations yn+1 = R(hλ)yn withlimhλ→−∞ |R(hλ)| = 0.



A−stabil

Real ζ

0

Imag ζ

A(α)−stabil

Real ζ

0

Imag ζ

α

α

steif−stabil

Real ζ

0

Imag ζ

γ

−γ

−β



Obviously: (0 < α ≤ arg(β + iγ))

L-stable⇒ A-stable⇒ stiffly-stable⇒ A(α)-stable⇒ A0-stable.

The trapezoidal rule

yn+1 = yn +h

2(fn + fn+1), d.h. R(h) =

1 + h/2

1− h/2,

ist A-stabil, but not L-stable.

By contrast, the implicit Euler method

yn+1 = yn + hfn+1, d.h. R(h) =1

1− h,

is L-stable.



We illustrate the difference by applying both methods to (IVP2) from § 3.2.To activate the transient component we select the initial data y(0) = [0, 0]>,yielding the exact solution

y(t) = −1999 exp(−t)

[1

1

]+ 1

999 exp(−1000t)

[1

−998

]+

[sin t

cos t

].

The following figure shows the second component of the approximatesolution using h = 0.2.



0 2 4 6 8 10−2

−1.5

−1

−0.5

0

0.5

1

1.5

2h=0.2

exakte Loesung Trapezregel implizites Euler−Verfahren



The example

y ′ =

42.2 50.1 −42.1−66.1 −58.0 58.1

26.1 42.1 −34.0

y , y(0) =

1

0

2

,with exact solution

y(t) =

exp(0.1t) sin(8t) + exp(−50t)exp(0.1t) cos(8t)− exp(−50t)

exp(0.1t) (sin(8t) + cos(8t)) + exp(−50t)

shows that L-stable methods are not always better than A-stable methods.The following figure shows the first component of the approximate solutionfor h = 0.04 and t ∈ [0, 1.5]).

Using implicit Euler would require choosing h < 0.0031 . . . to obtain anacceptable solution. (why?)



0 0.5 1 1.5−1.5

−1

−0.5

0

0.5

1

1.5h=0.04

t

y 1

exact solutiontrapezoidal ruleimplicit Euler


Documents

3 Stability and Stiffness - TU Bergakademie Freibergernst/Lehre/Numerik_II/Folien/nade... · 3 Stability and Stiffness TU Bergakademie Freiberg, ... Finally, ode23t is an absolutely