Numerical Programming 2 (CSE) - Angewandte Numerische Analysis · I stability issues and analysis by the Fourier method I nite di erence for the advection equation I Lax-Wendro method

Numerical Programming 2 (CSE)

Massimo Fornasier

Fakultat fur MathematikTechnische Universitat [email protected]://www-m15.ma.tum.de/

Johann Radon Institute (RICAM)Osterreichische Akademie der Wissenschaften

[email protected]://hdsparse.ricam.oeaw.ac.at/

Department of MathematicsTechnische Universitat Munchen

Lecture 2

Why are you here?In this course we shall provides the rudiments (the basics!) of thenumerical solution of (partial) differential equations, i.e, equationsinvolving the derivatives of a (possibly) multivariate functionu : Ω ⊂ Rd → R:

F (x1, . . . , xd , u(x),∂

∂x1u(x), . . . ,

∂

∂xdu(x),

∂2

∂x1∂x1,

∂2

∂x1∂x2, . . . ,

∂2

∂x1∂xd, . . . ) = 0.

Often one of the variables indicates the time, e.g., x1 = t, and theequation governs a phenomena which evolves in time:

∂u

∂t+ F (x , u,Du,D2u, . . . ) = 0.

Differential equations were for thefirst time formulated in the 17th

century by I. Newton (1671) and byG.W. Leibniz (1676).


F (x1, . . . , xd , u(x),∂

∂x1u(x), . . . ,

∂

∂xdu(x),

∂2

∂x1∂x1,

∂2

∂x1∂x2, . . . ,

∂2

∂x1∂xd, . . . ) = 0.


∂u

∂t+ F (x , u,Du,D2u, . . . ) = 0.




F (x1, . . . , xd , u(x),∂

∂x1u(x), . . . ,

∂

∂xdu(x),

∂2

∂x1∂x1,

∂2

∂x1∂x2, . . . ,

∂2

∂x1∂xd, . . . ) = 0.


∂u

∂t+ F (x , u,Du,D2u, . . . ) = 0.



Just old boring stuff?

Yes, of course!?But also NO, because the history did not stop ...

used by Euler, Maxwell, Boltzmann, Navier, Stokes, Einstein,Prandtl, Schrodinger, Pauli, Dirac,Turing, Black&Scholes ...........

I provide an important modeling tool for the physical sciences,theoretical chemistry, biology, socio-economic sciences,engineering sciences ....

I accessible to numerical simulations


Yes, of course!?

But also NO, because the history did not stop ...














For instance ...Fourier in Theorie analytique de la chaleur (1822) formulated and solvedanalytically the equation governing the heat conduction in homogenous media:

∂u

∂t(t, x)− α

„∂2u

∂x21

(t, x) +∂2u

∂x22

(t, x) +∂2u

∂x23

(t, x)

«= 0.

A similar equation is studied by Fornasier and March in 2007 to recolor ancientItalian frescoes from fragments:

See Restoration of color images by vector valued BV functions and variationalcalculus (M. Fornasier and R. March), SIAM J. Appl. Math., Vol. 68 No. 2,2007, pp. 437-460.

http://www.ricam.oeaw.ac.at/people/page/fornasier/SIAM_Fornasier_

March_067187.pdf

http://www.ricam.oeaw.ac.at/people/page/fornasier/SIAM_Fornasier_March_067187.pdf


For instance ...Fourier in Theorie analytique de la chaleur (1822) formulated and solvedanalytically the equation governing the heat conduction in homogenous media:

∂u

∂t(t, x)− α

„∂2u

∂x21

(t, x) +∂2u

∂x22

(t, x) +∂2u

∂x23

(t, x)

«= 0.

A similar equation is studied by Fornasier and March in 2007 to recolor ancientItalian frescoes from fragments:

See Restoration of color images by vector valued BV functions and variationalcalculus (M. Fornasier and R. March), SIAM J. Appl. Math., Vol. 68 No. 2,2007, pp. 437-460.

http://www.ricam.oeaw.ac.at/people/page/fornasier/SIAM_Fornasier_

March_067187.pdf



Illustrations of differential equations and some of their(modern) applications

Reference: P. A. Markowich, Applied Partial Differential Equations - A Visual

Approach, Springer, 2006

Slides: http:

//homepage.univie.ac.at/peter.markowich/galleries/vortrag.pdf

I Gas dynamics Boltzmann equation

I Fluid/gas dynamics: Navier-Stokes/Euler Equations

I Kinetic modeling of granular flows

I Chemotaxis and formation of biological patterns

I Semiconductor modeling

I Free boundary problems and interfaces

I Reaction-diffusion equations

I Monge-Kantorovich optimal transportation

I Wave equations

I Digital image processing

I Socio-Economic modeling

http://homepage.univie.ac.at/peter.markowich/galleries/vortrag.pdf


Illustrations of differential equations and some of their(modern) applications

Reference: P. A. Markowich, Applied Partial Differential Equations - A Visual

Approach, Springer, 2006

Slides: http:

//homepage.univie.ac.at/peter.markowich/galleries/vortrag.pdf

I Gas dynamics Boltzmann equation

I Fluid/gas dynamics: Navier-Stokes/Euler Equations

I Kinetic modeling of granular flows

I Chemotaxis and formation of biological patterns

I Semiconductor modeling

I Free boundary problems and interfaces

I Reaction-diffusion equations

I Monge-Kantorovich optimal transportation

I Wave equations

I Digital image processing

I Socio-Economic modeling



From partial differential equations to ordinary differentialequations

As we shall see in the course of this lecture, several partialdifferential equations (PDE) can be reduced, after discretization ofthe space variable, to a system of ordinary differential equations(ODE):

∂u

∂t+ F (t, x , u,Du,D2u, . . . ) = 0⇒ u′h(t) + Fh(u(t)) = 0,

where h is a space discretization parameter.

Hence, the first part of this course is dedicated to the numericalsolution of ODE, while the second part of the course is dedicatedto the solution of some relevant equations of elliptic (e.g., materialelasticity), parabolic (e.g., heat conduction), hyperbolic (e.g., waveevolution) types.



∂u






∂u




Program of the course in a nutshellODE:

I forward Euler method

I theta method

I Adams methods and more general multi-step methods

I Runge-Kutta methods

I stability and stiffness

I Backward differentiation formulae

I Predictor-corrector methods

PDE:

I finite difference

I numerical solution of the Poisson equation in one dimension

I Poisson equation in two dimensions: five-point method

I finite element method

I spectral method

I finite difference for parabolic equations: the heat equation

I stability issues and analysis by the Fourier method

I finite difference for the advection equation

I Lax-Wendroff method for hyperbolic equations

I hyperbolic systems

Program of the course in a nutshellODE:

I forward Euler method

I theta method

I Adams methods and more general multi-step methods

I Runge-Kutta methods

I stability and stiffness

I Backward differentiation formulae

I Predictor-corrector methods

PDE:

I finite difference

I numerical solution of the Poisson equation in one dimension

I Poisson equation in two dimensions: five-point method

I finite element method

I spectral method

I finite difference for parabolic equations: the heat equation

I stability issues and analysis by the Fourier method

I finite difference for the advection equation

I Lax-Wendroff method for hyperbolic equations

I hyperbolic systems

References for this course

Besides the slides provided after the lecture online ...

Books:

I09 A. Iserles, A First Course in the Numerical Analysis of DifferentialEquations (2nd ed.), Cambridge University Press, 2009.

QSG10 A. Quarteroni, F. Saleri, P. Gervasio, Scientific Computing with Matlaband Octave (3rd ed.), Springer, 2010.

Lecture notes:

F13 M. Fornasier, Numerik der gewohnlichen Differentialgleichungen,http://www-m3.ma.tum.de/foswiki/pub/M3/NumerikDG13/WebHome/

Numerik_2013-08-04.pdf (password required)

QSG10 C. Lasser, Numerical Programming 2 (CSE), synthesis of class lectures,http://www-m3.ma.tum.de/foswiki/pub/M3/Allgemeines/

NumericalCSE12/CSE_SS12.pdf

All you find at the webpage of the course:

http://www-m15.ma.tum.de/Allgemeines/NumericalProgramming

http://www-m3.ma.tum.de/foswiki/pub/M3/NumerikDG13/WebHome/Numerik_2013-08-04.pdf


http://www-m3.ma.tum.de/foswiki/pub/M3/Allgemeines/NumericalCSE12/CSE_SS12.pdf




Besides the slides provided after the lecture online ...Books:



Lecture notes:













Besides the slides provided after the lecture online ...Books:



Lecture notes:












References do not mean ...

that you do not come to the lecture and think to be smart ...

’cause I will NOT let you escape ...

At the end there will be the exam ... and I’ll be waiting for youthere (REMEMBER IT!) ... you better show up! (just a friendly -Italian - invitation not to skip the lectures :-) !)








At the end there will be the exam ...

and I’ll be waiting for youthere (REMEMBER IT!) ... you better show up! (just a friendly -Italian - invitation not to skip the lectures :-) !)




At the end there will be the exam ... and I’ll be waiting for youthere (REMEMBER IT!) ...

you better show up! (just a friendly -Italian - invitation not to skip the lectures :-) !)





Let’s start ... why ODE?

As just mentioned, ODE comes often after the space discretizationof PDE.

But ODE have their own dignity and relevance, since, actually,PDE can be derived as the “limit” of sytems of ODE governing theevolution of particles driven by Newton laws:

F = m · a.

Hence, ODE ⇒ PDE ⇒ ODE.




F = m · a.





F = m · a.


Particle systemsBesides in physics, large particle systems arise in many modernapplications:

Image halftoning via variational

dithering.

Dynamical data analysis: R. palustris

protein-protein interaction network.

Large Facebook “friendship” network

Computational chemistry: molecule

simulation.

Social dynamics

We consider large particle systems ofform:

xi = vi ,

vi =∑N

j=1 H(xj − xi , vj − vi ),

Several“social forces” encoded in theinteraction kernel H:

I Repulsion-attraction

I Alignment

I ...

Possible noise/uncertainty by addingstochastic terms.

Patterns related to different balance of

social forces.

Understanding how superposition of re-iterated binary “socialforces” yields global self-organization.

Social dynamics

We consider large particle systems ofform:

xi = vi ,

vi =∑N

j=1 H(xj − xi , vj − vi ),

Several“social forces” encoded in theinteraction kernel H:

I Repulsion-attraction

I Alignment

I ...

Possible noise/uncertainty by addingstochastic terms.

Patterns related to different balance of

social forces.

Understanding how superposition of re-iterated binary “socialforces” yields global self-organization.

An example inspired by nature

Mills in nature and in our simulations.

J. A. Carrillo, M. Fornasier, G. Toscani, and F. Vecil, Particle, kinetic,

hydrodynamic models of swarming, within the book “Mathematical modeling

of collective behavior in socio-economic and life-sciences”, Birkhauser (Eds.

Lorenzo Pareschi, Giovanni Naldi, and Giuseppe Toscani), 2010.

http://www.ricam.oeaw.ac.at/people/page/fornasier/bookfinal.pdf

http://www.ricam.oeaw.ac.at/people/page/fornasier/bookfinal.pdf

The genearal ODEOur goal is to approximate the solution of the problem

y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)

(In the previous modeling, yi = (xi , vi ) andf (t, y)i = (vi ,

∑Nj=1 H(xj − xi , vj − vi )).)

I f is a map of [t0,∞)× Rd to Rd and the initial conditiony0 ∈ Rd is a given vector

I f is “nice”, obeying, in a given vector norm, the Lipschitzcondition

‖f (t, x)− f (t, y)‖ ≤ λ‖x − y‖, ∀x , y ∈ Rd , t ≥ t0 (2)

Here λ > 0 is a real constant that is independent of thechoice of x and y . In this case we say that f is a Lipschitzcontinuous function.

I Subject to (2), it is possible to prove that the ODE system(1) possesses a unique solution (see, for instance [Theorem3.5, F13] pag. 52).


y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)


∑Nj=1 H(xj − xi , vj − vi )).)







y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)


∑Nj=1 H(xj − xi , vj − vi )).)







y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)


∑Nj=1 H(xj − xi , vj − vi )).)




Here λ > 0 is a real constant that is independent of thechoice of x and y .

In this case we say that f is a Lipschitzcontinuous function.



y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)


∑Nj=1 H(xj − xi , vj − vi )).)







y ′ = f (t, y), t ≥ t0, y(t0) = y0. (1)


∑Nj=1 H(xj − xi , vj − vi )).)






Just an idea of the proof of existence and uniqueness(Picard 1890, Lindelof 1894)

Actually (1) can be rewritten by integration

y(t) = y0 +

∫ t

t0

f (s, y(s))ds = F (y(t)). (3)

Hence, a solution y(t) is a fixed point trajectory of the equationy = F (y). How can one solve fixed point equations? Well, if Fwere a contraction, i.e., F is a Lipschitz continuous function withLipschitz constant 0 < Λ < 1, then the iteration

yn+1 = F (yn), n ≥ 0 (4)

converges always to the unique fixed point!Indeed, assume thatsuch fixed point y exists then

‖y−yn+1‖∗ = ‖F (y)−F (yn)‖∗ ≤ Λ‖y−yn‖∗ ≤ Λn‖y−y0‖∗ → 0, n→∞,

because Λ < 1. The tricky part of the proof is to show that thereexists a fixed point and that there exists always a norm ‖ · ‖∗ whichmakes F a contraction as soon as f is Lipschitz continuous.



y(t) = y0 +

∫ t

t0

f (s, y(s))ds = F (y(t)). (3)


yn+1 = F (yn), n ≥ 0 (4)

converges always to the unique fixed point!

Indeed, assume thatsuch fixed point y exists then





y(t) = y0 +

∫ t

t0

f (s, y(s))ds = F (y(t)). (3)


yn+1 = F (yn), n ≥ 0 (4)



because Λ < 1.

The tricky part of the proof is to show that thereexists a fixed point and that there exists always a norm ‖ · ‖∗ whichmakes F a contraction as soon as f is Lipschitz continuous.



y(t) = y0 +

∫ t

t0

f (s, y(s))ds = F (y(t)). (3)


yn+1 = F (yn), n ≥ 0 (4)




Did you noticed that ...

... what we wrote in (4) is actually an ALGORITHM?!

OMG! Really, an ALGORITHM?!

... is it a bad thing?!Actually, no, that’s what you are here for! Or, perhaps yes ... Ohwell, nevertheless, an ALGORITHM!

For the brave student: if you find out a way to discretize (4) and a way of

properly “scaling” the equation so that the iteration always converges on a

finite set of time point t0 < t1 < · < tn, then let me know! Actually this course

is (IMPLICITLY) all about this!

Did you noticed that ...... what we wrote in (4) is actually an ALGORITHM?!
















... is it a bad thing?!

Actually, no, that’s what you are here for! Or, perhaps yes ... Ohwell, nevertheless, an ALGORITHM!







... is it a bad thing?!Actually, no, that’s what you are here for!

Or, perhaps yes ... Ohwell, nevertheless, an ALGORITHM!







... is it a bad thing?!Actually, no, that’s what you are here for! Or, perhaps yes ...

Ohwell, nevertheless, an ALGORITHM!












From Picard-Lindelof back to EulerInstead of solving globally the fixed point equation (3) by amultitude of iterations (4), we may consider the simpler idea ofsolving it locally, step by step, by iterating the approximation

y(t) = y0+

∫ t

t0

f (s, y(s))ds ≈ y(t0)+(t−t0)f (t0, y(t0)), for t ≈ t0.

(5)Given a sequence t0, t1 = t0 + h, t2 = t0 + 2h, ... , where h > 0 isa time step, we denote by yn a numerical estimate of the exactsolution y(tn), n = 0, 1, . . . .

Motivated by (5), we choose

y1 = y0 + hf (t0, y0).

If h is small, it should not be that wrong! But then, why not tocontinue, assuming that we did not that bad before, at t2, t3 andso on. In general, we obtain the recursive scheme

yn+1 = yn + hf (tn, yn), (6)

the celebrated Euler method.


y(t) = y0+

∫ t

t0


(5)Given a sequence t0, t1 = t0 + h, t2 = t0 + 2h, ... , where h > 0 isa time step, we denote by yn a numerical estimate of the exactsolution y(tn), n = 0, 1, . . . . Motivated by (5), we choose

y1 = y0 + hf (t0, y0).


yn+1 = yn + hf (tn, yn), (6)



y(t) = y0+

∫ t

t0



y1 = y0 + hf (t0, y0).

If h is small, it should not be that wrong!

But then, why not tocontinue, assuming that we did not that bad before, at t2, t3 andso on. In general, we obtain the recursive scheme

yn+1 = yn + hf (tn, yn), (6)



y(t) = y0+

∫ t

t0



y1 = y0 + hf (t0, y0).


yn+1 = yn + hf (tn, yn), (6)


Graphic interpretationConsider the Euler method applied to the logistic equation

y ′ = y(1− y), y(0) =1

10,

with step h = 1:

It’s clear that at each step we produce an error, but our goal is not to avoid

any (numerical error)! Our final goal is to have a practical method that

approximates the analytic solution with increasing accuracy (i.e., decreasing

error) the more computational effort we do.


y ′ = y(1− y), y(0) =1

10,

with step h = 1:


any (numerical error)!

Our final goal is to have a practical method that




y ′ = y(1− y), y(0) =1

10,

with step h = 1:


any (numerical error)! Our final goal is to have a practical method that



Convergence of a numerical method

Assume that h > 0 is variable and h→ 0. On each grid t0,t1 = t0 + h, t2 = t0 + 2h, ... we associate a different numericalsequence yn = yn,h, n = 0, 1, . . . , bt∗/hc (not necessarily producedby the Euler method!).

A method is said to be convergent if, for every ODE (1) with aLipschitz function f and every t∗ > 0 it is true that

limh→0

maxn=0,1,...,bt∗/hc

‖yn,h − y(tn)‖ = 0,

where bαc ∈ Z is the integer part of α ∈ R.

Convergence means that, for every Lipschitz function,the numerical solution tends to the true solution as thegrid becomes increasingly fine.




limh→0


‖yn,h − y(tn)‖ = 0,






limh→0


‖yn,h − y(tn)‖ = 0,



Convergence of the Euler method

TheoremThe Euler method (6) is convergent

Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.)Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.Taylor expansion of y(t)

y(tn+1) = y(tn) + hy ′(tn) +O(h2) = y(tn) + hf (tn, y(tn)) +O(h2).

Subtracting this equation to (6), we obtain

en+1,h = en,h + h[f (tn, yn)− f (tn, y(tn))] +O(h2).

Triangle inequality and (2) imply

‖en+1,h‖ ≤ ‖en,h‖+ h‖f (tn, yn)− f (tn, y(tn))‖+ ch2

≤ (1 + hλ)‖en,h‖+ ch2.

By induction over this estimate we get

‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .



Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.)

Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.Taylor expansion of y(t)






≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .



Proof. (For this proof we assume that the Taylor expansion of f has uniformlybounded coefficients, i.e., f is analytic, implying that y is analytic as well.)Letus consider the error en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.

Taylor expansion of y(t)






≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .









≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .









≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .









≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .









≤ (1 + hλ)‖en,h‖+ ch2.


‖en,h‖ ≤c

λh[(1 + hλ)n − 1], n = 0, 1, . . .

Convergence of the Euler method continues ...

We notice now that 1 + ξ ≤ eξ for all ξ > 0, hence (1 + hλ) ≤ ehλ and(1 + hλ)n ≤ ehnλ. But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying

(1 + hλ)n ≤ et∗λ

and‖en,h‖ ≤

h c

λ(et∗λ − 1)

i· h. (7)

Sinceh

cλ

(et∗λ − 1)i

is independent of n and h, it follows

limh→0

max0≤nh≤t∗

‖en,h‖ = 0.

The error bound in (7) tells us that actually Euler’s methodconverges with order q = 1 since it decays as O(hq). However,letus stress that the constant

[cλ(et∗λ − 1)

]is by far over-pessimistic

and should not be used for numerical puroposes (it’s justtheoretical!).



(1 + hλ)n ≤ et∗λ

and‖en,h‖ ≤

h c

λ(et∗λ − 1)

i· h. (7)

Sinceh

cλ

(et∗λ − 1)i


limh→0

max0≤nh≤t∗

‖en,h‖ = 0.


[cλ(et∗λ − 1)





(1 + hλ)n ≤ et∗λ

and‖en,h‖ ≤

h c

λ(et∗λ − 1)

i· h. (7)

Sinceh

cλ

(et∗λ − 1)i


limh→0

max0≤nh≤t∗

‖en,h‖ = 0.


[cλ(et∗λ − 1)





(1 + hλ)n ≤ et∗λ

and‖en,h‖ ≤

h c

λ(et∗λ − 1)

i· h. (7)

Sinceh

cλ

(et∗λ − 1)i


limh→0

max0≤nh≤t∗

‖en,h‖ = 0.

The error bound in (7) tells us that actually Euler’s methodconverges with order q = 1 since it decays as O(hq).

However,letus stress that the constant

[cλ(et∗λ − 1)





(1 + hλ)n ≤ et∗λ

and‖en,h‖ ≤

h c

λ(et∗λ − 1)

i· h. (7)

Sinceh

cλ

(et∗λ − 1)i


limh→0

max0≤nh≤t∗

‖en,h‖ = 0.


[cλ(et∗λ − 1)



Example

Let’s consider

y ′ = −100y , y(0) = 1, t∗ = 1.

The error estimate (7) would give us the following bound on theerror

|en,h| ≤ 2.69× 1045h,

while the exact and numerical solutions are respecivelyy(t) = e−100t and yn = (1− 100h)n, giving

|yn − yn,h| = |(1− 100h)n − e−100nh|,

which is by far smaller than 2.69× 1045h!

Example

Let’s consider

y ′ = −100y , y(0) = 1, t∗ = 1.


|en,h| ≤ 2.69× 1045h,


|yn − yn,h| = |(1− 100h)n − e−100nh|,


Example

Let’s consider

y ′ = −100y , y(0) = 1, t∗ = 1.


|en,h| ≤ 2.69× 1045h,


|yn − yn,h| = |(1− 100h)n − e−100nh|,


Consistency error and orderEuler’s method can be rewritten

yn+1 − yn − hf (tn, yn) = 0.

How well this equation represents the original solution?

Let’s plug the originalsolution in this equation

y(tn+1)− y(tn)− hf (tn, y(tn)) = y(tn+1)− y(tn)− hy ′(tn) = O(h2), (8)

by Taylor’s expansion error. This residual error O(h2) is called the consistencyerror. In general, given an arbitrary time-stepping method

yn+1 = Yn(f , h, y0, y1, . . . , yn),

for the ODE (1), we say that it is of consistency order p if

y(tn+1)− Yn(f , h, y(t0), y(t1), . . . , y(tn)) = O(hp+1).

for every analytic f and n = 0, 1, . . . . Hence, by (8) Euler’s method hasconsistency order 1!

Never confuse the order of convergence with the consistency order! They may

well be different! A method can have a positive consistency order but not be

convergent! (see later)


yn+1 − yn − hf (tn, yn) = 0.

How well this equation represents the original solution? Let’s plug the originalsolution in this equation


by Taylor’s expansion error. This residual error O(h2) is called the consistencyerror.

In general, given an arbitrary time-stepping method

yn+1 = Yn(f , h, y0, y1, . . . , yn),








yn+1 − yn − hf (tn, yn) = 0.




yn+1 = Yn(f , h, y0, y1, . . . , yn),



for every analytic f and n = 0, 1, . . . .

Hence, by (8) Euler’s method hasconsistency order 1!





yn+1 − yn − hf (tn, yn) = 0.




yn+1 = Yn(f , h, y0, y1, . . . , yn),








yn+1 − yn − hf (tn, yn) = 0.




yn+1 = Yn(f , h, y0, y1, . . . , yn),







How to implement Euler’s method in MATLAB

Try out Euler’s method on the logistic equation:

f = @(t,x) x.*(1-x); [t,yy]=ode45(f,[0:5],1/10); y(1)=1/10;h=1; for n=1:5, y(n+1)=y(n)+h*y(n)*(1-y(n)); end,plot([0:5],y,’r*-’,[0:5],yy,’b’) h=1/2; for n=1:10,y(n+1)=y(n)+h*y(n)*(1-y(n)), end, hold on,plot([0:0.5:5],y,’g-*’)

How does Euler’s method discretize a PDE?

Given∂u

∂t− F (t, x , u,Du,D2u, . . . ) = 0

one considers

un+1i − un

i

∆t= Fi (tn, x , u,Du,D2u, . . . ).

Trapezoidal rule alias Crank-Nicolson method

Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn.

Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) ) and it makes more sense to make theconstant approximation of the derivative equal to the average of itsvalues at the endpoints:

y(t) = y0+

∫ t

t0

f (s, y(s))ds ≈ y(t0)+1

2(t−t0)[f (t0, y(t0))+f (t, y(t))],

(9)for t ≈ t0. This brings us to consider the trapezoidal rule or theCrank-Nicolson method

yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]. (10)


Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn. Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) )

and it makes more sense to make theconstant approximation of the derivative equal to the average of itsvalues at the endpoints:

y(t) = y0+

∫ t

t0

f (s, y(s))ds ≈ y(t0)+1

2(t−t0)[f (t0, y(t0))+f (t, y(t))],


yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]. (10)


Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn. Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) ) and it makes more sense to make theconstant approximation of the derivative equal to the average of itsvalues at the endpoints:

y(t) = y0+

∫ t

t0

f (s, y(s))ds ≈ y(t0)+1

2(t−t0)[f (t0, y(t0))+f (t, y(t))],

(9)for t ≈ t0.

This brings us to consider the trapezoidal rule or theCrank-Nicolson method

yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]. (10)


Euler’s method approximates the derivative by a constant in[tn, tn+1], namely by its value at tn. Clearly, the “leftist”approximation is not very good (it’s NOT a political comment!Left can be good :-) ) and it makes more sense to make theconstant approximation of the derivative equal to the average of itsvalues at the endpoints:

y(t) = y0+

∫ t

t0

f (s, y(s))ds ≈ y(t0)+1

2(t−t0)[f (t0, y(t0))+f (t, y(t))],


yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]. (10)

Consistency order of the Crank-Nicolson method

To obtain the order of (10), we substitute the exact solution,

y(tn+1)− y(tn)− h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))]

= [y(tn) + hy ′(tn) +h2

2y ′′(tn) +O(h3)]

− (y(tn) +h

2y ′(tn) + [y ′(tn) + hy ′′(tn) +O(h2)]) = O(h3).

Therefore the trapezoidal rule is of order 2.

This is very good, butwe should be careful (as we shall see later) already to concludethat the method is convergent. Actually we need to prove it.

Consistency order of the Crank-Nicolson method

To obtain the order of (10), we substitute the exact solution,

y(tn+1)− y(tn)− h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))]

= [y(tn) + hy ′(tn) +h2

2y ′′(tn) +O(h3)]

− (y(tn) +h

2y ′(tn) + [y ′(tn) + hy ′′(tn) +O(h2)]) = O(h3).

Therefore the trapezoidal rule is of order 2.This is very good, butwe should be careful (as we shall see later) already to concludethat the method is convergent. Actually we need to prove it.

Convergence of the Crank-Nicolson method

TheoremThe Crank-Nicolson method (10) is convergent.

Proof. (For this proof we assume again that f is analytic.) Let us consider theerror en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0. Substracting

y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).

(see for instance [Section 2.2.2, F13] pag. 21 for this asymptotic error) to (10),we obtain

‖en+1,h‖ ≤ ‖en,h‖+h

2[‖f (tn, yn)− f (tn, y(tn))‖+ ‖f (tn+1, yn+1)− f (tn+, y(tn+1))‖]

+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.

By assumption that h > 0 gets so small that hλ < 2, we get

‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.



Proof. (For this proof we assume again that f is analytic.)

Let us consider theerror en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0. Substracting

y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).


‖en+1,h‖ ≤ ‖en,h‖+h


+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.


‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.



Proof. (For this proof we assume again that f is analytic.) Let us consider theerror en,h = yn − y(tn). We shall prove limh→0 ‖en,h‖ = 0.

Substracting

y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).


‖en+1,h‖ ≤ ‖en,h‖+h


+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.


‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.




y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).

(see for instance [Section 2.2.2, F13] pag. 21 for this asymptotic error)

to (10),we obtain

‖en+1,h‖ ≤ ‖en,h‖+h


+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.


‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.




y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).


‖en+1,h‖ ≤ ‖en,h‖+h


+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.


‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.




y(tn+1) = y(tn) +h

2[f (tn, y(tn)) + f (tn+1, y(tn+1))] +O(h3).


‖en+1,h‖ ≤ ‖en,h‖+h


+ ch3

≤ ‖en,h‖+hλ

2[‖en,h‖+ ‖en+1,h‖] + ch3.


‖en+1,h‖ ≤

1 + 1

2hλ

1− 12hλ

!‖en,h‖+

c

1− 12hλ

!h3.

Convergence of the Crank-Nicolson method continues ...


‖en,h‖ ≤c

λh2

" 1 + 1

2hλ

1− 12hλ

!n

− 1

#, n = 0, 1, . . .

Since 0 < hλ < 2, it is true that 1 + 1

2hλ

1− 12hλ

!= 1 +

hλ

1− 12hλ

!≤∞X`=0

1

`!

hλ

1− 12hλ

!`= exp

hλ

1− 12hλ

!.

But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying

‖en,h‖ ≤

"c

λexp

t∗λ

1− 12hλ

!#· h2. (11)

It followslimh→0

max0≤nh≤t∗

‖en,h‖ = 0.

The error bound in (11) tells us that actually the methodconverges with order q = 2 since it decays as O(hq).

Convergence of the Crank-Nicolson method continues ...


‖en,h‖ ≤c

λh2

" 1 + 1

2hλ

1− 12hλ

!n

− 1

#, n = 0, 1, . . .

Since 0 < hλ < 2, it is true that 1 + 1

2hλ

1− 12hλ

!= 1 +

hλ

1− 12hλ

!≤∞X`=0

1

`!

hλ

1− 12hλ

!`= exp

hλ

1− 12hλ

!.

But n = 0, 1, . . . , bt∗/hc and n ≤ t∗/h, implying

‖en,h‖ ≤

"c

λexp

t∗λ

1− 12hλ

!#· h2. (11)

It followslimh→0

max0≤nh≤t∗

‖en,h‖ = 0.

The error bound in (11) tells us that actually the methodconverges with order q = 2 since it decays as O(hq).

But not so easy ...Notice that the Crank-Nicolson method

yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]︸︷︷︸

:=Fn(yn+1)

.

does not allow for the explicit/direct computation of yn+1 from yn

as it occurs for (6).

Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (10) in the knownyn+1. How can we do it? Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute

yn+1 = limm→∞

ymn+1,

where the sequence (ymn+1)m is generated (similarly to the

Picard-Lindelof iteration!!) by

ym+1n+1 = Fn(ym

n+1), m ≥ 0.


yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]︸︷︷︸

:=Fn(yn+1)

.


as it occurs for (6). Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (10) in the knownyn+1.

How can we do it? Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute

yn+1 = limm→∞

ymn+1,



ym+1n+1 = Fn(ym

n+1), m ≥ 0.


yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]︸︷︷︸

:=Fn(yn+1)

.


as it occurs for (6). Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (10) in the knownyn+1. How can we do it?

Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute

yn+1 = limm→∞

ymn+1,



ym+1n+1 = Fn(ym

n+1), m ≥ 0.


yn+1 = yn +h

2[f (tn, yn) + f (tn+1, yn+1)]︸︷︷︸

:=Fn(yn+1)

.


as it occurs for (6). Indeed it is a so-called implicit method and itrequires the solution of the nonlinear equation (10) in the knownyn+1. How can we do it? Well, for instance, we observe thatactaully yn+1 = Fn(yn+1) is a fixed point of an equation and thatFn is a contraction for h sufficiently small! Therefore we cancompute

yn+1 = limm→∞

ymn+1,



ym+1n+1 = Fn(ym

n+1), m ≥ 0.

Newton ... again

The solution of the equation

0 = G (yn+1) = yn+1 − Fn(yn+1)

can also be solved by Newton’ method:

ym+1n+1 = ym

n+1 −∇G (ymn+1)−1 · G (ym

n+1), m = 0, 1, . . .

where ∇G (ymn+1) is the Jacobian matrix of G computed at ym

n+1.Again we can compute

yn+1 = limm→∞

ymn+1,

Newton ... again


0 = G (yn+1) = yn+1 − Fn(yn+1)


ym+1n+1 = ym

n+1 −∇G (ymn+1)−1 · G (ym

n+1), m = 0, 1, . . .


n+1.

Again we can compute

yn+1 = limm→∞

ymn+1,

Newton ... again


0 = G (yn+1) = yn+1 − Fn(yn+1)


ym+1n+1 = ym

n+1 −∇G (ymn+1)−1 · G (ym

n+1), m = 0, 1, . . .


n+1.Again we can compute

yn+1 = limm→∞

ymn+1,

How to implement the Crank-Nicolson method inMATLAB?

Try this out:

f = @(t,x) -x + 2*exp(-t)*cos(2*t); t(1)=0; x(1)=0;y(1)=0; h=1/2; for n=1:(10/h), y(n+1)=fsolve(@(x)x-y(n)-h/2*(f(t(n),y(n))+f(t(n+1),x)),y(n)); end,semilogy(t,abs(y-xx),’k’,t, h2*ones(size(t)),’r:’)

How does the Crank-Nicolson method discretize a PDE?

Given∂u

∂t− F (t, x , u,Du,D2u, . . . ) = 0

one considers

un+1i − un

i

∆t=

1

2[Fi (tn, x , u,Du,D2u, . . . )+Fi (tn+1, x , u,Du,D2u, . . . )]

Documents

Numerical Programming 2 (CSE) - Angewandte Numerische Analysis · I stability issues and analysis by the Fourier method I nite di erence for the advection equation I Lax-Wendro method