26
Scientific computing III 2011: 13. Differential equations 1 Differential equations • A differential equation (DE) describes how a quantity changes (as a function of time, position, ...). - A ball dropped from a building: (1) - Uniformly loaded beam: (2) - Schrödinger equation: (3) - Heat conduction: (4) . . . - DEs can be classified in various ways: Ordinary DEs (ODEs) Partial DEs (PDEs) Only one independent variable Many independent variables Eqs (1) and (2) Eqs (3) and (4) Initial value problems (IVP) Boundary value problems (BVP) Function value(s) known at certain values of the variable(s). Function value(s) known on the domain borders. Want function value(s) at arbitrary variable value(s). Want function values in other point(s) in the domain. Eqs (1) and (4) Eqs (2), (3) and (4) In 1D: , compute . In 1D: , , compute . (two-point BVP) t d d ht gt = x 2 2 d d wx S EI ----- wx qx 2 EI -------- x L + = h 2 2 2 m -------------------- 2 r V r r + E r = c P t T r t k T r t = ft 0 y 0 = ft fx 1 y 1 = fx 2 y 2 = fx Scientific computing III 2011: 13. Differential equations 2 Differential equations • The difference between initial value problems and boundary value problems can be illustrated as below (according to NR, fig. 19.0.1). - In general boundary value problems are harder to solve than initial value problems: BVPs need iteration IVP you ‘only’ need to integrate the equations in ‘time’ to reach the desired point. - In this chapter we deal with ordinary differential equations both IVPs and BVPs. - Probably there will be no time partial differential equations. initial values boundary conditions boundary conditions boundary values

Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

  • Upload
    others

  • View
    10

  • Download
    0

Embed Size (px)

Citation preview

Page 1: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 1

Differential equations

• A differential equation (DE) describes how a quantity changes (as a function of time, position, ...).

- A ball dropped from a building: (1)

- Uniformly loaded beam: (2)

- Schrödinger equation: (3)

- Heat conduction: (4)

. . .

- DEs can be classified in various ways:

Ordinary DEs (ODEs) Partial DEs (PDEs)Only one independent variable Many independent variablesEqs (1) and (2) Eqs (3) and (4)

Initial value problems(IVP) Boundary value problems (BVP)Function value(s) known at certain values of the variable(s). Function value(s) known on the domain borders.Want function value(s) at arbitrary variable value(s). Want function values in other point(s) in the domain.Eqs (1) and (4) Eqs (2), (3) and (4)In 1D: , compute . In 1D: , , compute .

(two-point BVP)

tdd h t gt–=

x2

2

dd w x S

EI------w x qx

2EI--------- x L–+=

h 2 2

2m--------------------- 2 r– V r r+ E r=

cP tT r t k T r t=

f t0 y0= f t f x1 y1= f x2 y2= f x

Scientific computing III 2011: 13. Differential equations 2

Differential equations

• The difference between initial value problems and boundary value problems can be illustrated as below (according to NR, fig. 19.0.1).

- In general boundary value problems are harder to solve than initial value problems:BVPs need iteration IVP you ‘only’ need to integrate the equations in ‘time’ to reach the desired point.

- In this chapter we deal with ordinary differential equations both IVPs and BVPs.- Probably there will be no time partial differential equations.

initial values

boun

dary

con

ditio

ns

boun

dary

con

ditio

ns

boundaryvalues

Page 2: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 3

Differential equations: initial value problems

• The simplest ODE (IVP) is

and its solution (starting from ‘time’ )

. (Integration of the differential equation.)

- The solution is unique up to the constant i.e. we get a family of solutions.

- The initial condition is needed to fix the value of the constant and to get a unique solution:

.

• The general form of an IVP is

.

dydt------ f t=

t0

y t c f d

t0

t

+=

y t c

y t0 y0=

dydt------ f t y=

Scientific computing III 2011: 13. Differential equations 4

Differential equations: initial value problems

• Solution methods for a single ODE are easily generalized to a group if ODEs (Use of vector notation makes it easier.)

, ,

, .

• ODEs having also higher derivatives can be transformed to groups of ODEs so that methods presented here can be applied to these higher order ODEs:

y' dydt------y t

y1 t

y2 t

yn t

= f t y

f1 t y

f2 t y

fn t y

= y0

A1A2

An

=

y' f t y= y t0 y0=

y n f t y y' y n 1–=y t0 A1=

y' t0 A2=

y n 1– t0 An=

y1' y2=

y2' y3=

yn' f t y1 y2 yn=

y1 t0 A1=

y2 t0 A2=

yn t0 An=

Page 3: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 5

Differential equations: initial value problems

• Numerical solution calls for discretization of the problem.

- Instead of computing at any we obtain values in discrete points.

(We talk about time here, but can - of course - be any variable to be solved.)

- The interval between points (in IVPs the time step) may vary as a function of time (variable time step).

• There several error sources in the integration of ODEs:

- Discretization error: a too large time step decrease the time step.

- Instability: small errors introduced in the beginning grow exponentially change the algorithm.

- Stiff ODEs: phenomena with different time scales step size selected according to the fastest processes

y t t tit

Scientific computing III 2011: 13. Differential equations 6

Differential equations: initial value problems

• Something general can be said about the existence of the solution of the IVP

- A function is said to satisfy a Lipschitz condition in the variable in the set if so that whenever .

- is called the Lipschitz constant of .

- A set is said to be convex if whenever the point also

belongs to for each , .

- Suppose is defined in a convex set . If , such that , for all then

satisfies the Lipschitz condition on in variable with Lipschitz constant .

- Suppose that and that is continuous on . If satisfies the Lipschitz condition on in variable then the IVP

, , has a unique solution for .

y' f t y=

f t y y D R2

L 0 f t y1 f t y2– L y1 y2– t y1 t y2 D

L f

D R2 t1 y1 t2 y2 D 1 – t1 t2+ 1 – y1 y2+

D 0 1

f t y D R2 L 0yf t y L t y D

f D y L

D t y a t b – y= f t y Df D y

y' f t y= a t b y a =y t a t b

Page 4: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 7

Differential equations: initial value problems

- Example: , . .- Keeping constant we apply the mean value theorem to function :

, , .

.

- Thus and the IVP has a unique solution.

- What about the stability of the IVP?

- The IVP , , is said to be a well-posed problem (or a stable problem) if 1. A unique solution exists.2. For any , with the property that, whenever and ,

a unique solution to the problem , , (PB) (=perturbed problem)

exists with , .

- One can show that the IVP is a well-posed problem if 1) is a continuous function of and for all and2) it satisfies the Lipschitz condition , .

y' 1 t tysin+= 0 t 2 y 0 0=t f y 1 t tysin+=

dfdy

-------------f y2 f y1–

y2 y1–------------------------------= y1 y2 y1 y2

t2 tcosf y2 f y1–

y2 y1–------------------------------= 4

f y2 f y1–y2 y1–

--------------------------------- f y2 f y1– 4 y2 y1–

L 4=

y' f t y= a t b y a =

0 k 0 0 t

z tz' f t z t+= a t b z a 0+=

z t y t– k t a t b

y' f t y=f t y t y t y D

f t y1 f t y2– L y1 y2– t y1 t y2 D

Scientific computing III 2011: 13. Differential equations 8

Differential equations: initial value problems

- Example:

.

- Since , the function satisfies the Lipschitz condition.

- Morever, is continuous the problem is well posed.

- Let’s see what we get when we perturb the problem:

, , .

- Solutions to the original and perturbed problems are , .

- One can now easily see that if and then

.

D t y 0 t 1 – y=y' y– t 1+ +=y 0 1=

y– t 1+ +y

-------------------------------- 1= f t y y– t 1+ +=

f

dzdt----- z– t 1+ + += 0 t 1 z 0 1 0+=

y t e t– t+= z t 1 0 –+ e t– t+ +=

0

y t z t– 0– e t– – 0 1 e t––+ 2= t

Page 5: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 9

Differential equations: initial value problems

- The above discussion concerns the stability of the problem itself. - A problem is stable (well-posed) if the solution depends continuously on the initial value and on the right-hand side

function .

- The IVP could, however, be stable but ill-conditioned. - Condition number gives the sensitivity of the solution to the initial data. - A large condition number means an ill-conditioned problem.

- Assume we have a perturbed IVP: , , .

- Let’s write , where is the solution of the unperturbed IVP. This gives us

and .

- The above approximation is valid for small (we have a well-posed problem).- From the equation we can readily solve :

.

- This means that if

,

then probably remains bounded by as increases. In this case the problem is well-conditioned.

- An example of an ill-conditioned problem is: , , .

- Now we can calculate indicating that the change in becomes increasing large as increases.

f

y' t; f t y t;= y t0; y0 += t a b

z t; y t; y t–= y t

z' t; f t y t; f t y t– f t y ty

------------------------z t;= z t0; =

z

z t; f t' y t'y

-------------------------- t'd

t0

t

exp

f t y ty

------------------------ 0 t a b

z t; t

y' y g t+= y 0 y0= 0

z t; e t= y t t

Scientific computing III 2011: 13. Differential equations 10

Differential equations: initial value problems

- As another example of an ill-conditioned problem con-sider

, .

- The solution is .

- Perturbed problem:, ,

has the solution .

IVP:

Solution:

y' ay be t–+=

y 0 y0=

f t y ay be t–+=

yf a=

y t 1a 1+------------ y0 ay0 b+ + eat be t––=

y' 100y 101e t––= y 0 1=

y t e t–=

y' 100y 101e t––= y 0 1 +=

y t; e t– e100t+=

z t; f t' y t'y

-------------------------- t'd

0

t

exp 100y 101e t––y

------------------------------------------ t'd

0

t

exp e100t= =

Page 6: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 11

Differential equations: initial value problems

• Stability of the solution of an IVP :

- Solution is stable if , such that every solution of the problem that fulfills

also fulfills

for all .

- Solution is asymptotically stable if it is stable and .

- A simple example: . Solution is .- If all solutions grow and diverge from each other exponentially: unstable solution.- If all solution converge to zero: asymptotically stable solution.- If is complex ( ) then stability depends on : .- If then we get an oscillatory solution that is stable but no asymptotically stable.

• In addition to the stability definitions given above, the numerical method used to solve the IVP can also be stable or unstable.

y' t f t y=

0 0 y

y 0 y 0–

y t y t–t 0

y t y t–tlim 0=

y' y= y t y 0 e t=00

a ib+= Re y t y 0 e a ib+ t=Re 0=

Scientific computing III 2011: 13. Differential equations 12

Differential equations: initial value problems

• Euler’s method is the simplest possible method to solve an IVP.• It is not used much in practical problems but its derivation illustrates the concepts needed in deriving more advanced meth-

ods.

• The IVP we are trying to solve is the same as before

, , .

• The problem is discretized by computing the values in mesh points in interval :

, .

- The distance between the consecutive points is called the step size.

- Assume the solution has two continuous derivatives.- We will expand it as a Taylor series

,

or .

- Since is the solution to the IVP we can write the expansion as

.

dydt------ f t y= a t b y a =

y t0 t1 tN a b

ti a ih+= i 0 1 N=

h b a– N=

y t

y ti 1+ y ti ti 1+ ti– y' ti12--- ti 1+ ti– 2y'' i+ += i ti ti 1+

y ti 1+ y ti hy' ti12---h2y'' i+ +=

y t

y ti 1+ y ti hf ti y ti12---h2y'' i+ +=

Page 7: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 13

Differential equations: initial value problems

- In Euler’s method we simply drop the 2nd order term and get the iteration formula

,

, .

- So we get an approximation to the solution

.

- The algorithm is extremely simple

1. Set , , . .2. For do

Set .Set .

y0 =

yi 1+ yi hf ti yi+= i 0 1 N=

yi y ti

h b a– N= t a= y =i 1 2 N=

y y hf t y+t a ih+

Scientific computing III 2011: 13. Differential equations 14

Differential equations: initial value problems

- As a simple example take

. . .

- The exact solution is .

- The Euler algorithm in awk:

BEGIN { h=0.5; y=0; tm=3.0; n=tm/h; print 0,y; for (i=0;i<n;i++) { t=i*h; y=y+h*sin(t); t=(i+1)*h print t,y; }}

- In the figure one can see the graphical interpretation of the alrgorithm.- When the problem is well-posed

.

- Euler’s method is easily generalized to groups of equations:.

dy tdt

------------ tsin= y 0 0= 0 t 3

y t 1 tcos–=

1 tcos–

Euler solution

f ti yi y' ti f ti y ti=

yk 1+ yk hf tk yk+=

Page 8: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 15

Differential equations: initial value problems

- What about the error behavior of the Euler’s method?

- By subtracting the exact solution from the Euler iteration result we get

, . (1)

- The left hand side of this equation is the total error of the approximation and it consists of

1. Errors cumulated from the previous step at 2. Error formed in computing the current step (local error).

- Error 2 is called local because if we happened to get the exact value in the previous step ( ) then the only error that is left is .

- In general the local error is defined as

and is the solution of the local problem:

.

- Due to the local error Euler’s method is of the order one.

- In general the order of a method is if the local error behaves as .

cumulated local

y tk 1+ yk 1+– y tk yk– h f tk y tk f tk yk–h2y'' k

2---------------------+ += k tk tk 1+

yk 1+tk

k

h2y'' k 2 lk 1+

ln y tn yn–=

y

y'ˆ t f t y t=

y tn 1– yn 1–=

h2y'' k 2 O h2=

p O hp 1+

Scientific computing III 2011: 13. Differential equations 16

Differential equations: initial value problems

- Using the mean value theorem we can estimate the term in the brackets in (1):

, .

- Substituting this to eq. (1) we get

where

- So we get the total error at step as

- As far as the total error in Euler’s method does not grow.

- If the error grows during the iteration without limits.

h f tk y tk f tk yk– hf tk k

y---------------------- y tk yk–= k y tk yk

y tk 1+ yk 1+– y tk yk– hf tk k

y---------------------- y tk yk–

h2y'' k2

---------------------+ +

y tk yk– 1 hfy tk k+h2y'' k

2---------------------+

=

=

fy tk kf tk k

y----------------------=

k 1+

ek 1+ ek 1 hfy tk k+ lk 1++=

error atpreviousstep

amplificationfactor

localerror

1 hfy tk k+ 1

1 hfy tk k+ 1

Page 9: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 17

Differential equations: initial value problems

- Let’s look at the test problem ,

where is a complex constant.- The solution is .

- Applying Euler’s method we get

.

- Let now so that the solutions are asymptotically stable and approach zero. - This means that the numerical approximations must not grow.

In Euler’s method this means.

- The absolute stability region of a numerical method consists of those complex numbers that fulfill the condition

, .

- Absolute stability interval is the part of the real axis that belongs to .

- The absolute stability region of Euler’s method is the circle.

and the interval is .

y' y=

y t y 0 e t=

yk yk 1– h yk 1–+ 1 h+ yk 1– y 0 1 h+ k= = = =

Re 0yk

1 h+ 1

Generally for the Euler’s method:

.1 hfy+ 1

Dz h=

yk yk 1– k 1 2=

D

D z C z 1+ 1=2– 0

Scientific computing III 2011: 13. Differential equations 18

Differential equations: initial value problems

- Example: solving , .

using Euler’s method.- Exact solution is and it is clearly stable.

- Euler’s method gives a difference equation

with solution

.

- Let the initial value be .

- Exact solution is now .

- Euler’s method gives

- We see immediately that if the solution grows without limit when increases.

y' 100y– 100+= y 0 y0=

y t y0 1– e 100t– 1+=

yk 1+ yk h 100yk– 100++ 1 100h– yk 100h+= =

yk y0 1– 1 100h– k 1+=

y0 2=

y t e 100t– 1+=

yk 1 100h– k 1+=

h 0.02 yk k

Page 10: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 19

Differential equations: initial value problems

- The error formula of Euler’s method contains both in the error amplification term and in the local error term:

.

- In practical computations should be as large as possible in order to speed up the calculations.so small that the error amplification factor < 1so small that the local error term is acceptable.

- The derivatives are difficult to compute but the local term can be estimated based on the iterations:

.

- By requiring that the local error fulfills the condition

we get the upper limit to the step size

.

h

ek 1+ ek 1 hfy tk k+h2y'' k

2---------------------+=

h

ya''yk' yk 1– '–tk tk 1––

--------------------------=

ya'' h2

2----------------

h 2ya''

----------

Scientific computing III 2011: 13. Differential equations 20

Differential equations: initial value problems

- Even if it would be possible - from the point of view of the consumed CPU time - to decrease the step size indefinitely we wouldn’t get better and bet-ter results: finally the round-off errors show up.

- The figure on the right shows the relative error of the solution of the IVP . .

using step sizes

and using single precision floating point numers.

- You probably remember the same phenomenon in computing the deriva-tive of a function using simple difference quotient.

10 4–

10 5–

10 6–y' tsin= y 0 0= 0 t 3

h 10 4– 10 5– 10 6–=

Page 11: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 21

Differential equations: initial value problems

• We can go further in the Taylor’s series

and develop higher order methods.

- These methods have the advantage of higher-order local truncation error.

- However, one has to calculate the higher derivatives which is time consuming in many practical problems.

• An alternative to the Euler’s method is an implicit version of it (backward Euler’s method):- Integrate over one time step

.

- By approximating the intergral by we get the Euler’s method.

- However, we can also use when we get the backward Euler:

.

- The word implicit means that new point is on both sides of the iteration formula. It has to be solved by numerical methods.

- One can show that this methods has a error amplification factor which means that when applied to a stable

linear problem with constant coefficients when , we get for all .

y ti 1+ y ti ti 1+ ti– y' ti12--- ti 1+ ti– 2y'' ti+ + +=

f n t y

y' t f t y=

y tk 1+ y tk f t y t td

tk

tk 1+

+=

tk 1+ tk– f tk yktk 1+ tk– f tk 1+ yk 1+

yk 1+ yk hf tk 1+ yk 1++=

yk 1+

1 hfy– 1–

Re 0 1 h– 1– 1 h 0

Scientific computing III 2011: 13. Differential equations 22

Differential equations: initial value problems

• The methods described above were presented to illustrate the important concepts in numerical integration of IVPs.• They are seldom used in practical applications.• Probably the most often used method to solve IVPs is the Runge-Kutta (RK) method (or one of the many RK methods).• It offers the higher orders without the need to compute derivatives of . • The only drawback is that the number of function evaluations of per time step may be large.• We need a couple of results concerning the Taylor’s theorem of a function of two variables.

- Suppose has all its partial derivatives up to continuous on .- Let .

- For every with

.

- In other words is the Taylor polynomial and is the remainder term.

ff

f y t n 1+ D t y a t b c y d=t0 y0 D

t y D t t0 y y0f t y Pn t y Rn t y+=

Pn t y f t0 y0 t t0–f t0 y0

t---------------------- y y0–

f t0 y0y

----------------------+

t t0– 2

2-------------------

t2

2f t0 y0

y y0– 2

2---------------------

y2

2f t0 y0 t t0– y y0–

t y

2f t0 y0+ +

1n!----- n

jt t0– n j– y y0– j

tn j– yj

nf t0 y0

j 0=

n

+

+

+ +

=

Rn t y 1n 1+ !

------------------- n 1+j

t t0– n 1+ y y0– j

tn 1 j–+ yj

n 1+f

j 1=

n 1+=

Pn Rn

Page 12: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 23

Differential equations: initial value problems

- Let’s see how by using the RK method we could increase the order of the Euler’s method by one.- This means that we have to approximate the term

with error no greater than .- We take the approximation to have the form .

- Since and

.

- Now we expand into a Taylor polynomial of degree one about

.

- The coefficients , , and can be determined by setting

(remember )

and matching the coefficients of and its derivatives.

Higher order Taylor method:

y0 y t0=

yi 1+ yi hT n ti yi+=

T n f ti yih2---f' ti yi

hn 1–

n 1+ !-------------------f n 1– ti yi

+

+ +

=

T 2 t y f t y h2---f' t y+=

O h2

a1f t 1 y 1++

f' t y df t ydt

-----------------tf t y

yf t y y' t+= = y' t f t y=

T 2 t y f t y h2---

tf t y h

2---

yf t y y' t+ +=

f t 1 y 1++ t y

a1f t 1 y 1++ a1f t y a1 1 tf t y a1 1 y

f t y a1R1 t 1 y 1+++ + +=

R1 t 1 y 1++ 12

2------

t2

2f 1

2

2------

y2

2f 1 1 t y

2f+ +=

a1 1 1

f t y h2--- f' t y+ a1f t 1 y 1++=

f t y h2---

tf t y h

2---

yf t y y' t+ + a1f t y a1 1 t

f t y a1 1 yf t y+ += y' f=

f

Scientific computing III 2011: 13. Differential equations 24

Differential equations: initial value problems

- The results is

, , .

and

.

- If the 2nd partial derivatives of are bounded then the remainder term is of order .

- The method derived above is a 2nd order (and two-stage) RK method and the algorithm itself is simple:

1. .

2. .

- This has a name of its own: the midpoint method.

a1 1= 1h2---= 1

h2---f t y=

T 2 t y f t h2---+ y h

2---f t y+ R1 t h

2---+ y h

2--- f t y+–=

f R1 O h2

y0 y 0=

yi 1+ yi hf tih2---+ yi

h2---f ti yi++=

Page 13: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 25

Differential equations: initial value problems

- However, the most used RK method is the 4th order version:

.

- Note that in this algorithm the function is evaluated 4 times per time step.

- In general a -stage RK method can be written as

.

y0 y 0=

k1 hf ti yi=

k2 hf tih2---+ yi

k12-----+=

k3 hf tih2---+ yi

k22-----+=

k4 hf ti 1+ yi k3+=

yi 1+ yi16--- k1 2k2 2k3 k4+ + ++=

f

s

yi 1+ yi h bjkjj 1=

s+=

kj f ti cjh+ yi h ajlkll 1=

2+=

cj ajll 1=

s=

Scientific computing III 2011: 13. Differential equations 26

Differential equations: initial value problems

- One can describe the RK method as follows:

1. Take an Euler step of length from . The derivative at the new point is .

2. Return to the original point and take a new Euler step ( ) using the weighted average

of and as the derivative. We get a new derivative .

- In this way a number of approximation are computed for the derivative and they all are used in the end to compute the step taken from .

- The number of stages in a RK method is not directly the order of the method:

Order 1 2 3 4 5 6 7 8 9 10Stages 1 2 3 4 6 7 9 11

- The table above explains the popularity of the four-stage RK method.

c2h ti yi k2c3h

k1 k2 k3

ti yi

12 s 17 13 s 17

Page 14: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 27

Differential equations: initial value problems

• Most practical routines for integration of an IVP require the user to give an upper limit to the error in the end result.- The program then adjusts the time step accordingly.

- This means that the algorithm somehow has to be able to estimate the error.

- One way to do this is to compute the solution at the same time with step sizes and .- Assuming that the local error introduced by the larger step is twice that of the smaller step one can get an estimate for

the error by

,

where and are the solution at time .

- However, computing with two time steps is an extra effort.

- In the RK method the error estimate can be computed by solving the IVP with two RK solvers simultaneously.- The two solvers can be chosen in such a way that the derivatives needed by the lower order solver are already computed

by the higher order method. - This pair of solvers is called the embedded Runge-Kutta pair.- They are sometimes also called Runge-Kutta-Fehlberg methods.

- If the RK method is written in the form then the method is stable if fulfills the Lipschitz condi-tion

, for all , .

h

h 2h

ln12---

yn yn–

2p 1–-------------------

yn yn n

yn 1+ yn F tn yn h f;+= F

F t y h f; F t z h f;– L y z–a t b – y z

Scientific computing III 2011: 13. Differential equations 28

Differential equations: initial value problems

• Below is a comparison of the maximum error of various one-step methods1.- A one-step method is a method that only uses to compute the next step .

- The IVP was , ,

- The exact solution is .

- The trapezoidal method is .

1. From J. Haataja et al., Numeeriset menetelmät käytännössä, CSC, 1999

yi yi 1+

y' 5ty2– 5t--- 1

t2----–+= y0 1= t 1 10

y t 1 t=

yk 1+ ykh2--- f tk 1+ yk 1+ f tk yk++=

Page 15: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 29

Differential equations: initial value problems

• In multistep (MS) methods one uses also information from earlier time steps than only the previous one.

- I.e. we want to use also, ,

, ,

to approximate the next point .

- In general a step linear multistep method can be written as

where the coefficients fulfill the conditions , , .

- Methods can be derived by approximating the integral in

with a polynomial

and by integrating it.

yk 1– yk 2–f tk 1– yk 1– f tk 2– yk 2–

yk 1+

r fj f tj yj

jyk j+j r– 1+=

1h jfk j+

j r– 1+=

1=

1 1= r– 1+ 0 r– 1+ 0

y tk 1+ y tk f t y t td

tk

tk 1+

+=

P t a0 a1t a2t2 asts+ + + +=

Scientific computing III 2011: 13. Differential equations 30

Differential equations: initial value problems

- The unknown coefficients are determined by fixing the polynomial to certain values and , .

- The good point in the MS methods is that the function is only evaluted once per time step.

- On the other hand, in the beginning of the iteration there are no previous data points available. - These can be generated by perfoming a couple of steps using a one-step method.

- Also, changing the time step is not so straightforward in MS methods.

- MS methods are divide to

explicit (open) methods, and

implicit (closed) methods, .

ai yi fi i k 1+ k k 1–=

f

1 0=

1 0

Page 16: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 31

Differential equations: initial value problems

• Of the explicit MS methods the Adams-Bashford (AB) is maybe the most well known.

- It is of the form

.

- The unknown new iteration point does not show up in the RHS of the equation so this is an open method.

- Below are the five first AB methods

.

- One can show that an AB method with steps is of the order .

- The stability interval of the AB methods are rather narrow, so that they are not suitable for stiff problems.

yk 1+ yk h jfk j+j r– 1+=

0+=

yk 1+ yk hfk+=

yk 1+h2--- 3fk fk 1––=

yk 1+ ykh12------ 23fk 16fk 1–– 5fk 2–++=

yk 1+ ykh24------ 55fk 59fk 1–– 37fk 2– 9fk 3––++=

yk 1+ ykh

720--------- 1901fk 2774fk 1–– 2616fk 2– 1274fk 3–– 251fk 4–+ ++=

r O hr

Scientific computing III 2011: 13. Differential equations 32

Differential equations: initial value problems

• The implicit Adams-Moulton (AM) method is more suitable for stiff problems:

.

- The five first AM methods are

.

- When the order of the AM method is .

- The absolute stability intervals of the methods are123 4

yk 1+ yk h jfk j+j r– 2+=

1+=

yk 1+ yk hfk 1++=

yk 1+ ykh2--- fk 1+ fk++=

yk 1+ ykh12------ 5fk 1+ 8fk fk 1––++=

yk 1+ ykh24------ 9fk 1+ 19fk 5fk 1–– fk 2–+ ++=

yk 1+ ykh

720--------- 251fk 1+ 646fk 264fk 1–– 106fk 2– 19fk 3–+ + ++=

r 2 r 1+

– 06– 08– 01.8– 0

Page 17: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 33

Differential equations: initial value problems

- The AM methods give better results than the AB methods.

- However, the drawback of the AM method is that due to its implicity one has to solve the new iteration point from a group of equations.

- A combination method often used is so called predictor-corrector method.- In it the AB method produces a initial prediction to the new value while the AM method is used to correct it more pre-

cise. - The correction may be done many times but not until the final convergence.- For example the pair of 3rd degree:

.

y

yk 1+p yk

h12------ 23fk 16fk 1–– 5fk 2–++=

yk 1+c yk

h12------ 5f tk 1+ yk 1+

p 8fk fk 1––++=

Scientific computing III 2011: 13. Differential equations 34

Differential equations: initial value problems

• So called backward differentiation formulae (BDF) methods are well suited for stiff ODEs. - They are derived by requiring a polynomial going through the previous iteration points:

, where is the Lagrange interpolation polynomial.

- Because is (an approximate) solution

- Combining the above equations and solving we get

,

where and are obtained by expanding the Lagrange interpolation polynomial in the first equation above.

- The method is implicit and the order is . - Below are the five first BDF methods:

(This is backward Euler.)

.

Pr t y tn j– ljn tj 1–=

r 1–= ljn t

PrPr' tn 1+ y' tn 1+ f tn 1+ y tn 1+= =

y tn 1+

yn 1+ jyn j–j 0=

r 1–h f tn 1+ yn 1++=

jr

yk 1+ yk hfk 1++=

yk 1+43---yk

13---yk 1–– 2

3---hfk 1++=

yk 1+1811------yk

911------yk 1–– 2

11------yk 2–

611------hfk 1++ +=

yk 1+4825------yk

3625------yk 1–– 16

25------yk 2–

325------yk 3–– 12

25------hfk 1++ +=

yk 1+300137---------yk

300137---------yk 1–– 200

137---------yk 2–

75137---------yk 3–– 12

137---------yk 4–

60137---------hfk 1++ + +=

Page 18: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 35

Differential equations: initial value problems

• Stiff ODEs are those where there are two or more disparate time scales.

- An example: , .

- Solution:

- By varying the initial value slightly causes large deviations in the initial behavior of the solution.

- Euler method is results very sensitive to the initial value: even a small error during the iteration causes the iteration to pick a solution with a different ; par-

ticularly one with . - Morever, with large enough time steps we get outside

the stability interval.- Demo.

- Stiffness is difficult to define exactly:1) Process described by the ODE contains disparate timescales.2) A group of ODEs is stiff if the eigenvalues of the Jacobian ( ) has eigenvalues that dif-fer greatly in magnitude.3) When using an explicit method a much smaller time step must be used as which would be dictated by the accuracy criteria.

y' 100y– 100t 101+ += y 0 y0=

Exact solutions for y0 = 0 ... 2

y t y0 1– e 100t– t 1+ +=

y0

y0y0 1

J Jij fi yj=

Scientific computing III 2011: 13. Differential equations 36

Differential equations: initial value problems

- Example (from NR, section 16.6)

.

- Exact solution

.

- Using Euler’s method we have to use really small steps:

- Although the smoothness of the function would allow a longer time step outside the initial transient behavior (term )

- What about having a small step in the beginning and increasing it at - say -

t

u

0 0.05

u' 998u 1998v+=v' 999u– 1999v–=

u 2e t– e 1000t––=

v e t–– e 1000t–+=

exacth 10 6–=

h 10 4–=

t0 1

u

e 1000t–

t 10 2–

Page 19: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 37

Differential equations: initial value problems

- Below is the results of a version of Euler which changes the stepsize after a certain time:

- This does not help because the instability of the transient is still there though the solution itself is a smoothly varying function.

- Or to put it in another way: In order the solution to be stable we must use a smaller time step than the one based only on the accuracy criteria.

t

u

0

h 10 6–= h 10 3–=

0.06

Scientific computing III 2011: 13. Differential equations 38

Differential equations: initial value problems

- For comparison consider the following equation that is not stiff but has a rather similar solution:

- And the solution with initial conditions , is

.

- Using Euler with gives reasonable results:

u' u– v+=v' u v–=

u 0 1= v 0 0=

u 12--- 1 e 2t–+=

v 12--- 1 e 2t––=

h 10 3– 10 2–=

t0 1

u

h 10 2–=

h 10 3–=exact

Page 20: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 39

Differential equations: initial value problems

- For stiff ODEs the implicit methods work better. - Take a simple example:

, .- Euler’s iteration gives us

.

- The method is unstable if .

- Now using implicit method (backward Euler) we get or

.

- This method is stable for all values of .

- So with implicit methods we can use larger time steps in considering the stability of the method.- However, the accuracy may require to use smaller stepsize.

y' cy–= c 0

yn 1+ yn hy'n+ 1 hc– yn= =

h 2c---

yn 1+ yn hy'n 1++=

yn 1+yn

1 ch+---------------=

h

Scientific computing III 2011: 13. Differential equations 40

Differential equations: initial value problems

• Practical tools:

GSL: routines for many stepping algorithms and for timestep control(including various Runge-Kutta and implicit methods)

Matlab: also many routines: say help funfun to get a list.Maxima: symbolic solver, example solving the last example equation:

Page 21: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 41

Differential equations: initial value problems

• Practical tools (continued)

- SLATEC library includes (at least) the following algorithms:DDEABM: Adams-Bashforth methodDDEBDF: backward differentiation; for stiff problemsDDERKF: Runge-Kutta-Fehlberg method

- User should, in addition, provide the function that calculates the RHS of the equation(s) and for the backward differen-tiation method Jacobian of the problem needed:

• What method to use? For non-stiff ODEs explicit RK, or predictor-corrector and AM if computing the derivative is expensive.For stiff ODEs BDF and implicit RK.

- Now a nice demo.

J

y1' f1 t y1 yN=

y2' f2 t y1 yN=

yN' fN t y1 yN=

J

y1

f1y2

f1yN

f1

y1

f2y2

f2yN

f2

y1

fNy2

fNyN

fN

=

Scientific computing III 2011: 13. Differential equations 42

Differential equations: boundary value problems1

• Boundary value problems (BVPs) are harder to solve because the conditions have to be fulfilled in more than one point.- The general form of 1D BVP is

( and have components)

- Linear boundary conditions (BC) can be written in the form .- If the BCs are separable they can be written as

.

- Sometimes the DE and BC contain a parameter :

and the objective is to determine those values of for which the system has non-trivial solutions (eigenvalues and eigen-functions).- E.g. Schrödinger equation

- In 1D a particle constrained by infinite potential walls:

, which can be written as a system of 1st order DEs

1. Sources: J. Haataja et al., Numeeriset menetelmät käytännössä, CSC, 1999; R. L. Burden, J. D. Faires, Numerical Analysis, PWS-KENT, 1989.

y' f x y=g y a y b 0=

y g N

Bay a Bby b+ b=

y a ba=

y b bb=

y' f x y=g y a y b 0=

12---– 2 V r+ r E r=

12---– '' x V x x+ E x=

a 0=b 0=

x 2 V x E– x=x ' x=

Page 22: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 43

Differential equations: boundary value problems

- In the shooting method (SM) we utilize methods developed for IVPs.

- We integrate the DE starting from a chosen initial condition:, .

- We have to choose so that the solution fulfills the BC.

- The solution of the original BVP is thus where is the solution of the group of equations

.

- This generally a non-linear group of equations.

y' f x y= y a s=

s y x s;g y a s; y b s; 0=

y x s*; s*

F s g s y b s; 0= =

Scientific computing III 2011: 13. Differential equations 44

Differential equations: boundary value problems

- A common form of BVPs is the 2nd order equation

, ,, .

- This is converted to a pair of 1st order DEs as ( , )

.

- Existence of the solution:

- Assume that the function and its derivatives , are continuous in.

- Now, if for all for all the following apply

a) and

b) such that

then the abovementioned BVP has a unique solution.

y'' f x y y'= a x by a =y b =

y y1 y' y2

y1' y2=

y2' f x y1 y2=

f f y f y'D x y y' a x b – y – y'=

x y y' D

yf x y y' 0

My'

f x y y' M

Page 23: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 45

Differential equations: boundary value problems

- An example:, , ,

or , .

Now , .

So, the equation has a unique solution.

y'' e xy– y'sin+ + 0= x 1 2 y 1 0= y 2 0=

y'' f x y y'= f x y y' e xy–– y'sin–=

yf x y y' xe xy– 0=

y'f x y y' y'cos– 1=

Solution of the IVP

with values

y'' e xy– y'sin+ + 0=y 1 0=y' 1 c=

c0.60.521690.4

=

c 0.6=

c 0.52169=

c 0.4=

Scientific computing III 2011: 13. Differential equations 46

Differential equations: boundary value problems

- The linear form of the 2nd order BVP can be written as, , , . (1)

- For this form the existence of a unique solution requires thata) , , are continuous in andb) in .

- Consider two IVPs based on the BVP above:, , , ,

solution

, , , ,solution

- It is easy to show that the unique solution to the original equation (1) can be expressed in terms of and :

,

assuming 1.

- A demo using GSL.

1. No need to worry about this. One can show that if is the solution of with then .

y'' p x y' q x y r x+ += a x b y a = y b =

p x q x r x a bq x 0 a b

x

y

a b

y1y2

y

y'' p x y' q x y r x+ += x a b y a = y' a 0=y1 x

y'' p x y' q x y+= x a b y a 0= y' a 1=y2 x

y1 x y2 x

y x y1 xy1 b–

y2 b----------------------y2 x+=

y2 b 0

y2 y'' p x y' q x y+= y2 a y2 b 0= = y2 x 0= x a b

Page 24: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 47

Differential equations: boundary value problems

- In the multiple shooting method (MSM) the interval is divided into smaller parts and the SM is applied to every part so that the parts join to each other continuously.- Using the method you end up solving a (possibly nonlinear) group of equations.

- Divide the interval to parts.

- Let be the solution to the IVP , on the interval .

- In the MSM method we pursue the vectors , that satisfy the continuity and boundary conditions

,

.

- The unknown initial values are obtained from the nonlinear system of equations

a ba x1 x2 xm b= =

y' x xk sk; y' f x y= y xk sk= x xk xk 1+

sk k 1 2 m 1–=

y xk 1+ xk sk; sk 1+= k 1 2 m 2–=

g s1 y b xm 1– sm 1–; 0=

s s1 s2 sm 1–T=

F s

y x2 x1 s1; s2–

y x3 x2 s2; s3–

g s1 y xm xm 1– sm 1–;

0= =

Scientific computing III 2011: 13. Differential equations 48

Differential equations: boundary value problems

• The finite difference method is not based on solving the corresponding IVP but the BVP is discretized to a finite mesh and derivatives are approximated by finite differences.- An example shows how the method works1

- The BVP is

.

- Divide interval to parts: , ,

- Approximate the derivatives in the BVP by finite differences:

.

- Now we get the difference equation

,

, .

1. J.Haataja et al., Numeeriset menetelmät käytännössä, example 7.5.2

y'' f x y y'=y a =y b =

a b m h b a–m

------------= xi a ih+= i 0 m=

y' xnyn 1+ yn 1––

2h--------------------------------

y'' xnyn 1+ 2yn– yn 1–+

h2-----------------------------------------------

yn 1+ 2yn– yn 1–+ h2f xn ynyn 1+ yn 1––

2h--------------------------------= n 1 2 m 1–=

y0 = ym =

Page 25: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 49

Differential equations: boundary value problems

- In matrix form

, , .

- Note that this is a nonlinear group of equations.

Ay h2f y=

A

2– 1 0 0 01 2– 1 0 00 1 2– 0 0

0 0 0 2– 10 0 0 1 2–

= y

y1y2y3

ym 1–

= f y

f x1 y1y2 y0–

2h---------------- –

f x2 y2y3 y1–

2h----------------

f x3 y3y4 y2–

2h----------------

f xm 1– ym 1–ym ym 2––

2h-------------------------- –

=

Scientific computing III 2011: 13. Differential equations 50

ifferential equations: boundary value problems

- There are also methods based on expansion of the solution in terms of basis functions; e.g. collocation method (CM), Galerkin’s method.

- Take for example the equation, , .

- We expand the solution in terms of basis functions :

.

- The basis set can be chosen in an appropriate way considering the boundary conditions.- In the CM method the above expansion is substituted in the equation and we require that it solves the equation in

points within the interval :

, .

- From this equation we can determine the coefficients .

y'' f x y y'= y a = y b =

1 2 N

y x ci i xi 1=

N=

Nxj a b

ci i'' xji 1=

Nf xj ci i xj

i 1=

Nci i' xj

i 1=

N= j 1 2 N=

c1 c2 cN

Page 26: Differential equations - Acclab h55.it.helsinki.fiDifferential equations 2 The difference between initial value problems and boundary value problems can be illustrated as below (according

Scientific computing III 2011: 13. Differential equations 51

Differential equations: boundary value problems

• Practical tools:

Matlab: see help funfun for more information.

NAG: many routines for BVPs.

SLATEC: many routines for BVPs.

NETLIB: see the package ode (http://www.netlib.org/ode/)