Initial-Value Problems for Ordinary Di erential Equations

Chapter 5

Initial-Value Problems for OrdinaryDi↵erential Equations

Di↵erential equations are used to model problems in science and engineering that involve thechange of some variable with respect to another. Most of these problems require the solutionof an initial-value problem, that is, the solution to a di↵erential equation that satisfies a giveninitial condition.

In common real-life situations, the di↵erential equation that models the problem is too compli-cated to solve exactly, and one of two approaches is taken to approximate the solution. The firstapproach is to modify the problem by simplifying the di↵erential equation to one that can besolved exactly and then use the solution of the simplified equation to approximate the solutionto the original problem. The other approach, which we examine in this chapter, uses methodsfor approximating the solution of the original problem. This is the approach that is most com-monly taken because the approximation methods give more accurate results and realistic errorinformation.

The methods that we consider in this chapter do not produce a continuous approximation tothe solution of the initial-value problem. Rather, approximations are found at certain specified,and often equally spaced, points.

5.1 The Elementary Theory of Initial-Value Problem

This chapter is concerned with approximating the solution yptq to a problem of the form

dy

dt“ fpt, yq for a § t § b,

subject to an initial condition ypaq “ ↵.

We need some definition and results from the theory of ordinary di↵erential equations beforeconsidering methods for approximating the solutions to initial-value problems.

Definition 5.1.1. A function fpt, yq is said to satisfy a Lipschitz condition in the variable yon a set D Ä R2 if a constant L ° 0 exists with

|fpt, y1q ´ fpt, y2q| § L |y1 ´ y2| ,

whenever pt, y1q and pt, y2q are in D. The constant L is called a Lipschitz constant for f .

77

78CHAPTER 5. INITIAL-VALUE PROBLEMS FORORDINARYDIFFERENTIAL EQUATIONS

Example 5.1.2. The function fpt, yq “ t |y| satisfies a Lipschitz condition on the intervalD “ tpt, yq : 1 § t § 2, ´3 § y § 4u with Lipschitz constant L “ 2.

Definition 5.1.3 (Convex Set). A set D Ä R2 is said to be convex if whenever pt1, y1q andpt2, y2q belong to D, then pp1´�qt1 `�t2, p1´�qy1 `�y2q also belongs to D for every � P r0, 1s.

In geometric terms, Definition 5.1.3 states that a set is convex provided that whenever two pointsbelong to the set, the entire straight-line segment between the points also belongs to the set. Thesets we consider in this chapter are generally of the form D “ tpt, yq : a § t § b, ´8 † y † 8ufor some constants a and b. It is easy to see that these sets are convex.

Theorem 5.1.4. Suppose fpt, yq is defined on a convex set D Ä R2. If a constant L ° 0 existswith ��

BfBy pt, yq

�� § L, for all pt, yq P D,

then f satisfies a Lipschitz condition on D in the variable y with Lipschitz constant L.

Theorem 5.1.5. Suppose that D “ tpt, yq : a § t § b, ´8 † y † 8u and that fpt, yq iscontinuous on D. If f satisfies a Lipschitz condition on D in the variable y, then the initial-value problem

y1ptq “ fpt, yq, a § t § b, ypaq “ ↵

has a unique solution yptq for a § t § b.

The initial-value problem

y1ptq “ fpt, yq, a § t § b, ypaq “ ↵, (5.1)

is said to be stable if there exist constants "0 ° 0 and k ° 0 such that for any " with "0 ° " ° 0whenever �ptq is continuous with |�ptq| † " for all t P ra, bs, and when |�0| † ", the initial-valuepropblem

z1ptq “ fpt, zq ` �ptq, a § t § b, zpaq “ ↵ ` �0, (5.2)

has a unique solution zptq that satisfies

|zptq ´ yptq| † k" for all t P ra, bs.

This problem (5.2) is called a perturbed problem associated with the original problem (5.1). Itassumes the possibility of an error being introduced in the statement of the di↵erential equation,as well as an error �0 being present in the initial condition.

5.2. EULER’S METHOD 79

Numerical methods will always be concerned with solving a perturbed problem since any round-o↵ error introduced in the representation perturbs the original problem. Unless the originalproblem is stable and has a unique solution, there is little reason to expect that the numericalsolution to a perturbed problem will accurately approximate the solution to the original problem.

We remark that If fpt, yq is continuous and satisfies a Lipschitz condition in the variable y onthe set D then (5.1) is stable. If the initial-value problem has a unique solution and stable, thenwe call it is well-posed.

Example 5.1.6. The initial-value problem

dy

dt“ y ´ t2 ` 1, 0 § t § 2, yp0q “ 0.5 (5.3)

is well-posed on the set D “ tpt, yq : 0 § t § 2, ´8 † y † 8u. Since��Bpy ´ t2 ` 1q

By

�� “ 1,

then fpt, yq “ y ´ t2 ` 1 satisfies a Lipschitz condition in y on D with Lipschitz constant L “ 1.Since f is continuous on D, it implies that the problem is well-posed.

As an illustration, consider the solution to the perturbed problem

dz

dt“ z ´ t2 ` 1 ` �, 0 § t § 2, yp0q “ 0.5 ` �0, (5.4)

where � and �0 are constants. The solutions to (5.3) and (5.4) are

yptq “ pt ` 1q2 ´ 0.5et and zptq “ pt ` 1q2 ` p� ` �0 ´ 0.5qet ´ �,

respectively. Suppose that " is a positive number. If |�| † " and |�0| † ", then

|yptq ´ zptq| “��p� ` �0qet ´ �

�� § |� ` �0| e2 ` |�| § p2e2 ` 1q",

for all t. This implies that the problem (5.3) is well-posed with kp"q “ 2e2 ` 1 for all " ° 0.

5.2 Euler’s Method

Euler’s method is the most elementary approximation technique for solving initial-value problem.Although it is seldom used in practice, the simplicity of its derivation can be use to illustratethe techniques involved in the construction of some of the more advanced techniques, withoutthe cumbersome algebra that accompanies these constructions.

The object of Euler’s method is to obtain approximations to the well-posed initial-value problem

dy

dt“ fpt, yq, a § t § b, ypaq “ ↵. (5.5)

A continuous approximation to the solution yptq will not be obtained; instead, approximationsto y will be generated at various values, called mesh points, in the interval ra, bs. Once theapproximation solution is obtained at the points, the approximate solution at order points inthe interval can be found by interpolation.

We first make the stipulation that the mesh points are equally distributed throughout the intervalra, bs. This condition is ensured by choosing a positive integer N and selecting the mesh points

ti “ a ` ih for each i “ 0, 1, ¨ ¨ ¨N.


The common distance between the points h “ pb ´ aq{N “ ti`1 ´ ti is called the step size. Wewill use Taylor’s Theorem to derive Euler’s method. Suppose that yptq, the unique solution to(5.5), has two continuous derivatives on ra, bs, so that for each i “ 0, 1, 2, ¨ ¨ ¨ , N ´ 1,

ypti`1q “ yptiq ` pti`1 ´ tiqy1ptiq ` pti`1 ´ tiq22

y2p⇠iq,

for some number ⇠i P pti, ti`1q. Because h “ ti`1 ´ ti, we have

ypti`1q “ yptiq ` hy1ptiq ` h2

2y2p⇠iq,

and, because yptq satisfies the di↵erential equation (5.5),

ypti`1q “ yptiq ` hfpti, yptiqq ` h2

2y2p⇠iq. (5.6)

Euler’s method construct wi « yptiq for each i “ 1, 2, ¨ ¨ ¨ , N , by deleting the remainder term.Thus, Euler’s method gives

w0 “ ↵,

wi`1 “ wi ` hfpti, wiq, for each i “ 0, 1, 2, ¨ ¨ ¨ , N ´ 1.(5.7)

The equation (5.7) is called the di↵erence equation associated with Euler’s method. As wewill see later in this chapter, the theory and solution of di↵erence equations parallel, in manyways, the theory and solution of di↵erential equations.

Example 5.2.1. We use Euler’s method to approximate the solution to

y1 “ y ´ t2 ` 1, 0 § t § 2, yp0q “ 0.5,

at t “ 2. We will simply illustrate the steps in the technique when we have h “ 0.5. For thisproblem fpt, yq “ y ´ t2 ` 1, so

w0 “ yp0q “ 0.5;

w1 “ w0 ` 0.5pw0 ´ p0.0q2 ` 1q “ 0.5 ` 0.5p1.5q “ 1.25;

w2 “ w1 ` 0.5pw1 ´ p0.5q2 ` 1q “ 1.25 ` 0.5p2.0q “ 2.25;

w3 “ w2 ` 0.5pw2 ´ p1.0q2 ` 1q “ 2.25 ` 0.5p2.25q “ 3.375;

andyp2q « w4 “ w3 ` 0.5pw3 ´ p1.5q2 ` 1q “ 3.375 ` 0.5p2.125q “ 4.4375.

To interpret Euler’s method geometrically, note that when wi is a close approximation to yptiq,the assumption that the problem is well-posed implies that

fpti, wiq « y1ptiq “ fpti, yptiqq.

The graph of the function highlighting yptiq and one step in Euler’s method appear in Figure5.1.

Example 5.2.2. We use Euler’s method to approximate the solution to

y1 “ y ´ t2 ` 1, 0 § t § 2, yp0q “ 0.5,


Figure 5.1: The true solution and Euler’s method approximation.

at t “ 2. We perform the steps in the technique when we have h “ 0.2 and compare these withthe exact values given by yptq “ pt ` 1q2 ´ 0.5et. With h “ 0.2 we have N “ 10, ti “ 0.2i fori “ 0, 1, ¨ ¨ ¨ , 9, w0 “ 0.5, and

wi`1 “ wi `hpwi ´ t2i ` 1q “ wi ` 0.2rwi ´ 0.04i2 ` 1s “ 1.2wi ´ 0.008i2 ` 0.2 for i “ 0, 1, ¨ ¨ ¨ 9.

Hence, we have

w1 “ 1.2p0.5q ´ 0.008p0q2 ` 0.2 “ 0.8; w2 “ 1.2p0.8q ´ 0.008p1q2 ` 0.2 “ 1.152;

and so on. The following table shows the comparison between the approximate values at ti andthe actual values.

ti wi yptiq |yptiq ´ wi|0.0 0.5000000 0.5000000 0.00000000.2 0.8000000 0.8292986 0.02929860.4 1.1520000 1.2140877 0.06208770.6 1.5504000 1.6489406 0.09854060.8 1.9884800 2.1272295 0.13874951.0 2.4581760 2.6408591 0.18268311.2 2.9498112 3.1799415 0.23013031.4 3.4517734 3.7324000 0.28062661.6 3.9501281 4.2834838 0.33335571.8 4.4281538 4.8151763 0.38702252.0 4.8657845 5.3054720 0.4396874

Note that the error grows slightly as the value of t increases. This controlled error growth is aconsequence of the stability of Euler’s method, which implies that the error is expected to growin now worse than a linear manner.

Moreover, we test the e�ciency of Euler’s method using di↵erent step size h. The results arelisted in the graphs below. See Figure 5.2. One can expect that the error is getting smaller if asmaller step size is used.

Although Euler’s method is not accurate enough to warrant its use in practice, it is su�cientlyelementary to analyze the error that is produced from its application. The error analysis for the


0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

Figure 5.2: Euler’s methods with di↵erent step size.


more accurate methods that we consider later follows the same pattern but is more complicated.To derive an error bound for Euler’s method, we need two computational lemmas.

Lemma 5.2.3. For all x • ´1 and any positive number m, we have 0 § p1 ` xqm § emx.

Proof. Since we have

ex “ 1 ` x ` 1

2x2 ` 1

6x3 ` ¨ ¨ ¨ • 1 ` x

for all x P R, then we have emx “ pexqm • p1 ` xqm • 0 for all x • ´1.

Lemma 5.2.4. If s and t are positive real numbers, taiuki“0 is a sequence satisfying a0 • ´t{s,and

ai`1 § p1 ` sqai ` t, for each i “ 0, 1, 2, ¨ ¨ ¨ , k ´ 1,

then

ai`1 § epi`1qsâ0 ` t

s

˙´ t

s.

Proof. Note that

ai`1 § p1 ` sqai ` t § p1 ` sqrp1 ` sqai´1 ` ts ` t “ p1 ` sq2ai´1 ` t p1 ` p1 ` sqq§ p1 ` sq3ai´2 ` t

`1 ` p1 ` sq ` p1 ` sq2

˘

§ ...

§ p1 ` sqi`1a0 ` t`1 ` p1 ` sq ` p1 ` sq2 ` ¨ ¨ ¨ ` p1 ` sqi

˘

“ p1 ` sqi`1a0 ` t1 ´ p1 ` sqi`1

1 ´ p1 ` sq “ p1 ` sqi`1a0 ` t

sp1 ` sqi`1 ´ t

s

“ p1 ` sqi`1

â0 ` t

s

˙´ t

s

§ epi`1qsâ0 ` t

s

˙´ t

s.

We have used Lemma 5.2.3 to show the last inequality.

The following theorem provides a theoretical estimate of Euler’s method. In general, we assumethat the function fpt, yq is good enough in the sense that it is Lipschitz continuous with respectto y variable with the Lipschitz constant L. We also assume that the domain D of the functionfpt, yq is a convex set in R2.

Theorem 5.2.5. Suppose f is continuous and satisfies a Lipschitz condition with constant Lon D “ tpt, yq : a § t § b, ´8 † y † 8u and that a constant M exists with

��y2ptq�� § M, for all t P ra, bs,

where yptq denotes the unique solution to the initial-value problem

y1 “ fpt, yq a § t § b, ypaq “ ↵.

Let twiuNi“0 be the approximations generated by Euler’s method for some positive integer N .Then, for each i “ 0, 1, 2, ¨ ¨ ¨ , N ,

|yptiq ´ wi| § hM

2L

”eLptiáq ´ 1

ı.


Proof. When i “ 0 the result is clearly true, since ypt0q “ w0 “ ↵. From (5.6), we have

ypti`1q “ yptiq ` hfpti, yptiqq ` h2

2y2p⇠iq,

for i “ 0, 1, ¨ ¨ ¨ , N ´ 1, and from the equations in (5.7),

wi`1 “ wi ` hfpti, wiq.

Using the notation yi “ yptiq and yi`1 “ ypti`1q, we subtract these two equations two obtain

yi`1 ´ wi`1 “ yi ´ wi ` hrfpti, yiq ´ fpti, wiqs ` h2

2y2p⇠iq.

Hence,

|yi`1 ´ wi`1| § |yi ´ wi| ` h |fpti, yiq ´ fpti, wiq| ` h2

2

��y2p⇠iq�� .

Using the fact that f satisfies a Lipschitz condition in the second variable with constant L, and|y2ptq| § M , we have

|yi`1 ´ wi`1| § p1 ` hLq |yi ´ wi| ` h2M

2.

Referring to Lemma 5.2.4 and letting s “ hL, t “ h2M{2, and aj “ |yj ´ wj |, for each j “0, 1, ¨ ¨ ¨ , N , we see that

|yi`1 ´ wi`1| § epi`1qhLˆ|y0 ´ w0| ` h2M

2hL

˙´ h2M

2hL.

Because |y0 ´ w0| “ 0 and pi ` 1qh “ ti`1 ´ t0 “ ti`1 ´ a, this implies that

|yi`1 ´ wi`1| § hM

2Lpepti`1áqL ´ 1q,

for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1.

The weakness of Theorem 5.2.6 lies in the requirement that a bound be known for the secondderivative of the solution. Although this condition often prohibits us from obtaining a realisticerror bound, it should be noted that if Bf{Bt and Bf{By both exist, the chain rule for partialdi↵erentiation implies that

y2ptq “ dy1

dtptq “ df

dtpt, yptqq “ Bf

Bt pt, yptqq ` BfBy pt, yptqq ¨ dy

dtpt, yptqq

“ BfBt pt, yptqq ` Bf

By pt, yptqq ¨ fpt, yptqq.

Hence, it is at times possible to obtain an error bound for y2ptq without explicit knowing yptq.The principal importance of the error-bound formula given in Theorem 5.2.6 is that the bounddepends linearly on the step size h. Consequently, diminishing the step size should give corre-spondingly greater accuracy to the approximations.

Neglected in the result of Theorem 5.2.6 is the e↵ect that round-o↵ error plays in the choice ofstep size. As h becomes smaller, more calculations are necessary and more round-o↵ error isexpected. In actuality, the di↵erence-equation form

w0 “ ↵,

wi`1 “ wi ` hfpti, wiq, for each i “ 0, 1, 2, ¨ ¨ ¨ , N ´ 1,


is not used to calculate the approximation to the solution yi at a mesh point ti. We use insteadan equation of the form

u0 “ ↵ ` �0,

ui`1 “ ui ` hfpti, uiq ` �i`1, for each i “ 0, 1, 2, ¨ ¨ ¨ , N ´ 1,(5.8)

where �i denotes the round-o↵ error associated with ui. Using methods similar to those in theproof of Theorem 5.2.6, we can produce an error bound for the finite-digit approximations to yigiven by Euler’s method.

Theorem 5.2.6. Let yptq denote the unique solution to the initial-value problem

y1 “ fpt, yq a § t § b, ypaq “ ↵.

and tuiuNi“0 be the approximations generated by (5.8). If |�i| † � for each i “ 0, 1, 2, ¨ ¨ ¨ , N andthe hypotheses of Theorem 5.2.6 hold for the initial-value problem, then

|yptiq ´ ui| § 1

L

ˆhM

2` �

h

˙ ”eLptiáq ´ 1

ı` |�0| eLptiáq, (5.9)

for each i “ 0, 1, 2, ¨ ¨ ¨ , N .

The error bound (5.9) is no longer linear in h. In fact, since

limhÑ0

ˆhM

2` �

h

˙“ 8,

the error would be expected to become large for su�ciently small values of h. Calculus can beused to determine a lower bound for the step size h. Letting

Ephq “ hM

2` �

hùñ E1phq “ M

2´ �

h2.

Then, we have the following results:

• If h †a2�{M , then E1phq † 0 and Ephq is decreasing.

• If h °a2�{M , then E1phq ° 0 and Ephq is increasing.

The minimal value of Ephq occurs when h “a2�{M . Decreasing h beyond this value tends to

increase the total error in the approximation. Normally, however, the value of � is su�cientlysmall that this lower bound for h does not a↵ect the operation of Euler’s method.

In fact, the Euler’s method can be obtained by integrating the equation (5.5) in the subintervalrti, ti`1s. To be more precise, we have

dy

dt“ f pt, yptqq ùñ ypti`1q ´ yptiq “

ª ti`1

ti

f pt, yptqq dt.

If we approximate the integral in the right-hand side by

ª ti`1

ti

fpt, yptqq dt « fpti, yptiqqpti`1 ´ tiq “ hfpti, yptiqq,

then we obtain the Euler’s method

ypti`1q « yptiq ` hfpti, yptiqq.


If we approximate the integral in another way:ª ti`1

ti

fpt, yptqq dt « hfpti`1, ypti`1qq,

then we obtain the another method to solve the di↵erential equation. That is, we have

w0 “ ↵,

wi`1 “ wi ` hfpti`1, wi`1q for all i “ 0, 1, ¨ ¨ ¨ , n ´ 1.(5.10)

This method is called backward Euler’s method. In order to find wi`1, one needs to solvean equation in each step ti`1.

In the formula (5.10), when wi is known, one can iteratively solve an approximation of wi`1.

Let wp0qi`1 “ wi `hfpti, wiq be the initial guess of iteration (that is generated by Euler’s method).

Then, we define a sequence of wpsqi`1 as follows:

wp1qi`1 “ wi ` hfptn`1, w

p0qi`1q,

wp2qi`1 “ wi ` hfptn`1, w

p1qi`1q,

¨ ¨ ¨ ¨ ¨ ¨wps`1qi`1 “ wi ` hfptn`1, w

psqi`1q.

Given a small number ", and when��wps`1q

i`1 ´ wpsqi`1

�� † ", we take wps`1qi`1 as a good approximation

of wi`1 in (5.10). If the function fpt, yq is Lipschitz continuous with respect to y variable withconstant L ° 0, one can show that

��wps`1qi`1 ´ wi`1

�� § pLhqs`1��wp0q

i`1 ´ wi`1

�� .

One may conclude that when Lh † 1, wps`1qi`1 converges to wi`1 as s Ñ 8.

5.3 Higher-Order Taylor Methods

In this section, we introduce the higher-order Taylor methods. To determine accurate ap-proximations with minimal e↵ort, one needs a means for comparing the e�ciency of variousapproximation methods.

We first consider the local truncation error of the method. We want to know how wellthe approximations generated by the methods satisfy the di↵erential equation. The local trun-cation will serve quite well to determine not only the local error of a method but the actualapproximation error.

Definition 5.3.1 (Local Truncation Error). Consider the initial value problem

y1 “ fpt, yq, a § t § b, ypaq “ ↵.

The di↵erence method

w0 “ ↵,

wi`1 “ wi ` h�pti, wiq, for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1,

is said to has local truncation error

⌧i`1phq “ yi`1 ´ pyi ` h�pti, yiqqh

“ yi`1 ´ yih

´ �pti, yiq,

for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1, where yi and yi`1 denote the solution at ti and ti`1, respectively.

5.3. HIGHER-ORDER TAYLOR METHODS 87

For example, Euler’s method has local truncation error at the i-th step

⌧i`1phq “ yi`1 ´ yih

´ fpti, yiq, for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1.

This error is a local error because it measures the accuracy of the method at a specific step,assuming that the method was exact at the previous step. As such, it depends on the di↵erentialequation, the step size, and the particular step in the approximation.

By considering (5.6) in the previous section, we see that Euler’s method has

⌧i`1phq “ h

2y2p⇠iq, for some ⇠i P pti, ti`1q.

When y2ptq is known to be bounded by a constant M on ra, bs, this implies that

|⌧i`1phq| § h

2M,

so the local truncation error in Euler’s method is Ophq. One way to select di↵erence-equationmethods for solving ordinary di↵erential equation is in such a manner that their local truncationerrors are Ophpq for as large a value of p as possible, while keeping the number and complexityof calculations of the methods within a reasonable bound.

Since Euler’s method was derived by using Taylor’s Theorem with n “ 1 to approximate thesolution of the di↵erential equation, our first attempt to find methods for improving the conver-gence properties of di↵erence methods is to extend this technique of derivation to larger valuesof n. Suppose that the solution yptq to the initial value problem

y1 “ fpt, yq, a § t § b, ypaq “ ↵

has n ` 1 continuous derivatives. Then, we expand y in terms of its n-th Taylor polynomialabout ti and evaluate at ti`1 and obtain

ypti`1q “ yptiq ` hy1ptiq ` h2

2y2ptiq ` ¨ ¨ ¨ ` hn

n!ypnqptiq ` hn`1

pn ` 1q!ypn`1qp⇠iq, (5.11)

for some ⇠i P pti, ti`1q. Successive di↵erentiation of the solution yptq gives

y1ptq “ fpt, yptqq, y2ptq “ f 1pt, yptqq, and ypkqptq “ f pk´1qpt, yptqq.

Substituting these results into (5.11) gives

ypti`1q “ yptiq ` hfpt, yptiqq ` h2

2f 1pt, yptiqq ` ¨ ¨ ¨

` hn

n!f pn´1qpti, yptiqq ` hn`1

pn ` 1q!fpnqp⇠i, yp⇠iqq.

(5.12)

The di↵erence-equation method corresponding to (5.12) is obtained by deleting the remainderterm involving ⇠i and we have the following Taylor method of order n. We remark that Euler’smethod is Taylor’s method of order one.

Taylor method of order n

w0 “ ↵,

wi`1 “ wi ` hT pnqpti, wiq, for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1,

T pnqpti, wiq “ fpt, yptiqq ` h

2f 1pt, yptiqq ` ¨ ¨ ¨ ` hn´1

n!f pn´1qpti, yptiqq.

(5.13)


Theorem 5.3.2. If Taylor’s method of order n is used to approximate the solution to

y1ptq “ fpt, yptqq, a § t § b, ypaq “ ↵,

with step size h and if y P Cn`1ra, bs, then the local truncation error is Ophnq.

Solution. Note that from (5.11), we have

ypti`1q ´ yptiq ´ hy1ptiq ´ h2

2y2ptiq ´ ¨ ¨ ¨ ´ hn

n!ypnqptiq “ hn`1

pn ` 1q!ypn`1qp⇠iq,

for some ⇠i P pti, ti`1q. So the local truncation error is

⌧i`1phq “ yi`1 ´ yih

´ T pnqpti, yiq “ hn

pn ` 1q!ypn`1qp⇠iq,

for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1. Since y P Cn`1ra, bs, we have ypn`1qptq “ f pnqpt, yptqq bounded onra, bs and ⌧i`1 “ Ophnq, for each i “ 0, 1, ¨ ¨ ¨ , N ´ 1.

Example 5.3.3. We apply Taylor’s method of orders two and four with N “ 10 to the initialvalue problem

y1 “ y ´ t2 ` 1, 0 § t § 2, yp0q “ 0.5.

(a) For the method of order two we need the first derivative of fpt, yptqq with respect to thevariable t. Because y1 “ y ´ t2 ` 1, we have

f 1pt, yptqq “ d

dtpy ´ t2 ` 1q “ y1 ´ 2t “ y ´ t2 ´ 2t ` 1

and

T p2qpti, wiq “ fpti, wiq ` h

2f 1pti, wiq “ wi ´ t2i ` 1 ` h

2pwi ´ t2i ´ 2ti ` 1q

“ˆ1 ` h

2

˙pwi ´ t2i ` 1q ´ hti.

Because N “ 10 we have h “ 0.2, and ti “ 0.2i for each i “ 1, 2, ¨ ¨ ¨ , 10. Thus, thesecond-order method becomes

w0 “ 0.5,

wi`1 “ wi ` h

„ˆ1 ` h

2

˙pwi ´ t2i ` 1q ´ hti

⇢

“ wi ` 0.2

„ˆ1 ` 0.2

2

˙pwi ´ 0.04i2 ` 1q ´ 0.04i

⇢

“ 1.22wi ´ 0.0088i2 ´ 0.008i ` 0.22.

The first two steps gives the approximations

yp0.2q « 0.83 and yp0.4q « 1.2158.

(b) For the method of order four we need the first three derivatives of fpt, yptqq with respectto t. Again, using y1 “ y ´ t2 ` 1, we have

f 1pt, yptqq “ y ´ t2 ´ 2t ` 1,

f2pt, yptqq “ d

dtpy ´ t2 ´ 2t ` 1q “ y1 ´ 2t ´ 2

“ y ´ t2 ´ 2t ´ 2 ` 1 “ y ´ t2 ´ 2t ´ 1,

f3pt, yptqq “ d

dtpy ´ t2 ´ 2t ´ 1q “ y1 ´ 2t ´ 2

“ y ´ t2 ´ 2t ´ 1,

5.3. HIGHER-ORDER TAYLOR METHODS 89

so

T p4qpti, wiq “ fpti, wiq ` h

2f 1pti, wiq ` h2

6f2pti, wiq ` h3

24f3pti, wiq

“ wi ´ t2i ` 1 ` h

2pwi ´ t2i ´ 2ti ` 1q ` h2

6pwi ´ t2i ´ 2ti ´ 1q

` h3

24pwi ´ t2i ´ 2ti ´ 1q,

“ˆ1 ` h

2` h2

6` h3

24

˙pwi ´ t2i q ´

ˆ1 ` h

3` h2

12

˙phtiq

` 1 ` h

2´ h2

6´ h3

24.

Hence, Taylor’s method of order four is

w0 “ 0.5,

wi`1 “ wi ` h

„ˆ1 ` h

2` h2

6` h3

24

˙pwi ´ t2i q ´

ˆ1 ` h

3` h2

12

˙phtiq

`1 ` h

2´ h2

6´ h3

24

⇢,

for i “ 0, 1, ¨ ¨ ¨ , N ´ 1. Because N “ 10 and h “ 0.2, the method becomes

wi`1 “ wi ` 0.2

„ˆ1 ` 0.2

2` 0.04

6` 0.008

24

˙pwi ´ 0.04i2q ´

ˆ1 ` 0.2

3` 0.04

12

˙p0.04iq

`1 ` 0.2

2´ 0.04

6´ 0.008

24

⇢

“ 1.2214wi ´ 0.008856i2 ´ 0.00856i ` 0.2186,

for i “ 0, 1, ¨ ¨ ¨ , 9. The first two steps give the approximations

yp0.2q « 0.8293 and yp0.4q « 1.214091.

0 0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 2

0.5

1

1.5

2

2.5

3

3.5

4

4.5

5

5.5

0.2 0.4 0.6 0.8 1 1.2 1.4 1.6 1.8 210

-6

10-5

10-4

10-3

10-2

10-1

100

Figure 5.3: Comparison of di↵erence numerical methods. Left: solution profiles; right: absoluteerror.


5.4 Runge-Kutta Methods

In this section, we introduce Runge-Kutta methods that have the high-order local truncationerror of the Taylor methods but eliminate the need to compute and evaluate the derivatives offpt, yq. We need the following result of Taylor’s Theorem in two variables.

Theorem 5.4.1. Suppose that fpt, yq and all its partial derivatives of order less than or equalto n ` 1 are continuous on D “ tpt, yq : a § t § b, c § y § du, and let pt0, y0q P D. For everypt, yq P D, there exists ⇠ between t and t0 and µ between y and y0 with

fpt, yq “ Pnpt, yq ` Rnpt, yq,

where

Pnpt, yq “ fpt0, y0q `„

pt ´ t0qBfBt pt0, y0q ` py ´ y0qBf

By pt0, y0q⇢

`„pt ´ t0q2

2

B2f

Bt2 pt0, y0q ` pt ´ t0qpy ´ y0q B2f

BtBy pt0, y0q ` py ´ y0q22

B2f

By2 pt0, y0q⇢

` ¨ ¨ ¨ `«1

n!

nÿ

j“0

ˆnj

˙pt ´ t0qn´jpy ´ y0qj Bnf

Btn´jByj pt0, y0q�

and

Rnpt, yq “ 1

pn ` 1q!n`1ÿ

j“0

ˆn ` 1j

˙pt ´ t0qn`1´jpy ´ y0qj Bn`1f

Btn`1´jByj fp⇠, µq.

The function Pnpt, yq is called the n-th Taylor polynomial in two variables for the functionf about pt0, y0q, and Rnpt, yq is the remainder term associated with Pnpt, yq.

Runge-Kutta methods of order two

The first step in deriving a Runge-Kutta method is to determine values for a1,↵1, and �1 withthe property that a1fpt ` ↵1, y ` �1q approximates

T p2qpt, yq “ fpt, yq ` h

2f 1pt, yq,

with error no greater than Oph2q, which is the same as the order of the local truncation errorfor the Taylor method of order two. Since

f 1pt, yq “ df

dtpt, yq “ Bf

Bt pt, yq ` BfBy pt, yq ¨ y1ptq and y1ptq “ fpt, yq,

we have


2

BfBt pt, yq ` Bf

By pt, yq ¨ fpt, yq. (5.14)

Expanding fpt ` ↵1, y ` �1q in its Taylor polynomial of degree one about pt, yq gives

a1fpt ` ↵1, y ` �1q “ a1fpt, yq ` a1↵1BfBt pt, yq

` a1�1BfBy pt, yq ` a1R1pt ` ↵1y ` �1q,

(5.15)

5.4. RUNGE-KUTTA METHODS 91

where

R1pt ` ↵1y ` �1q “ ↵21

2

B2f

Bt2 p⇠, µq ` ↵1�1B2f

BtBy p⇠, µq ` �21

2

B2f

By2 p⇠, µq, (5.16)

for some ⇠ between t and t`↵1 and µ between y and y `�1. Matching the coe�cients of f andits derivatives in (5.14) and (5.15) gives the three equations

a1 “ 1, a1↵1 “ h

2, and a1�1 “ h

2fpt, yq.

Therefore, we have

a1 “ 1, ↵1 “ h

2, and �1 “ h

2fpt, yq

and

T p2qpt, yq “ f

ˆt ` h

2, y ` h

2fpt, yq

˙´ R1

ˆt ` h

2, y ` h

2fpt, yq

˙,

and from (5.16),

R1

ˆt ` h

2, y ` h

2

˙“ h2

8

B2f

Bt2 p⇠, µq ` h2

4fpt, yq B2f

BtBy p⇠, µq ` h2

8pfpt, yqq2 B2f

By2 p⇠, µq.

If all the second-order partial derivatives of f are bounded, then

R1

ˆt ` h

2, y ` h

2fpt, yq

˙“ Oph2q.

As a consequence, this new method has the same order of error as that of the Taylor method oforder two. The di↵erence-equation method resulting from replacing T p2qpt, yq in Taylor’s methodof order two by

f

ˆt ` h

2, y ` h

2fpt, yq

˙

is a specific Runge-Kutta method known as the Midpoint method.

Midpoint method

w0 “ ↵,

wi`1 “ wi ` hf

ˆti ` h

2, wi ` h

2fpti, wiq

˙, for i “ 0, 1, ¨ ¨ ¨ , N ´ 1.

(5.17)

Only three parameters are present in a1fpt`↵1, y`�1q and all are needed in the match of T p2q.Hence, a more complicated form is required to satisfy the conditions for any of the higher-orderTaylor methods.

If we use the following form

a1fpt, yq ` a2fpt ` ↵2, y ` �2fpt, yqq

to approximate


2f 1pt, yq ` h2

6f2pt, yq

one can obtain, by matching the coe�cients of f and its derivatives in T p3q and the aboveparametric form, the so-called Modified Euler method. In this case, one chooses a1 “ a2 “ 1{2and ↵2 “ �2 “ h. It has the following di↵erence-equation form.


Modified Euler Method

w0 “ ↵,

wi`1 “ wi ` fpti, wiq ` fpti`1, wi ` hfpti, wiqq2

h, for i “ 0, 1, ¨ ¨ ¨ , N ´ 1.(5.18)

Assume that the numerical scheme for solving the initial-value problem is of the following form:

w0 “ ↵,

wi`1 “ wi ` h�pti, wiq.(5.19)

In general, the s-stage Runge-Kutta method has the following form: For a given positive integers ° 0, the increment function �spt, yq is set to be

�spt, yq “ c1k1 ` c2k2 ` ¨ ¨ ¨ ` csks,

where

k1 “ fpt, yq,

ki “ f

˜t ` aih, y ` h

i´1ÿ

j“1

bijkj

¸.

(5.20)

We require thatsÿ

i“1

ci “ 1 and ai “i´1ÿ

j“1

bij .

To determine the coe�cients ai’s, ci’s, and bij ’s, we perform the Taylor’s expansions of T psqpt, yqand �spt, yq and compare the coe�cients of the functions fpt, yq and its derivatives. The mostcommon Runge-Kutta method in use is of order four in di↵erence-equation form, is given by thefollowing.

Runge-Kutta order four

w0 “ ↵,

k1 “ hfpti, wiq,

k2 “ hf

ˆti ` h

2, wi ` 1

2k1

˙,

k3 “ hf

ˆti ` h

2, wi ` 1

2k2

˙,

k4 “ hfpti`1, wi ` k3q,

wi`1 “ wi ` 1

6pk1 ` 2k2 ` 2k3 ` k4q,

(5.21)

for each i “ 0, 1, ¨ ¨ ¨ , N ´1. This method has local truncation error Oph4q, provided the solutionyptq has five continuous derivatives.

Documents

Initial-Value Problems for Ordinary Di erential Equations