CHAPTER 1

Numerical Methods for Ordinary DifferentialEquations

In this chapter we discuss numerical method for ODE . We willdiscuss the two basic methods, Euler’s Method and Runge-KuttaMethod.

1. Numerical Algorithm and Programming in Mathcad

1.1. Numerical Algorithm. If you look at dictionary, you willthe following definition for algorithm,

1. a set of rules for solving a problem in a finitenumber of steps; 2. a sequence of steps designed forprogramming a computer to solve a specific problem.

A numerical algorithm is a set of rules for solving a problem in finitenumber of steps that can be easily implemented in computer using anyprogramming language. The following is an algorithm for compute theroot of f(x) = 0,

Input f , a, N and tol .Output: the approximate solution to f(x) = 0 withinitial guess a or failure message.• Step One: Set x = a• Step Two: For i=0 to N do Step Three - Four

Step Three: Compute x = x− f(x)f ′(x)

Step Four: If f(x) ≤ tol return x• Step Five return ”failure”.

In analogy, a numerical algorithm is like a cook recipe that specify theinput — cooking material, the output—the cooking product, and stepsof carrying computation — cooking steps.

In an algorithm, you will see loops (for, while), decision makingstatements(if, then, else(otherwise)) and return statements.

• for loop: Typically used when specific number of steps needto be carried out. You can break a for loop with return orbreak statement.

1

2 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

• while loop: Typically used when unknown number of stepsneed to be carried out. You can break a while loop with abreak statement or a return statement.

• if – else(otherwise): Used when condition(s) must meet be-fore carrying out some steps. The else(otherwise) provide al-ternative steps if any.

1.2. Programming in Mathcad. To translate a numerical al-gorithm into a program in Mathcad , you need to define a functionname with function arguments and function body. The functionname can be any sequence of alphabetic letter begin with an Englishletter, such any ”myFunction”, ”Myfunction1” etc. The function ar-guments specify the input to the function, typically the items specifiedin the input statement of an algorithm and is an comma(,) separatedlist enclose by () after function name, such as myFunction(f, a, N, tol),myOne(a,b). Notice the function argument is optional, you can definea function with out argument as myFunction(). The function bodyimplements the step section of an algorithm and the output statement.In Mathcad the function body is on the right side of := with verticalbars as grouping symbol. The entire function body is considered asa group, inside, it might has many subgroups. For the algorithm offinding f(x) = 0, there is an subgroup that contains step three andfour, as shown in the following screen shot, the screen shot also showshow to get the programming toolbar from the math toolbar.

Figure 1. mySolver and Programming toolbar

1. NUMERICAL ALGORITHM AND PROGRAMMING IN Mathcad 3

To enter code in Mathcad ,

(1) First enter the function name with arguments list and assignment operator:=, with sequence of typing,

mySolver(f,a,N,tol):

(2) Click the Add Line button or press ] to get a vertical bar and two (youshould add more by press ] several times).

(3) enter x and hold [Shift] type ] to get the local assignment ← or click on theprogramming toolbar, at the type a. Notice x ← a reads as ”assign a to x.”

(4) Click for on the programming menu or [Shift][Ctrl][’] to get

for ∈

in after for enter i, in after ∈, enter ”1;N”, and in under for, press leftbracket ] to get a vertical line and several .

(5) To get if operator, you need either click on the toolbar or press [Shift][Ctrl][] ]. you will get

if

enter condition in after if and other statements in before if.(6) To get return operator, you need either click on the toolbar or press

[Shift][Ctrl][ ], you getreturn

in type any information you want to return.

After defining a function, we call it with concrete input data, asshown in the screen shot, that call the mySolver function to find zerofor both f(x) = x2−1, with initial guess x = 2 and g(t) = t3−3t2−2t,with initial guess t = −2. Notice there is two ways to call a function,one way is to define all arguments before calling the function, anotheris to define some arguments and call the function with concrete valuesfor the undefined arguments.

As in all programming language, Mathcad allows you to call onefunction from within another. For example, we could create a functioncalled myDerivative which will compute derivative of a given func-tion f(x) at a given number x and step size h > 0 using the forward

difference formula: f ′(s) ≈ f(x+h)−f(x)h

. The following screen shot showsthe call of myDerivative inside mySolver,

Figure 2. Call another function

4 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

The algorithm for this function is very simple (here, we break downthe computation in three steps),

Input f , x, and h .Output: the approximation to f ′(x) .• Step One: Set w = f(x + h)• Step Two: Set d = f(x)• Step Three: return w−d

h

2. Euler’s Method

2.1. Euler’s Method. Euler’s method is the simplest method infind approximate solutions to first order equations. From the forwarddifference formula

f ′(x) ≈ f(x + h)− f(x)

h,

we have

f(x + h) ≈ f(x) + f ′(x)h(1)

Now if x′(t) = f(t, x) is a first order differential equations, apply (1),we have

x(t + h) ≈ x(t) + f(t, x(t))h.

Suppose we want to find approximate solution over interval [a, b] withinitial value x(a) = x0, we divided the interval into n subintervalseach with length h = b−a

n, the ends of the subintervals is t0 = a, t1 =

a + h, t2 = a + 2h, · · · , tn = a + nh = b. Start with x0 we can compute

x1 = x0 + f(t0, x0)hx2 = x1 + f(t1, x1)h...xn = xn−1 + f(tn−1, xn−1)h

Example 2.1. Find approximate to x(1) if x′(t) = t2−ex sin(t), x(0) =1 with h = 0.25.

Solution Here the interval is [0, 1], so a = 0, b = 1. Since h =b−an

= 0.25 we have n = 4 and we need to compute x1, x2, x3, x4 startingwith x0 = x(0) = 1.

x1 = x0 + f(t0, x0)h = 1 + f(0, 1) ∗ 0.25 = 0.15853x2 = x1 + f(t1, x1)h = 0.15853 + f(0.25, 0.15853) ∗ 0.25 = 0.01833x3 = x2 + f(t2, x2)h = 0.01833 + f(0.5, 0.01833) ∗ 0.25 = 0.23811x4 = x3 + f(t3, x3)h = 0.23811 + f(0.75, 0.15853) ∗ 0.25 = 0.30128

So the approximation to x(1) is x(1) ≈ x4 = 0.30128. Also x(0.25) ≈x1 = 0.15853, x(0.5) ≈ x2 = 0.01833, x(0.75) ≈ x3 = 0.23811. a

2. EULER’S METHOD 5

2.2. Mathcad implementation of Euler’s Method. First, wehave the following simple algorithm for the Euler’s method,

Input f , a, b, x0 n.Output: the approximate solution to x′ = f(t, x)with initial guess x0 over interval [a, b].• Step One: Initialization

Set h = b−an

Set x0 = x0Set t0 = a

• Step Two: For i=1 to n do Step Three

Step Three: Set xi = xi−1 − f(ti−1, xi−1) ∗ hSet ti = ti−1 + h

• Step Four return x.

Notice, algorithm return an array of values, the ith element of thereturn array is an approximations of x(t) at t = a + ih.

The following screen shot shows Mathcad code for implementingthe algorithm. Notice, we also create a function tMesh(a, b, N) tocompute the ending of subinterval for graphing purpose.

Figure 3. Mathcad code for Euler’s Method

Notice the line to line corresponding between the Mathcad and thealgorithm. Since Mathcad programming language is a scripting lan-guage, the translation between algorithm and code is straight forward,and you don’t need to worry about the variable type, io, etc. Also,without explicit return statement, the result of last is, by default, willbe returned.

6 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

The next screen shot shows a call to the myEuler and tMesh forthe equation x′ = t2x + t2sin(t3). It also shows the graph of approxi-mate solution comparing with the exact solution x(t) = − 3

10cos(t3) −

110

sin(t3) + 310

e110

t3

Figure 4. Mathcad Euler’s Approximation to x′ = t2x + t2sin(t3)

2.3. Error analysis of Euler’s Method. There are two sourceof error in every numerical method used to approximate a solution ofx′(t) = f(t, x),

• Local Truncation Error• The roundoff error.

The local truncation error is due to the method and roundoff erroris due to computer that is used. For Euler’s method, we use xi =xi−1 +f(ti−1, xi−1)∗h to approximate the value of x(ti), the difference,assuming no rounding is introduced, is |xi − x(ti)|, and is called thelocal truncation error. The following theorem shows how the localtruncation error is depending on f(t, x) and h,

Theorem 2.1. Suppose f(t, x), ∂f(t,x)∂t

, and ∂f(t,x)∂x

are continuous

on [a, b], and h = b−1n

is the step size. Furthermore let x(t) be thesolution of initial value problem x′ = f(t, x), x(a) = x0 and xi = xi−1 +f(ti−1, xi−1) ∗ h be the approximate of x(ti), and let ei = x(ti) − xi be

3. RUNGE-KUTTA METHOD 7

the local truncation error, then

ei+1 ≤ (1 + hK)|ei|+ 1

2h2M, i = 1, ...n.

, where |∂f(t,x)∂x

| ≤ K, and |x′′(t)| ≤ M.

Theorem 2.2 indicates that, without roundoff error, the smaller hgives better approximate solution. However, due the roundoff errorintroduced in each step of computation, (such as compute can onlygive approximate value for

√2) we can’t choose h arbitrarily small.

The following result gives the optimal step size h,

Theorem 2.2. With the same assumption of Theorem 2.2,and sup-pose at each step the roundoff error is bounded by ε then the optimalvalue for the step length is

hopt =

√2ε

M

3. Runge-Kutta Method

3.1. Runge-Kutta Method. There are several Runge-Kutta meth-ods, but the fourth order Runge-Kutta method is most popular. Sup-pose x′(t) = f(t, x), x(a) = x0. and h = b−a

nis the step length, the

fourth order Runge-Kutta method is given below,

k1 = hf(ti, xi)k2 = hf(ti + 1

2h, xi + 1

2k1)

k3 = hf(ti + 12h, xi + 1

2k2)

k4 = hf(ti + h, xi + k3)xi+1 = xi + 1

6(k1 + 2k2 + 2k3 + k4)

In general Runge-Kutta method gives more accurate result than Eu-ler’s method at the same step length. However, Runge-Kutta methodis more expensive to implement, that is, at each step, Runge-Kuttamethod requires more computation than Euler’s method( four func-tion evaluations compare one in Euler’s method).

Example 3.1.

Example 3.2. Find approximate to x(1) if x′(t) = t2−ex sin(t), x(0) =1 with h = 0.25.

Solution Here the interval is [0, 1], so a = 0, b = 1. Sinceh = b−a

n= 0.25 we have n = 4 and we need to compute x1, x2, x3, x4

starting with x0 = x(0) = 1, t0 = 0

8 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

x1:

k1 = hf(t0, x0) = 0.25 ∗ f(0, 1) = 0k2 = hf(t0 + 1

2h, x0 + 1

2k1) = f(0.125, 1) = −0.373276

k3 = hf(t0 + 12h, x0 + 1

2k2)

= 0.25f(0.125, 1 + 0.5(−0.373276)) = −0.373276k4 = hf(t0 + h, x0 + k3) = 0.25f(0.25, 1− 0.373276) = −0.309854x1 = x0 + 1

6(k1 + 2k2 + 2k3 + k4)

= 1 + 16(0 + 2 ∗ (−0.373276) + 2 ∗ (−0.373276)− 0.309854)

= 0.923911t1 = t0 + h = 0 + 0.25 = 0.25

x2:

k1 = hf(t1, x1) = 0.25 ∗ f(.25, 0.923911) = −0.560741k2 = hf(t1 + 1

2h, x1 + 1

2k1)

= f(0.125, 0.923911 + 0.5(−0.560741)) = −0.719601k3 = hf(t1 + 1

2h, x1 + 1

2k2)

= 0.25f(0.125, 0.923911 + 0.5(−0.719601)) = −0.702688k4 = hf(t1 + h, x1 + k3)

= 0.25f(0.5, 0.923911− 0.702688) = −0.763158x2 = x1 + 1

6(k1 + 2k2 + 2k3 + k4)

= 0.923911 + 16(−0.560741 + 2 ∗ (−0.719601) + 2 ∗ (−0.702688)− 0.763158)

= 0.750224t2 = t1 + h = 0.25 + 0.25 = 0.5

x3:

k1 = hf(t2, x2) = 0.25 ∗ f(.5, 0.750224) = −0.765171k2 = hf(t2 + 1

2h, x2 + 1

2k1)

= f(0.625, 0.750224 + 0.5(−0.765171)) = −0.735295k3 = hf(t2 + 1

2h, x2 + 1

2k2)

= 0.25f(0.625, 0.750224 + 0.5(−0.735295)) = −0.739508k4 = hf(t2 + h, x2 + k3)

= 0.25f(0.75, 0.750224− 0.739508) = −0.637224x3 = x2 + 1

6(k1 + 2k2 + 2k3 + k4)

= 0.750224 + 16(−0.765171 + 2 ∗ (−0.735295) + 2 ∗ (−0.739508)− 0.637224)

= 0.568891t3 = t2 + h = 0.5 + 0.25 = 0.75

3. RUNGE-KUTTA METHOD 9

x4:

k1 = hf(t3, x3) = 0.25 ∗ f(0.75, 0.568891) = −0.641483k2 = hf(t3 + 1

2h, x3 + 1

2k1)

= f(0.875, 0.568891 + 0.5(−0.641483)) = −0.485628k3 = hf(t3 + 1

2h, x3 + 1

2k2)

= 0.25f(0.875, 0.568891 + 0.5(−0.485628)) = −0.510243k4 = hf(t3 + h, x3 + k3)

= 0.25f(1, 0.568891− 0.510243) = −0.308297x4 = x3 + 1

6(k1 + 2k2 + 2k3 + k4)

= 0.568891 + 16(−0.641483 + 2 ∗ (−0.485628) + 2 ∗ (−0.510243)− 0.308297)

= 0.446327t4 = t3 + h = 0.75 + 0.25 = 1.0

So the approximation to x(1) is x(1) ≈ x4 = 0.446327. a

3.2. Mathcad implementation. First, we have the following sim-ple algorithm for the Euler’s method,

Input f , a, b, x0 n.Output: the approximate solution to x′ = f(t, x)with initial guess x0 over interval [a, b].• Step One: Initialization

Set h = b−an

Set x0 = x0Set t0 = a

• Step Two: For i=1 to n do Step ThreeStep Three:

Set k1 = hf(ti−1, xi−1)Set k2 = hf(ti−1 + 1

2h, xi−1 + 1

2k1)

Set k3 = hf(ti−1 + 12h, xi−1 + 1

2k2)

Set k4 = hf(ti−1 + h, xi−1 + k3)Set xi = xi + 1

6(k1 + 2k2 + 2k3 + k4)

Set ti = ti−1 + h• Step Four return x.

This algorithm returns an array of values, the ith element of thereturn array is an approximations of x(t) at t = a + ih.

The following screen shot shows Mathcad code for implementingthe algorithm. Again notice the line to line corresponding between theMathcad and the algorithm.

10 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Figure 5. Mathcad code for Euler’s Method

The next screen shot shows a call to the myRungeKutta and tMeshfor the equation x′ = t2x+t2sin(t3). It also shows the graph of approx-imate solution comparing with the exact solution x(t) = − 3

10cos(t3)−

110

sin(t3) + 310

e110

t3 , and with the approximate solution using Euler’smethod. In this case Runge-Kutta provides much superior result, analmost exact match!

Figure 6. Mathcad Runge-Kutta’s Approximation to x′ = t2x + t2sin(t3)

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 11

4. Numerical Method for System of Equations

Both Euler’s method and Runge-Kutta method can be used to findthe approximate solution to the system of first order differential equa-tions. In fact, the Mathcad codes for system of first differential equa-tions are exactly the same as Mathcad does not differentiate scalar andvectors when performs most computations. The only thing needs to betaken care is that at each step the result is vector instead of a scalar.Suppose x(t) is a vector-valued function that satisfies

x′(t) = F (t, x(t)),

where F (t, x) is also vector-valued function, ti = a+ih then the Euler’smethod compute the ith approximate,

xi = xi−1 + F (ti−1,xi−1).(2)

Example 4.1. Let

x′(t) = t2 sin(x(t)) + et cos(y(t))y′(t) = 2tx(t) + ey(t)

x(0) = 1, y(0) = −1

Find approximate solution for x(t), y(t) over interval [0, 1] with h =0.1

Solution Set x(t) =

[x(t)y(t)

]and F (t, x) =

[t2 sin(x) + et cos(y)

2tx + ey

]

The equation can then be written, in vector form,

x′(t) = F (t, x(t)),

x(0) =

[1−1

]From (2) we can find, with t0 = 0,

x1 = x0 + h ∗ F (t0,x0)

=

[1−1

]+

[t20 sin(x0) + et0 cos(y0)

2t0x0 + ey0

]

=

[cos(−1)

e−1

]

The following screen shot shows the results obtain by calling ourMathcad implementation of Euler’s method. Notice that in definingthe vector-valued function F (t, X), for Mathcad to know that X is anvector, in the definition, we use subscript to access the component ofX, as shown in the screen shot.

12 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Figure 7. Euler’s method for system of equations

a

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 13

Similarly, we can use the same Mathcad code of Runge-Kutta methodto find approximate solution as shown in the following screen shot forthe above system of equations,

Figure 8. Runge-Kutta’s method for system of equations

4.1. Numerical Method for Higher Order Equations. Tofind numerical approximate for solutions of higher order differentialequations using either Euler’s method or Runge-Kutta method, wefirst transform an higher order differential equations into a system offirst order differential equations and then apply the methods.

Example 4.2. Rayleigh Equation In modeling the oscillationsof a clarinet reed. Lord Rayleigh introduced an equation of the form

mx′′ + kx = ax′ − b(x′)3

Find approximate solution for m = 1, k = 1, a = 2, b = 3. and initialconditions x(0) = 1, x′(0) = −2

14 1. NUMERICAL METHODS FOR ORDINARY DIFFERENTIAL EQUATIONS

Solution Suppose y(t) = x′(t) is the velocity, then Rayleighequation becomes

x′ = yy′ = 2y − 3y3 − 2x

So we apply the numerical method for F =

[y

2y − 3y3 − 2x

]and

x′(t) = F (t, x) with

[x(t) = x(t)

y(t)

],x(0) =

[1−2

]. The following

screen shot gives approximate solution using Runge-Kutta method,

Figure 9. Runge-Kutta’s method for Rayleigh’s equation

awe also plot the solution in xy-plane, which is called the velocity-

position phase plane, with y-axis represent the velocity and x-axis is the

4. NUMERICAL METHOD FOR SYSTEM OF EQUATIONS 15

position. The velocity-position plane shows that all velocity-positionsolution pairs are eventually being attracted to an closed orbit, theclosed orbit is called the global attractor.