Numerical Analysis by Dr. Anita Pal Assistant Professor

Numerical Analysis

by

Dr. Anita Pal

Assistant Professor

Department of Mathematics

National Institute of Technology Durgapur

Durgapur-713209

email: [email protected]

1

.

Chapter 7

Numerical Differentiation and Integration

Module No. 4

Monte-Carlo Method and Double Integration

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

It is mentioned in Module 2 of this chapter that there are three types of quadrature

formulae available in literature, viz. Newton-Cotes, Gaussian and Monte-Carlo. Two

such methods are discussed in Modules 2 and 3. The third method, i.e. Monte-Carlo

method is discussed in this module. This method is based on random numbers, i.e. it

is a statistical method whereas earlier two methods are non-statistical.

4.1 Monte-Carlo method

The Monte Carlo method is a statistical method and applied to solve different type

of problems. For example, it is used in computer based simulation methods, to find

extreme values of a function, etc. This method was invented by a Polish mathematician

Stanislaw Ulam, in the year 1946 and the first paper on it was published in 1949.

The basic principle of this method is stated below.

Generate a set of random numbers. The Monte Carlo method is then performed for this

random numbers for one random trial. The trial is repeated for several number of times

and the trials are independent. The final result is the average of all the results obtained

in different trials.

This method is not suitable for hand calculation, because it depends on a large number

of random numbers.

Now, we discuss the Monte Carlo method to find numerical integration of a single

valued function.

Let the definite integral be

I =

∫ b

ag(x) dx, (4.1)

where g(x) is a real valued function defined on the closed interval [a, b]. Now, this

definite integral is converted to a particular form that can be solved by Monte Carlo

method. For this purpose, a uniform probability density function (pdf) is defined on

[a, b] in the following.

f(x) =

{1

b−a , a < x < b

0, otherwise.

This function is included to the equation (4.1) to obtain the following integral I.

I = (b− a)

∫ b

ag(x)f(x) dx. (4.2)

1

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .Monte-Carlo Method and Double Integration

Note that the integral on the right hand side of the equation (4.2) is nothing but the

expectation of the function g(x) under uniform probability density function. Thus, the

integral I can be written as

I = (b− a)

∫ b

ag(x)f(x) dx = (b− a)g, (4.3)

where g is the expected value or mean value of g(x) over the interval [a, b].

Now, a sample {x1, x2, . . . , } is drawn from the probability density function f(x).

For each such xi the value of g(xi) is evaluated. To get a better approximation, a

large size of sample, say N , is considered. Let G be the average of all such values of

g(xi), i = 1, 2, . . . , N . Then

G =1

N

N∑i=1

g(xi). (4.4)

It can easily be proved that the expectation of the average of N samples is the

expectation of g(x), i.e. G = g. Hence

I = (b− a)G � (b− a)

(1

N

N∑i=1

g(xi)

). (4.5)

The Monte Carlo method is illustrated in Figure 4.1.

�

�x=a x=b

g(x)�

x

y = g(x)

Figure 4.1: Random points are chosen in [a, b]. The value of g(x) is evaluated at each

random point (marked by straight lines in the figure).

Thus the approximate value of the integral I on the interval [a, b] is determined by

taking the average of N results of the integrand with the random variable x distributed

2

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

uniformly over the interval [a, b]. This indicates that the interval [a, b] is finite, because

there is no uniform probability density function in an infinite interval. The infinite

interval can be accommodated with more sophisticated techniques.

The true variance in the average G is equal to the true variance in g, i.e.

var(G) =1

Nvar(g). (4.6)

If an error is committed in the estimation of the integral I with the standard deviation,

then one may expect the error in the estimate of I to decrease by the factor 1/√N .

Since the Monte-Carlo method is based on random numbers, so a suitable method

is required to generate random numbers. Several methods are available to generate

random numbers. In C programming language, the function rand() generates random

numbers.

In the next section, a method is described to generate pseudo-random numbers.

4.2 Generation of random numbers

The random numbers are generated by a random process and these are the values of a

random variable. Several methods are available to generate random numbers, but these

methods do not generate real random numbers, because they follow some mathematical

formula.

A very simple method to generate random numbers is known as power residue

method. In this method, a sequence of non-negative integers x1, x2, . . . is generated by

the following formula:

xn+1 = (a xn) (mod m) (4.7)

where x0 is a starting value called the seed, a and m are two positive integers (a < m).

The expression (axn) (mod m) gives the remainder when axn is divided by m. Note

that the possible values of xn+1 are 0, 1, 2, . . . ,m− 1. That is, the number of different

random numbers is m. The period of random number depends on the values of the

parameter a and the seed x0 and m. The proper values of a, x0 and m generate a long

period random numbers.

Different choices are suggested by many people. One such choice is presented below.

Suppose the word-length of the computational machine be b bits. Let m = 2b−1−1, a is

3


an odd integer of the form 8k±3 and near to 2b/2, and x0 is an odd integer between 0 and

m. Thus, ax0 is a 2b bits integer. The least b significant bits form the random number

x1. The process is repeated for a desired number of times. For a 32-bit computer, a set

of values of the parameters as per above suggestion are m = 231 − 1 = 2147483647, a =

216 + 3 = 65539, x0 = 1267835015, (0 < x0 < m).

The above method generates the random numbers between 0 and m − 1. To obtain

the random numbers between [0, 1], all numbers are divided by m− 1. These numbers

are uniformly distributed over the interval [0, 1].

In C, a function rand() is available in the header file stdlib.h which generates a random

number between 0 and RAND MAX (the maximum integer provided by the computer).

The C statement (float)rand()/RAND MAX may be used to generate random numbers

between 0 and 1. Then the random number between a and b can be obtained by using

the statement a+(b-a)*(float)rand()/RAND MAX.

The following C program finds the value of an integration

∫ 1

0

1

1 + x2dx by Monte-

Carlo method.

/* Program Monte Carlo

Program to find the value of the integration of 1/(1+x^2)

between 0 and 1 by Monte Carlo method for different values of N. */

#include<stdio.h>

#include<stdlib.h>

void main()

{

float g(float x); /* g(x) may be changed accordingly */

float x,y,I,sum=0.0,a=0.0,b=1.0;

int i, N;

srand(100); /* seed for random number */

printf("Enter the sample size ");

scanf("%d",&N);

for(i=0;i<N;i++)

{ /* rand() generates a random number between 0 and RAND_MAX */

y=(float)rand()/RAND_MAX;

/* generates a random number between 0 and 1*/

4

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

x=a+(b-a)*y;

sum+=g(x);

}

I=sum*(b-a)/N;

printf("%f",I);

}

/* definition of function */

float g(float x)

{

return(1/(1+x*x));

}

The results obtained for different values of N are tabulated below.

N value of integration

500 0.790020

1000 0.789627

1500 0.786979

2000 0.787185

3000 0.786553

4000 0.784793

9000 0.784254

10000 0.784094

15000 0.782420

The exact answer is π/4 � 0.785398. No values shown in the above table does not

matched with the exact value (correct up to six decimal places). The closed value is

0.784793 (correct up to three decimal places) and it is obtained for N = 4000.

Note 4.1 Note that the Monte-Carlo method can be used to find the integration of a

function, but it does not produce better result for all type of functions.

The three types of integration methods are discussed and all these methods are appli-

cable for single valued functions. Now, we discuss trapezoidal and Simpson’s methods

to find the double integration.

5


4.3 Double integration

Let us consider the double integration

I =

∫ d

c

∫ b

af(x, y) dx dy. (4.8)

This integration can be evaluated by using the Newton-Cotes, Gaussian and Monte-

Carlo methods. In this section, trapezoidal or Simpson’s methods are considered to find

double integral.

The integral can be evaluated numerically in two successive steps. In first step, the

integration is evaluated with respect to x taking y constant and in second step the

resultant integrand is evaluated with respect to y.

4.3.1 Trapezoidal method

Let the double integral be

I =

∫ d

c

∫ b

af(x, y) dx dy.

Suppose y is constant. Then we integrate I with respect to x by trapezoidal method.

By trapezoidal formula,

I =b− a

2

∫ d

c[f(a, y) + f(b, y)]dy

=b− a

2

[ ∫ d

cf(a, y)dy +

∫ d

cf(b, y)dy

]. (4.9)

In right hand side, there are two integrals with variable y. Again, applying trapezoidal

rule on two integrals of right hand side and obtain the trapezoidal formula for double

integral as

I =b− a

2

[d− c

2{f(a, c) + f(a, d)}+ d− c

2{f(b, c) + f(b, d)}

]

=(b− a)(d− c)

4[f(a, c) + f(a, d) + f(b, c) + f(b, d)]. (4.10)

6

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

In terms of x and y, the above formula can be written as follows:

Let h = b− a, k = d− c, x0 = a, x1 = b, y0 = c, y1 = d. Then

I =hk

4[f(x0, y0) + f(x0, y1) + f(x1, y0) + f(x1, y1)]. (4.11)

Thus, only four values at the corner points (a, c), (a, d), (b, c) and (b, d) of the rect-

angular region [a, b; c, d] are required to find the double integration by trapezoidal rule.

This formula gives a very rough value of the integration.

Note 4.2 The trapezoidal formula finds the repeated integration of the double integral

(4.8). In many times repeated integral is equal to double integral.

Composite trapezoidal formula

The composite trapezoidal formula is generally used to get more better result of the

integral (4.8). In this case, the interval [a, b] is divided into n equal subintervals each of

length h. Similarly, the interval [c, d] is divided into m equal subintervals each of length

k.

That is,

x0 = a, xi = x0 + ih, i = 1, 2, . . . , n; xn = b, h =b− a

n,

y0 = c, yj = y0 + jk, j = 1, 2, . . . ,m; ym = d, k =d− c

m.

Now, integrating (4.8) with respect to x between a and b by using trapezoidal formula

as in case of single variable and we get

∫ b

af(x, y) =

h

2[f(x0, y) + 2{f(x1, y) + f(x2, y) + · · ·

+f(xn−1, y)}+ f(xn, y)]. (4.12)

Again, integrating the above integral with respect to y between c and d by the same

formula, we get

7


I =

∫ d

c

∫ b

af(x, y)dxdy

=h

2

[k

2

{f(x0, y0) + 2(f(x0, y1) + f(x0, y2) + · · ·+ f(x0, ym−1)) + f(x0, ym)

}

+2.k

2

{f(x1, y0) + 2(f(x1, y1) + f(x1, y2) + · · · + f(x1, ym−1)) + f(x1, ym)

}

+2.k

2

{f(x2, y0) + 2(f(x2, y1) + f(x2, y2) + · · · + f(x2, ym−1)) + f(x2, ym)

}+ · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·+2.

k

2

{f(xn−1, y0) + 2(f(xn−1, y1) + · · ·+ f(xn−1, ym−1)) + f(xn−1, ym)

}

+k

2

{f(xn, y0) + 2(f(xn, y1) + f(xn, y2) + · · ·+ f(xn, ym−1)) + f(xn, ym)

}]

=hk

4

[f00 + 2{f01 + f02 + · · · + f0,m−1}+ f0m

+2

n−1∑i=1

{fi0 + 2(fi1 + fi2 + · · ·+ fi,m−1) + fim}

+fn0 + 2(fn1 + fn2 + · · ·+ fn,m−1) + fnm

], (4.13)

where fij = f(xi, yj), i = 0, 1, . . . , n; j = 0, 1, 2, . . . ,m.

The method is of second order in both h and k.

y0 y1 y2 ym−1 ym

x0 [f00 + 2(f01 + f02 + · · · + f0,m−1) + f0,m]k2 = I0

x1 [f10 + 2(f11 + f12 + · · · + f1,m−1) + f1,m]k2 = I1

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·xn [fn0 + 2(fn1 + fn2 + · · ·+ fn,m−1) + fn,m]k2 = In

Table 4.1: Tabular form of trapezoidal formula for double integration

Using the notations of Table 4.1, the trapezoidal formula for double integral can be

8

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

written as

I =h

2[I0 + 2(I1 + I2 + · · · + In−1) + In] where (4.14)

Ij =k

2[fj0 + 2(fj1 + fj2 + · · ·+ fj,m−1) + fj,m], j = 0, 1, . . . , n.

Now, we consider the Simpson’s 1/3 formula which gives more better result than

trapezoidal formula.

4.3.2 Simpson’s 1/3 method

Let the double integral be

I =

∫ d

c

∫ b

af(x, y) dx dy. (4.15)

In Simpson’s 1/3 formula both the intervals are divided into two subintervals. That

is, x0 = a, x1 = x0 + h, x2 = b are the three points on the interval [a, b] and sim-

ilarly y0 = c, y1 = y0 + k, y2 = d be the three points on the interval [c, d], where

h =b− a

2, k =

d− c

2.

Then by Simpson’s 1/3 rule used in single variable case, taking y is constant, we get∫ b

af(x, y) dx =

h

3[f(x0, y) + 4f(x1, y) + f(x2, y)].

Again, by the same rule on each term of the above formula, we get

I =h

3

[k

3{f(x0, y0) + 4f(x0, y1) + f(x0, y2)}

+4.k

3{f(x1, y0) + 4f(x1, y1) + f(x1, y2)}

+k

3{f(x2, y0) + 4f(x2, y1) + f(x2, y2)}

]

=hk

9[f00+f02+f20+f22+4(f01+f10+f12+f21)+16f11], (4.16)

where fij = f(xi, yj), i = 0, 1, 2; j = 0, 1, 2.

In general, over the intervals [xi−1, xi+1] and [yj−1, yj+1] the formula is∫ yj+1

yj−1

∫ xi+1

xi−1

f(x, y) dx dy =hk

9

[fi−1,j−1+fi−1,j+1+fi+1,j−1+fi+1,j+1

+4(fi−1,j+fi,j−1+fi,j+1+fi+1,j)+16fij

]. (4.17)

9


y0 y1 y2 y3 y4 ym−1 ym

x0 [f00 + 4f01 + 2f02 + 4f03 + 2f04 + · · ·+ 4f0,m−1 + f0,m]k3 = I0

x1 [f10 + 4f11 + 2f12 + 4f13 + 2f14 + · · ·+ 4f1,m−1 + f1,m]k3 = I1

· · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · · ·xn [fn0 + 4fn1 + 2fn2 + 4fn3 + 2fn4 + · · · + 4fn,m−1 + fn,m]

k3 = In

Table 4.2: Tabular form of Simpson’s formula for double integration

This formula is known as Simpson’s 1/3 rule for double integration.

The composite Simpson’s 1/3 formula can easily be derived from the Table 4.2.

The composite Simpson’s 1/3 formula is

I =h

3[I0 + 4(I1 + I3 + · · · + In−1) + 2(I2 + I4 + · · ·+ In−2) + In] where (4.18)

Ij =k

3[fj0 + 4fj1 + 2fj2 + 4fj3 + 2fj4 + · · ·+ 4fj,m−1 + fj,m], j = 0, 1, . . . , n.

Example 4.1 Find the value of

∫ 1

0

∫ 1

0

dx dy

(1 + x2)(1 + y2)by trapezoidal and Simpson’s

1/3 formulae taking h = k = 0.25.

Solution. Since h = k = 0.25, let x = 0.00, 0.25, 0.50, 0.75, 1.00 and y = 0.00, 0.25, 0.50, 0.75, 1.00.

Let f(x, y) =1

(1 + x2)(1 + y2). Then the following table gives the values of f(x, y) for

different values of x and y.

y

x 0.00 0.25 0.50 0.75 1.00

0.00 1.00000 0.94118 0.80000 0.64000 0.50000

0.25 0.94118 0.88581 0.75294 0.60235 0.47059

0.50 0.80000 0.75294 0.64000 0.51200 0.40000

0.75 0.64000 0.60235 0.51200 0.40960 0.32000

1.00 0.50000 0.47059 0.40000 0.32000 0.25000

Let x be fixed and y be varying variable. Then by the trapezoidal formula on each

row of the above table, we get

10

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

I0 =

∫ 1

0f(0, y)dy =

0.25

2[1.00000 + 2(0.94118 + 0.80000 + 0.64000) + 0.50000]

= 0.78279.

I1 =

∫ 1

0f(0.25, y)dy =

0.25

2[0.94118 + 2(0.88581 + 0.75294 + 0.60235) + 0.47059]

= 0.73675.

I2 =

∫ 1

0f(0.5, y)dy =

0.25

2[0.80000 + 2(0.75294 + 0.64000 + 0.51200) + 0.40000]

= 0.62624.

I3 =

∫ 1

0f(0.75, y)dy =

0.25

2[0.64000 + 2(0.60235 + 0.51200 + 0.40960) + 0.32000]

= 0.50099.

I4 =

∫ 1

0f(1.0, y)dy =

0.25

2[0.50000 + 2(0.47059 + 0.40000 + 0.32000) + 0.25000]

= 0.39140.

Hence, finally∫ 1

0

∫ 1

0

dx dy

(1 + x2)(1 + y2)=

h

2[I0 + 2(I1 + I2 + I3) + I4]

=0.25

2[0.78279 + 2(0.73675 + 0.62624 + 0.50099) + 0.39140]

= 0.61277.

Again, by Simpson’s 1/3 formula to each row of the above table, we have

I0 =

∫ 1

0f(0, y)dy =

0.25

3[1.00000 + 4(0.94118 + 0.64000) + 2(0.80000) + 0.50000]

= 0.78539.

Similarly, I1 = 0.73919, I2 = 0.62831, I3 = 0.50265, I4 = 0.39270.

Hence, by Simpson’s 1/3 formula, the double integration is

I =h

3[I0 + 4(I1 + I3) + 2I2 + I4]

=0.25

3[0.78539 + 4(0.73919 + 0.50265) + 2(0.62831) + 0.39270]

= 0.61684.

11


Merits and demerits of the quadrature formulae

Three types of quadrature formulae are discussed in last three modules. Their ad-

vantages, disadvantages and limitations are discussed below.

(i) The Newton-Cotes quadrature formulae are applicable only for proper integral.

Whereas Gaussian quadrature formulae are useful for both proper and improper

integrals.

(ii) The simplest but crudest integration formula is the trapezoidal formula. This

formula gives a rough value of the integral using a very few number of calculations.

(iii) Simpson’s 1/3 rule gives more accurate result than the trapezoidal formula and it

is also simple. If f(x) does not fluctuate rapidly and is explicitly known then this

method with a suitable subinterval can be used. If high accuracy is required then

the Gaussian quadrature may be used.

(iv) If the integrand f(x) has infinite discontinuity(ies) at one or both of the limits,

then open type Newton-Cotes formulae may be used. But, in this case, Gaussian

quadrature gives more better result.

(v) Gauss-Legendre or Gauss-Chebyshev formulae give more accurate result than the

trapezoid, Simpson’s 1/3, Simpson’s 3/8, Boole and Weddle’s formulae with less

number of calculations.

(vi) The Gauss-Legendre or Gauss-Chebyshev formulae may directly be applied for

second type improper integral. If a function is second type improper and it has

singularity(ies) at some nodes of k-point Gauss-Legendre (or Gauss-Chebyshev)

formula, then use either (k−1)-point or (k+1)-point or any other Gauss-Legendre

(or Gauss-Chebyshev) formula to avoid such singularities. Provided that the im-

proper integral is convergent.

(vii) If one of the limits of the integration is infinite, then replace x by 1/x to make it

finite. Then use appropriate formula.

(viii) Simpson’s 1/3 formula gives more better result than trapezoidal formula for double

integration also. Provided that the repeated integral and double integral are same

12

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .

for the given problem, otherwise Simpson’s 1/3 (and also trapezoid) formula gives

completely different answer.

(ix) If the integrand is violently oscillating or fluctuating then the Monte Carlo method

can be used.

13

Documents

Numerical Analysis by Dr. Anita Pal Assistant Professor