Chapter 3

Some Mathematical Preliminaries

The concepts of limits, sequences and series, differentiation and integration, matri-ces and determinants of real and complex numbers are required for the study ofnumerical methods. Also some familiarity with the solution of ordinary differentialequations is required.

♣ Definition:-Let f(x) be a continuous function on set X of real numbers; f is said to have

the limit L at x◦ writtenlim

x→x◦f(x) = L

if given any real number ε > 0, there exist a real number δ > 0 such that |f(x)−L| <ε whenever |x − x◦| < δ for xεX.

When the h−increment notation x = x◦+h is used, above equation is equivalentto

limh→◦

f(x◦ + h) = L

♣ Definition:-Let f(x) be a function defined on a set X of real numbers; and x◦εX; f is said

to be continuous at x◦ , if

limx→x◦

f(x) = f(x◦).

Equivalently,limh→◦

f(x◦ + h) = f(x◦).

The function f(x) is said to be continuous on X if it is continuous at each pointxεX.

The functions used when discussing numerical methods are assumed to be con-tinuous, since this is the minimal requirement for predictable behavior. Functionthat is not continuous can skip over points of interest, which is not a satisfactorytrait when attempting to approximate a solution.

45

♣ Definition:-Suppose that {xn}∞n=1 is an infinite sequence. Then the sequence is said to

have the limit L, and we write

limn→∞

xn = L. (3.1)

if given any ε > 0, there exists a positive integer N = N(ε) such that

n > N implies that |xn − L| < ε.

When a sequence has a limit, we say that it is a convergent sequence. Anotherpapular notation is that xn → L as n → ∞. Equation (??) is equivalent to

limn→∞

(L − xn) = 0.

Thus we can view the sequence εn = L − xn as an error sequence.

♣ Definition:-

Let {an}∞n=1 be a sequence. Then∞∑

n=1

an is an infinite series. The nth patrial sum

is Sn =n∑

k=1

ak. The infinite series converges if and only if the sequence {Sn}∞n=1

converges to a limit S, that is,

limn→∞

Sn = limn→∞

n∑

k=1

ak = S.

If a series does not converge, we say that it diverges.

♣ Definition:-If f(x) is a function defined in an open interval containing x◦. f(x) is said to be

differentiable at x◦ , if

limx→x◦

f(x) − f(x◦)

x − x◦

exists. When this limit exists it is denoted by f ′(x◦) and is called the derivativeof f(x) at x◦. An equivalent way to express this limit is to use the h−incrementnotation:

limh→◦

f(x◦ + h) − f(x◦)

h= f ′(x◦)

A function that has a derivative at each number in a set X is said to be differ-entiable on X. The number m = f ′(x◦) is the slope of the tangent line to thecurve y = f(x) at (x◦, f(x◦)).

46

♣ Definition:-If f(x) is differentiable at x◦ , then f(x) is continuous at x◦.

♣ Definition:-A function f(x) is said to be analytic at x = x◦ if f(x) can be represented by a

power series in powers of x − x◦ within a radius of convergence, D > |x − x◦| > 0.A necessary condition for a function to be analytic is that all its derivatives be con-tinuous at x = x◦ and in the neighborhood.

♣ Definition:-A point at which a function f(x) is not analytic is called a singular point. If

f(x) is differentiable everywhere in the neighbourhood of x◦ except at x◦, then x◦is a singular point. For example, tan(x) is analytic except at x = ±(n + 1

2)π, n =

0, 1, 2, 3, · · · ,∞, which are singular points. Polynomials are analytic everywhere.

♣ Definition:-Assume that function f(x) is continuous on the interval [a, b] and suppose that

a = x◦ ≤ x1 ≤ x2 ≤ · · · ≤ xn = b is a partition of [a, b]. For each i = 1, 2, 3, · · · , n,select an arbitrary point zi in the subinterval [xi−1, xi] and introduce the differencenotation ∆xi = xi − xi−1. Then the sum

n∑

i=1

f(zi)∆xi

is called a Riemann sum approximation for the definite integral f(x) over [a, b].

♣ Definition:-The Riemann integral of the function f(x) on the interval [a, b] is the following

limit, provided it exists:-

∫ b

af(x) dx = lim

max∆xi→0

n∑

i=1

f(zi)∆xi

where the numbers x◦, x1, x2, · · · , xn satisfy a = x◦ ≤ x1 ≤ x2 ≤ · · · ≤ xn = b andwhere, for each i = 1, 2, 3, 4, · · · , n, ∆xi = xi − xi−1 and zi is arbitrarly chosen inthe interval [xi−1, xi].

A function f that is continuous on an interval [a, b] is Riemann integrable onthe interval. This permits us to choose, for computational convenience, the pointsxi to be equally spaced in [a, b] and for each i = 1, 2, 3, 4, · · · , n to choose zi = xi. Inthis case ∫ b

af(x) dx = lim

n→∞

b − a

n

n∑

i=1

f(xi)

47

where xi = a +i(b − a)

n

♠ Theorem:- Intermediate-Value Theorem for Continuous Func-tions

Let f(x) be a continuous function on the interval [a, b]. If f(x1) ≤ α ≤ f(x2) forsome number α and some x1, x2 ε [a, b], then

α = f(ξ) for some ξ ε [a, b]

♠ Theorem:- Extreme-Value Theorem for a Continuous FunctionIf f(x) be a continuous function on the closed and finite interval [a, b]. Then there

exists a lower bound M1 and an upper bound M2 and and two numbers c1, c2 ε [a, b]such that

M1 = f(c1) ≤ f(x) ≤ f(c2) = M2 for each x ε [a, b].

If in addition, f(x) is differentiable on (a, b) , then the number c1 and c2 occur eitherat endpoints of [a, b] or where f ′(x) is zero.

♠ Theorem:- Mean-Value Theorem for DerivativesIf f(x) be a continuous function on the closed and finite interval [a, b], and

differentiable on (a, b), then

f(b) − f(a)

b − a= f ′(ξ) for some ξ ε (a, b)

♠ Theorem:- Rolle’s Theorem

48

Let f(x) be a continuous function on the closed and finite interval [a, b], anddifferentiable on (a, b). If f(a) = f(b) = 0, then

f ′(ξ) = 0 for some ξ ε (a, b)

♠ Theorem:- Generalized Rolle’s TheoremLet f(x) ε [a, b] be n times differentiable on (a, b). If f vanishes at the (n + 1)

distinct numbers x◦, x1, x2, · · · , xn in [a, b], then a number c in (a, b) exists with

f (n)(c) = 0

♠ Theorem:- First Fundamental TheoremIf f(x) is continuous function over [a, b]. Then there exists a function F, called

the antiderivative of f , such that∫ b

af(x) dx = F (b) − F (a) where F ′(x) = f(x).

♠ Theorem:- Second Fundamental TheoremIf f(x) is continuous function over [a, b] and a < x < b, then

d

dx

∫ x

af(t) dt = f(x).

♠ Theorem:- Mean-Value Theorem for IntegralsIf f(x) is continuous function over [a, b] and a ≤ x ≤ b, then there exists a

number c ε (a, b) such that

f(c) =1

b − a

∫ b

af(x) dx

♠ Theorem:- Weighted Mean-Value Theorem for IntegralsIf f ε [a, b], g is integrable on [a, b], and g(x) does not change sign on [a, b], then

there exists a number c ε (a, b) with

∫ b

af(x) g(x) dx = f(c)

∫ b

ag(x) dx

when g(x) ≡ 1, we get Mean-Value Theorem for Integrals.♠ Theorem:- Taylor Theorem for One-Dimensional FunctionsSuppose f(x) is n times differentiable on [a, b], f (n+1) exists on [a, b] and x◦ ε [a, b].

For every x ε [a, b],

f(x) = f(x◦) + (x − x◦)f′(x◦) +

(x − x◦)2

2!f ′′(x◦) +

(x − x◦)3

3!f ′′′(x◦)

+(x − x◦)

4

4!f (4)(x◦) + · · · + (x − x◦)

n

n!f (n)(x◦) + Rn(x) (3.2)

49

where

Rn(x) =1

n!

∫ x

x◦(x − s)nf (n+1)(s) ds

Actually f (n+1)(x) need not be continuous for (??) to hold. However, if in (??) ,f (n+1)(x) is continuous, one gets

Rn(x) =f (n+1)(ξ)(x − x◦)

(n+1)

(n + 1)!where ξ = x◦ + θ(x − x◦) (3.3)

Put x − x◦ = h then (??) and (??) take the form

f(x◦ + h) = f(x◦) + hf ′(x◦) +h2

2!f ′′(x◦) +

h3

3!f ′′′(x◦) +

h4

4!f (4)(x◦) + · · ·

+hn

n!f (n)(x◦) +

hn + 1

(n + 1)!f (n+1)(x◦ + θh) (3.4)

for some θ ε (0, 1)The infinite series is obtained by taking the limit as n → ∞, is called the Tay-

lor series for f about x◦. In the case x◦ = 0, the Taylor polynomial is called aMaclaurin polynomial and Taylor series is called a Maclaurin series.

♠ Theorem:- Taylor series of a Two-Dimensional FunctionThe Taylor expansion of a two dimensional function f(x, y) about (a, b) is given

by

f(x, y) = f(a, b) + hfx + kfy +1

2

[h2fxx + 2hkfxy + k2fyy

]

+1

6

[h3fxxx + 3h2kfxxy + 3hk2fyy + k3fyyy

]

+1

24

[h4fxxxx + 4h3kfxxxy + 6h2k2fxxyy + 4hk3fxyyy + k4fyyyy

]+ · · ·

where

h = x − a, k = y − b

fx =∂

∂xf(x, y)|x=a,y=b

fy =∂

∂yf(x, y)|x=a,y=b

and similar notations such as fx···x, fxy···, and fyy··· are partial derivatives of f atx = a and y = b; each x and y in subscripts indicates one time of partial differenti-ation with respect to x or y, respectively.

50

⇒ Derivation of Taylor’s SeriesConsider the nth degree polynomial

P (x) = b◦ + b1x + b2x2 + b3x

3 + b4x4 + · · ·+ bnxn

If we are interested in P (x) in the neighborhood of some value x = a. Then

P (a + h) = b◦ + b1(a + h) + b2(a + h)2 + b3(a + h)3 + b4(a + h)4 + · · ·+ bn(a + h)n

which we could expand and write as a polynomial in h

P (a + h) = b◦ + b1a + b1h + b2a2 + b2h

2 + 2b2ah

+ · · · + bn(an + han−1 + h2an−2 + · · · + hn)

= c◦ + c1h + c2h2 + c3h

3 + c4h4 + · · · + cnh

n (3.5)

Since this equation is true for h = 0, c◦ = P (a), differentiating upto n times andeach derivative evaluated for h = 0 we get

c◦ = P (a) (3.6)

P ′(a + h) = c1 + 2c2h + 3c3h2 + 4c4h

3 + · · · + ncnhn−1

⇒ P ′(a) = c1 (3.7)

P ′′(a + h) = 2c2 + 6c3h + 12c4h2 + · · ·+ n(n − 1)cnhn−2

⇒ P ′′(a) = 2c2

⇒ c2 =P ′′(a)

2!(3.8)

· · · · · ·

⇒ cn =P (n)(a)

n!(3.9)

Thus

P (a + h) = P (a) + hP ′(a) +h2

2!P ′′(a) +

h3

3!P ′′′(a) + · · · + hn

n!P (n)(a)

(3.10)

Example:-1Taylor’s expansion of e−x about x◦ = 1

e−x = e−1 − (x − 1)e−1 +(x − 1)2

2!e−1 − (x − 1)3

3!e−1 +

(x − 1)4

4!e−1 − · · ·

Example:-2Taylor’s expansion of sin(x) about x◦ = 1

51

sin(x) = sin(1) + (x − 1) cos(1) − (x − 1)2

2!sin(1) − (x − 1)3

3!cos(1)

+(x − 1)4

4!sin(1) + · · ·

Example:-3If f(x) = lnx = loge x about x◦ = 1 is

f(x) = ln x, f(x◦) = 0, f ′(x◦) = 1, f ′′(x◦) = −1, · · ·

Then its Taylor expansion will be

ln x = (x − 1) − (x − 1)2

2+

(x − 1)3

3− (x − 1)4

4+ · · ·

+(−1)n−1(x − 1)n

n+

(−1)n(x − 1)n+1

(n + 1)ξ−(n+1) (3.11)

for some ξ ε (1, x) where 0 ≤ x ≤ 2.Example:-4If f(x) = ex. Then its Maclaurin series expansion will be

ex = e0 + (x − 0)e0 +(x − 0)2

2!e0 +

(x − 0)3

3!e0 +

(x − 0)4

4!e0 + · · ·

= 1 + x +x2

2!+

x3

3!+

x4

4!+ · · · x

n

n!+

xn+1

(n + 1)!eξ for some ξ ε (0, x)

about x◦ = 0Exapmle:-5Maclaurin series of sin(x), cos(x) and ln(x + 1) are, respectively,

sin(x) = x − x3

3!+

x5

5!− x7

7!+ · · ·

cos(x) = 1 − x2

2!+

x4

4!− x6

6!+ · · ·

ln(x + 1) = x − x2

2+

x3

3− x4

4+ · · ·

♠ Theorem:- Cramer’s ruleA practical method for solving 2 × 2 systems of linear equations is Cramer’s

rule, which is mentioned here for the sake of completeness. Although Cramer’s rule

52

can be used to solve 3 × 3 systems, it is intractable when N > 3. Consider the twolinear equations

ax1 + bx2 = e,

cx1 + dx2 = f,

with the condition that ad− bc 6= 0. We can solve for the variable x1 by eliminatingthe variable x2. This is accomplished by multiplying the first equation by d and thesecond equation by b, and subtracting:

adx1 + bdx2 = ed,

−bcx1 − bdx2 = −bf,

adx1 − bcx1 = ed − bf,

Hence (ad − bc)x1 = ed − bf and we can solve for x1 and obtain

x1 =ed − bf

ad − bcsimilarly, x2 =

af − ec

ad − bc. (3.12)

The quotients in (??) can be expressed using determinants:

x1 =

∣∣∣∣∣e bf d

∣∣∣∣∣∣∣∣∣∣

a bc d

∣∣∣∣∣

and x2 =

∣∣∣∣∣a ec f

∣∣∣∣∣∣∣∣∣∣

a bc d

∣∣∣∣∣

. (3.13)

53