fyc18

FIRST YEAR CALCULUS

W W L CHEN

c W W L Chen, 1987, 2008.This chapter originates from material used by the author at Imperial College, University of London, between 1981 and 1990.

It is available free to all individuals, on the understanding that it is not to be used for financial gain,

and may be downloaded and/or photocopied, with or without permission from the author.

However, this document may not be kept on any information storage and retrieval system without permission

from the author, unless such system is not accessible to any individuals other than its owners.

Chapter 18

INTERPOLATION AND APPROXIMATION

18.1. Exact Fitting

Example 18.1.1. We wish to find a polynomial through the points (0, 6), (1, 2) and (5, 6). To dothis, consider a polynomial p(x) = ax2 + bx+ c, where we shall determine suitable values for the threecoefficients. Since the polynomial p(x) passes through (0, 6), (1, 2) and (5, 6), we must have p(0) = 6,p(1) = 2 and p(5) = 6. It follows that we must have

6 = c,2 = a+ b+ c,6 = 25a+ 5b+ c.

This is a system of 3 linear equations in 3 unknowns. Solving this system, we get a = 1, b = 5 andc = 6. Hence p(x) = x2 5x+ 6.

Example 18.1.2. We wish to find a polynomial through the points (1, 5), (0, 1), (1,1) and (3, 49).To do this, consider a polynomial p(x) = ax3 + bx2 + cx + d, where we shall determine suitable valuesfor the four coefficients. Since the polynomial p(x) passes through (1, 5), (0, 1), (1,1) and (3, 49), wemust have p(1) = 5, p(0) = 1, p(1) = 1 and p(3) = 49. It follows that we must have

5 = a+ b c+ d,1 = d,1 = a+ b+ c+ d,49 = 27a+ 9b+ 3c+ d.

This is a system of 4 linear equations in 4 unknowns. Solving this system, we get a = 2, b = 1, c = 5and d = 1. Hence p(x) = 2x3 + x2 5x+ 1.

Chapter 18 : Interpolation and Approximation page 1 of 7

First Year Calculus c W W L Chen, 1987, 2008

The two examples above illustrate a very crude technique. When we attempt to fit a polynomialthrough k points, we use a polynomial of degree (k 1). We then have to determine the k coefficientsof this polynomial. This amounts to solving a system of k linear equations in the k unknowns. Clearlyit is rather tedious, particularly when k is large.

Let us therefore use a different approach on the same problems.

Example 18.1.3. As in Example 18.1.1, let us find a polynomial through the points (0, 6), (1, 2) and(5, 6). Try

p(x) = a(x 1)(x 5) + b(x 0)(x 5) + c(x 0)(x 1).Since p(0) = 6, we must have 6 = a(0 1)(0 5), so that a = 6/5. Since p(1) = 2, we must have2 = b(10)(15), so that b = 1/2. Since p(5) = 6, we must have 6 = c(50)(51), so that c = 3/10.Hence

p(x) =6(x 1)(x 5)

5 (x 0)(x 5)

2+

3(x 0)(x 1)10

= x2 5x+ 6.

Example 18.1.4. As in Example 18.1.2, let us find a polynomial through the points (1, 5), (0, 1),(1,1) and (3, 49). Try

p(x) = a(x 0)(x 1)(x 3) + b(x+ 1)(x 1)(x 3) + c(x+ 1)(x 0)(x 3) + d(x+ 1)(x 0)(x 1).

Since p(1) = 5, we must have 5 = a(10)(11)(13), so that a = 5/8. Since p(0) = 1, we musthave 1 = b(0+1)(01)(03), so that b = 1/3. Since p(1) = 1, we must have 1 = c(1+1)(10)(13),so that c = 1/4. Since p(3) = 49, we must have 49 = d(3 + 1)(3 0)(3 1), so that d = 49/24. Hence

p(x) = 5x(x 1)(x 3)8

+(x+ 1)(x 1)(x 3)

3+x(x+ 1)(x 3)

4+

49x(x+ 1)(x 1)24

= 2x3 + x2 5x+ 1.

Let us look at one more example. However, we shall be a little more systematic.

Example 18.1.5. We wish to find a polynomial through the points (1,3), (3, 3) and (4, 9). Try

p(x) = a(x 3)(x 4) + b(x 1)(x 4) + c(x 1)(x 3).

Substituting x = 1, x = 3, x = 4, we obtain respectively

a =p(1)

(1 3)(1 4) , b =p(3)

(3 1)(3 4) , c =p(4)

(4 1)(4 3) .

Hence

p(x) = p(1)(x 3)(x 4)(1 3)(1 4) + p(3)

(x 1)(x 4)(3 1)(3 4) + p(4)

(x 1)(x 3)(4 1)(4 3) .

Since p(1) = 3, p(3) = 3 and p(4) = 9, a little calculation gives p(x) = x2 x 3.

Consider now the general situation. Suppose that we wish to find a polynomial through the points(x1, y1), . . . , (xk, yk). Then it is not too difficult to see that the polynomial

p(x) =k

i=1

yi

kj=1j 6=i

(x xjxi xj

)(1)



satisfies the requirements. To see that, note that for every i = 1, . . . , k, we have

kj=1j 6=i

(x xjxi xj

)={

1 if x = xi,0 if x = x1, . . . , xi1, xi+1, . . . , xk.

The polynomial (1) is called the Lagrange interpolation polynomial.

18.2. Approximate Fitting

Fitting the points exactly is unsatisfactory from the numerical point of view, particularly so when thenumber of points is large. We therefore sometimes attempt to fit all points closely but not exactly. Afterall, experimental data are subject to errors anyway!

Consider a given set of n points (x1, y1), . . . , (xn, yn). We now attempt to fit these points with apolynomial p(x) = ak1xk1 + ak2xk2 + . . . + a0. Recall that when k n, this can always be done;for example, simply take p(x) to be the Lagrange interpolation polynomial. However, if k < n, an exactfit may not be possible; for example, it is not possible to fit a straight line (k = 2) to go through threenon-collinear points (n = 3) exactly.

We therefore consider the errors

i = |p(xi) yi|, where i = 1, . . . , n.

The problem now is to choose a0, . . . , ak1 in such a way in order to make the errors small. There aremany ways to make errors small, and the following are examples:

(A) Choose a0, . . . , ak1 to minimizeni=1

i.

(B) Choose a0, . . . , ak1 to minimize max1in

i minimax approximation.

(C) Choose a0, . . . , ak1 to minimizeni=1

2i least squares approximation.

Remark. It is generally considered that (A) is the best criterion but most awkward, and that (C) is theleast satisfactory criterion but easiest to use.

An analogous problem to that discussed in the previous section is the question of approximating afunction f(x) by a polynomial p(x) = ak1xk1 + ak2xk2 + . . .+ a0 in an interval a x b. Here weconsider the errors

(x) = |p(x) f(x)|, where a x b.

The problem now is to choose a0, . . . , ak1 in such a way in order to make the errors small. There aremany ways to make errors small, and the following are examples:

(A) Choose a0, . . . , ak1 to minimize ba

(x) dx.

(B) Choose a0, . . . , ak1 to minimize maxaxb

(x) minimax approximation.

(C) Choose a0, . . . , ak1 to minimize ba

2(x) dx least squares approximation.

Remark. As before, it is generally considered that (A) is the best criterion but most awkward, and that(C) is the least satisfactory criterion but easiest to use.



18.3. Minimax Approximation

We shall illustrate the technique by two of the simplest examples.

Example 18.3.1. Consider the points (1,3), (3, 3) and (4, 9). It was shown in Example 18.1.5 that wecan fit the polynomial x2 x 3 precisely. Suppose now that we wish to find a minimax approximationby a polynomial of degree 1 (linear minimax approximation). Suppose that p(x) = ax + b. We thenconsider the errors

1 = |p(x1) y1| = |a+ b+ 3|,2 = |p(x2) y2| = |3a+ b 3|,3 = |p(x3) y3| = |4a+ b 9|,

and minimize

max{|a+ b+ 3|, |3a+ b 3|, |4a+ b 9|}.

If we take a = 4 and b = 8, then max{|a+b+3|, |3a+b3|, |4a+b9|} = 1. Hence p(x) = 4x8 is a linearapproximation with maximum error 1. It can be shown that this is the best minimax approximation bya polynomial of degree 1, but demonstrating this point is not so straightforward.

Example 18.3.2. Consider the function f(x) = x2 in the interval 0 x 2. Suppose that we wish tofind a minimax approximation by a polynomial of degree 1 (linear minimax approximation). Supposethat p(x) = ax+ b. We then consider the errors

(x) = |ax+ b x2|, where 0 x 2.

Consider first of all the function h(x) = ax+ b x2. Then h(x) has a maximum value when dh/dx = 0.This occurs when x = a/2, and this is in the interval 0 x 2 provided that 0 a 4. Note thath(a/2) = a2/4 + b. Also h(0) = b and h(2) = 2a+ b 4. We now choose a and b such that

h(0) = h(2) < 0 < h(a

2

)= h(0)

(the reader should draw a picture of h(x) in the interval 0 x 2 to illustrate these special require-ments). Then we must have

b = 2a+ b 4 < 0 < a2

4+ b = b,

so that a = 2 and b = 1/2. Hence the linear polynomial p(x) = 2x 1/2 gives

max0x2

(x) =12.

It should by now be clear that minimax approximations are rather awkward to use, even in the simplestcases.

18.4. Least Squares Approximation

We shall illustrate the technique by two simple examples.



Example 18.4.1. Consider the points (1, 1), (2, 3), (3, 4), (4, 3), (5, 4) and (6, 2). Suppose that we wishto find a least squares approximation by a polynomial of degree 1 (linear least squares approximation).Suppose that p(x) = ax+ b. We then consider the errors

i = |p(xi) yi| = |axi + b yi|, where i = 1, 2, 3, 4, 5, 6,

and choose a and b so as to minimize

S(a, b) =6

i=1

2i =6

i=1

(axi + b yi)2.

Let us now think of S(a, b) as a function of the two variables a and b. We then must have

0 =S

a= 2

6i=1

xi(axi + b yi),

0 =S

b= 2

6i=1

(axi + b yi),

so that (6

i=1

x2i

)a+

(6

i=1

xi

)b =

6i=1

xiyi,(6

i=1

xi

)a+

(6

i=1

1

)b =

6i=1

yi.

Substituting for (xi, yi) for i = 1, 2, 3, 4, 5, 6, we have

91a+ 21b = 63,21a+ 6b = 17,

so that a = 1/5 and b = 32/15. Hence

p(x) =15x+

3215

is the best linear least squares approximation.

Example 18.4.2. As in Example 18.3.2, consider the function f(x) = x2 in the interval 0 x 2.Suppose that we wish to find a least squares approximation by a polynomial of degree 1 (linear leastsquares approximation). Suppose that p(x) = ax+ b. We then consider the errors

(x) = |ax+ b x2|, where 0 x 2,

and choose a and b so as to minimize

T (a, b) = 2

0

2(x) dx = 2

0

(ax+ b x2)2 dx.

Let us now think of T (a, b) as a function of the two variables a and b. We then must have

0 =T

a= 2

20

x(ax+ b x2) dx = 2(

83a+ 2b 4

),

0 =T

b= 2

20

(ax+ b x2) dx = 2(

2a+ 2b 83

),



so that a = 2 and b = 2/3. Hence

p(x) = 2x 23

is the best linear least squares approximation.

In general, if we try to fit a polynomial p(x) = akxk + . . .+ a0 to n points (x1, y1), . . . , (xn, yn), thenwe choose a0, . . . , ak to minimize

S(a0, . . . , ak) =ni=1

(p(xi) yi)2.

The requirement that

S

aj= 0 for every j = 0, . . . , k

gives rise to a system of (k + 1) linear equations in the (k + 1) unknowns a0, . . . , ak.

If we try to fit a polynomial p(x) = akxk + . . .+ a0 to a given function f(x) in an interval a x b,then we choose a0, . . . , ak to minimize

T (a0, . . . , ak) = ba

(p(x) f(x))2 dx.

The requirement that

T

aj= 0 for every j = 0, . . . , k

gives rise to a system of (k + 1) linear equations in the (k + 1) unknowns a0, . . . , ak.

Hence the determination of the best least squares approximations amounts to nothing more thansolving a system of linear equations. Squaring the errors removes any ambiguity on the sign of theerrors.



Problems for Chapter 18

1. Find a polynomial to pass through the points (2, 99), (1, 11), (0, 1), (1, 3) and (2, 47).

2. Find the best linear least squares approximation to the points (2, 99), (1, 11), (0, 1), (1, 3) and(2, 47).

3. Find the best linear least squares approximation to the function ex in the interval [0, 1].


Documents

fyc18