COSC 3361 Numerical Analysis Igabriel/courses/cosc3361_f05/NA_10...COSC 3361 – Numerical Analysis I Edgar Gabriel Data approximation • Given some points find a function that “captures

COSC 3361 – Numerical Analysis IEdgar Gabriel

COSC 3361Numerical Analysis I

Approximating Functions (I)Lagrange Interpolations

Edgar GabrielFall 2005


Data approximation

• Given some points find a function that “captures trend” of the data

• 1st possibility: ensure that “goes through the data”, i.e.

( ) ( ),...,,, 2211 yxyx

)(xf

)(

)(

22

11

xfy

xfy

==

…

�

�


Data approximation (II)

• 2nd possibility: define a metric and make sure, that the interpolation function is minimal according to this metric, e.g.

�

�

�=

−N

iii yyxf

0

)~(min(:)(


Standard procedure for simple functions

• e.g. given two points (1,3) and (-4,2), calculate the polynomial of degree 1 (straight line) which goes through both points

Thus

Leading to andTherefore

baxxf +=)(

ba +=3ba +−= 42

(1)(2)

ba −=� 3bb ++=� 4122

b510 =−

2−=b 5=a

25)( −= xxf


A four point interpolation (I)

• Find the cubic polynomialthat interpolates the data (-2,10),(-1,4),(1,6) and (2,3)

• Each point of the interpolation leads to linear equation

34

23213 )( xaxaxaaxp +++=

�=− 10)2(3p�=− 4)1(3p 44321 =−+− aaaa

10842 4321 =−+− aaaa

�= 6)1(3p 64321 =+++ aaaa�= 3)2(3p 3842 4321 =+++ aaaa


A four point interpolation (II)

or in matrix/vector terms

• Solving such a system of linear equations in Matlab

��

�

�

��

�

�

=

��

�

�

��

�

�

��

�

�

��

�

�

−−−−

364

10

8421111111118421

4

3

2

1

a

a

a

a

y=[10;4;6;3];V=[1 -2 4 -8; 1 -1 1 -1; 1 1 1 1; 1 2 4 8];a=V\y;


General n-points case (I)

• Given n points determine the coefficients of the polynomial of degree (n-1) [or less u.c.c]

such that

123211 ...)( −

− ++++= nnn xaxaxaaxp

( ) ( ) ( )nn yxyxyx ,,...,,, 2211

naaa ,...,, 21

iin yxp =− )(1


General n-points case (II)

• Using the matrix/vector form

• The matrix is called the Vandermode matrix– Usually non-singular– ill conditioned (= numerically instable)

��

�

�

��

�

�

=

��

�

�

��

�

�

��

�

�

��

�

�

−

−

−

−

nnnnnn

n

n

n

y

y

y

y

a

a

a

a

xxx

xxx

xxx

xxx

��

�

��

�

�

�

3

2

1

3

2

1

12

13

233

12

222

11

211

1

111


Setting up the Vandermode matrix in Matlab (I)

V=zeros(n,n);for i=1:n

% Set up row ifor j=1:n

V(i,j)= x(i)^(j-1);end

enda=V\y;

V=ones(n,n);for j=2:n

% Set up column jfor i=1:n

V(i,j)= x(i)*V(i,j-1);end

enda=V\y;


Setting up the Vandermode matrix in Matlab (II)

V=ones(n,n);for j=2:n

% Set up column jV(:,j)= x.*V(:,j-1);

enda=V\y;


Newton form of interpolation polynomials

• The polynomial

could also be expressed as

or

nnn xaxaxaaxp 1

2321 ...)( +++++=

...))(()()( 212110 +−−+−+= xxxxcxxccxpn

� ∏=

−

=

−=n

i

i

jjin xxcxp

0

1

0

)()(


Newton representation

• Note that the Newton representation is constructed by a sequence of interpolation polynomials, e.g.

or

00 )( cxp =)()( 0101 xxccxp −+=

))(()()( 1020102 xxxxcxxccxp −−+−+=

00 )( cxp =)()()( 0101 xxcxpxp −+=

))(()()( 10212 xxxxcxpxp −−+=(12:2)

(12:1)

∏−

=− −+=

1

01 )()()(

k

jjkkk xxcxpxp

…


1st algorithm for Newton’s representation

• Based on the representation (12:2), one can design a first algorithm for determining a function using Newton’s representation, which interpolates the given points

• Problem: how to determine efficiently

∏−

=

−

−

−= 1

0

1

)(

)(k

jjk

kkkk

xx

xpyc

00 yc =

)(1 kk xp −

nk :1=


Side note: nested multiplication

• A term having the form

can also be rewritten as

leading to the following algorithm for calculating u

...21031020

1

0010 ++++==� ∏

=

−

=

dddcddcdccdcun

i

i

jji

0012211 )...)))(( cdccdcdcu kkkkk ++++= −−−−

kk cu =111 −−− += kkkk cduu

2212 −−−− += kkkk cduu

0010 cduu +=

…


Side note: nested multiplication (II)

• Thus can be calculated by

– Horner’s algorithm!!!

� ∏−

=

−

=− −=

1

0

1

01 )()(

k

i

i

jjkikk xxcxp

u = c(k)for i= k-1:0 step -1

u=(x(k)-x(i))u +c(i); end


1st algorithm for Newton’s representation (II)

• Leading to the following algorithmc(0)= y(0);for k=1:nd = x(k) – x(k-1);u = c(k-1)for i= k-2:0 step -1

u = (x(k)-x(i)) * u +c(i);d = (x(k)-x(i)) * d;

endc(k)= (y(k)-u)/d;

end

∏−

=

−

−

−= 1

1

1

)(

)(k

jjk

kkkk

xx

xpyc


2nd algorithm for Newton’s representation: Divided differences

• Based on representation (12:1), the interpolation function can also be written as

with

Using the given points one obtains for each point a linear equation, leading to an matrix

�=

=n

iii xqcxp

0

)()(

1)(0 =xq)()( 01 xxxq −=

))(()( 102 xxxxxq −−=

( ) ( ) ( )nn yxyxyx ,,...,,, 2211

( ) ( )11 +×+ nn


Divided differences (II)

• e.g. for three points

leading to��

�

�

��

�

�

=��

�

�

��

�

�

��

�

�

��

�

�

−−−−

)()()(

))(()(10)(1001

2

1

0

2

1

0

120202

01

xf

xf

xf

c

c

c

xxxxxx

xx

)( 00 xfc =

01

011

)()(xx

xfxfc

−−=

12

01

01

02

02

2

)()()()(

xxxx

xfxfxx

xfxf

c−

−−−

−−

=


Divided differences (III)

• The coefficients are called divided differences.• The notion

stresses the dependence of on f and onThus

Furthermore, is the coefficient of the term in the interpolation function obtained by the points

],...,[ 0 kk xxfc =

kc kxx ,...,0

)(][ 000 xfxfc ==

01

01101

)()(],[

xxxfxf

xxfc−−==

],...,[ 0 kk xxfc =kx

kxx ,...,0


Theorem on divided differences

• Divided differences satisfy the equation

• Proof:– Consider, that by definition

– Furthermore, if denotes the polynomial of degree (n-1) that interpolates f at the nodes , and the

polynomial of degree (n-1) that interpolates f at the nodes , then

0

1102110

],...,,[],...,,[],...,,[

xxxxxfxxxf

xxxfn

nnn −

−= −

...))(](,,[)](,[][)( 102100100 +−−+−+= xxxxxxxfxxxxfxfxpn

)(1 xpn−

10 ,..., −nxx)(* xp

nxx ,...,1

))()(()()( 1*

0

* xpxpxxxx

xpxp nn

nn −−

−−+=

(20:1)

(20:2)


Theorem on divided differences (II)

– Polynomials on both sides are of degree at most n– Since both sides have identical values for n+1 points, they

have to be identical• For the points the parenthesis becomes 0• For the fraction becomes zero and thus

• For you get for the right side

))()(()()( 1*

0

* xpxpxxxx

xpxp nn

nn −−

−−+=

11,..., −nxx

nxx =

nnnn yxpxp == )()( *

0xx =

000*

0

00

* ))(()( yyxpxxxx

xpn

n =−−−−


Theorem on divided differences (III)

– Since

and

you can compare the coefficients of equation (20:2) forand you get equation (20:1)

))...(](,...,[...][)( 201001 −−− −−++= nnn xxxxxxfxfxp

))...(](,...,[...][)( 1111*

−−−++= nn xxxxxxfxfxpnx


Theorem on divided differences (IV)

• Since variables can easily be renamed, the term

can be generalized to 0

1102110

],...,,[],...,,[],...,,[

xxxxxfxxxf

xxxfn

nnn −

−= −

iji

jiijiijii xx

xxfxxfxxf

−−

=+

−++++

],...,[],...,[],...,[ 11

(23:1)


Calculating divided differences

• Set up a table of the form

][],[][

],,[],[][],,,[],,[],[][

33

3222

3212111

32102101000

xfx

xxfxfx

xxxfxxfxfx

xxxxfxxxfxxfxfx

Inputvalues

Calculated values using (23:1)


Example 1

• Calculate the divided difference table for the following function values

46225

203

4531

407

83213

−

−

4231)(6513

−xf

x

� the according Newton Polynomial is:

)5)(1)(3(407

)1)(3(83

)3(21)( −−−+−−−−+= xxxxxxxp


More generally

0

1,10,11

232221202

1,1131211101

,01,0030201000

nn

nnn

n

nn

cx

ccx

ccccx

cccccx

ccccccx

−−−

−

−

��

��

�

�

�

with being the input values of the function f at the pointsi.e.and being the coefficients of the interpolation polynomial

0ic ixii yc =0

ic0


Overall algorithm (I)

• The values c(i,0) are the input values y(i)

• Uses lots of memory, since all intermediate divided differences are stored as well

for j = 1:nfor i= 0:n-j

c(i,j)=(c(i+1,j-1)-c(i,j-1))/x(i+j)-x(i); end

end


Overall algorithm(II)

• c(0) contains already the final parameter• c(1)-c(n) contain the intermediate values, c(0) not touched• Next iteration: c(2)-c(n) contain the next column of

intermediate values, c(0) and c(1) are not touched• etc.

for i = 0:nc(i) = y(i)

end for j = 1:nfor i= n:j step -1

c(i)=(c(i)-c(i-1))/x(i)-x(i-j); end

end


Question

• Question: can I get two different polynomials whether I use the Newton representation or the Vandermodematrix?

• Answer: noTheorem: For distinct points there is a unique polynomial of degree at most n such that

Proof: Assume, there are two polynomials fulfilling condition (29:1). The polynomial is of degree n and has n zeros. For all we have however

and since there are n+1 points

( ) ( ) ( )nn yxyxyx ,,...,,, 1100

iin yxp =)(

nn qp ,

(29:1)

))(( xqp nn −( )ii yx ,

nn qp ≡0))(( =− inn xqp


Permutations on divided differences

• If is a permutation of then

• Remember, that the divided difference above is the coefficient at in the polynomial of degree ninterpolating the data. Since there is only one polynomial of degree n interpolating the same (n+1) points, the coefficient has to be identical.

),...,( 10 nzzz ),...,( 10 nxxx

],...,[],...,[ 1010 nn xxxfzzzf =

nx


Vandermode matrix vs. divided differences

• Vandermode matrix is strongly related to the way we are used to solve “smaller” problems– If you add a point and have to update the polynomial to

degree (n+1), the whole computation has to be repeated– Numerically difficult, since ill conditioned

• Newton’s representation – Is not as “straight forward”+ combined with the divided differences algorithm very fast

and robust+ easy to add a point, all calculated up to this point can

still be used kp


Simplification of Divided Differences

• If are equally spaced, then

with h being the spacing• For this case, the formula to calculate higher order

divided differences can be modified to

ixhixxi /)1(0 −+=

)1(],...,,[],...,,[

],...,,[ 1102110 −

−= −

khxxxfxxxf

xxxf nnn


Summary (I)

• A polynomial of degree (n-1) required to interpolate npoints

• There is exactly one polynomial of degree (n-1) which interpolates

• For the “regular” representation of a polynomial

– Set up the Vandermode matrix– Solve the system of linear equations– Numerically not recommended

( ) ( ) ( )nn yxyxyx ,,...,,, 2211

( ) ( ) ( )nn yxyxyx ,,...,,, 2211

nnn xaxaxaaxp 1

2321 ...)( +++++=


Summary (II)

• For Newton’s representation of polynomials

there are two possible algorithms to determine the coefficients– Using the recursive formulation

• Calculate the function values using Horner’s rule

for sums of products

� ∏=

−

=

−=n

i

i

jjin xxcxp

0

1

0

)()(

kc

∏−

=

−

−

−= 1

1

1

)(

)(k

jjk

kkkk

xx

xpyc

)(1 kk xp −


Summary (III)

– Using Divided Differences• Notion for expressing the dependence of a coefficient

on the function f and the points used to determine the interpolating polynomial

• Formula for calculating higher order divided differences

iji

jiijiijii xx

xxfxxfxxf

−−

=+

−++++

],...,[],...,[],...,[ 11

kc

],...,[ 0 kk xxfc =

Documents

COSC 3361 Numerical Analysis Igabriel/courses/cosc3361_f05/NA_10...COSC 3361 – Numerical Analysis I Edgar Gabriel Data approximation • Given some points find a function that “captures