Lecture 17 - Approximation Methods CVEN 302 July 17, 2002

Lecture 17 - Approximation Lecture 17 - Approximation MethodsMethods

CVEN 302

July 17, 2002

Lecture’s GoalsLecture’s Goals

• Discrete Least Square Approximation(cont.)

– Nonlinear

• Continuous Least Square– Orthogonal Polynomials– Gram Schmidt -Legendre Polynomial– Tchebyshev Polynomial– Fourier Series

Nonlinear Least Squared Nonlinear Least Squared Approximation MethodApproximation Method

How would you handle a problem, which is modeled as:

ax

a

b

or

b

ey

xy

Nonlinear Least Squared Nonlinear Least Squared Approximation MethodApproximation Method

Take the natural log of the equations

xy

xyxy

ab

ln ablnlnb a

xy

xyey

ab

ablnlnb ax

and

Least Square Fit Least Square Fit ApproximationsApproximations

Suppose we want to fit the data set.

Nonlinear Data Fit

0

2

4

6

8

10

12

0 2 4 6 8 10

X Values

Y V

alu

es

Data

x Data0.2 9.910.8 8.181.6 6.332.8 4.314.2 2.758 0.82

Linear Least Square Linear Least Square ApproximationsApproximations

x Data x2 xy ln(y) x ln(y) N0.2 9.91 0.04 1.982 2.293544 0.458709 10.8 8.18 0.64 6.544 2.101692 1.681354 11.6 6.33 2.56 10.128 1.8453 2.95248 12.8 4.31 7.84 12.068 1.460938 4.090626 14.2 2.75 17.64 11.55 1.011601 4.248724 18 0.82 64 6.56 -0.19845 -1.58761 1

17.6 32.3 92.72 48.832 8.514625 11.84429 6

baxy Use:

Least Square Fit Least Square Fit ApproximationsApproximations

We would like to find the best straight line to fit the data?

Nonlinear Data Fit

0

2

4

6

8

10

12

0 2 4 6 8 10

X Values

Y V

alu

es

Linear

Data

y = -1.11733x + 8.66608

Nonlinear Least Square Nonlinear Least Square ApproximationsApproximations

x Data x2 xy ln(y) x ln(y) N0.2 9.91 0.04 1.982 2.293544 0.458709 10.8 8.18 0.64 6.544 2.101692 1.681354 11.6 6.33 2.56 10.128 1.8453 2.95248 12.8 4.31 7.84 12.068 1.460938 4.090626 14.2 2.75 17.64 11.55 1.011601 4.248724 18 0.82 64 6.56 -0.19845 -1.58761 1

17.6 32.3 92.72 48.832 8.514625 11.84429 6

axbeyUse:

Nonlinear Least Square Nonlinear Least Square ExampleExample

356492.2

6.1772.926

84429.116.17514625.872.92b

31956.0

6.1772.926

514625.86.1784429.116a

2

2

The equation is:

Nonlinear Data Fit

0

2

4

6

8

10

12

0 2 4 6 8 10

X Values

Y V

alu

es

Natural Log

Data

xy

xy 0.31965 e 55386.10

31965.0356492.2


abxy Use:

x Data x2 xy lnx (ln x2) ln(y) x ln(y) lnx lny N0.2 9.91 0.04 1.982 -1.60944 2.59029 2.293544 0.458709 -3.69132 10.8 8.18 0.64 6.544 -0.22314 0.049793 2.101692 1.681354 -0.46898 11.6 6.33 2.56 10.128 0.470004 0.220903 1.8453 2.95248 0.867298 12.8 4.31 7.84 12.068 1.029619 1.060116 1.460938 4.090626 1.50421 14.2 2.75 17.64 11.55 1.435085 2.059468 1.011601 4.248724 1.451733 18 0.82 64 6.56 2.079442 4.324077 -0.19845 -1.58761 -0.41267 1

17.6 32.3 92.72 48.832 3.181568 10.30465 8.514625 11.84429 -0.74972 6


The exponential approximation fits the data.

The power approximation does not fit the data.

Nonlinear Data Fit

0

2

4

6

8

10

12

0 2 4 6 8 10

X Values

Y V

alu

es

Linear

Natural Log

Pow er

Data

Continuous Least Square Continuous Least Square FunctionsFunctions

Instead of modeling a known complex function over a region, we would like to model the values with a simple polynomial. This technique uses a least squares over a continuous region.

The coefficients of the polynomial can be determined using same technique that was used in discrete method.


The technique minimizes the error of the function uses an integral.

where

dxxsxfE b

a

2

2210 xaxaaxf


Take the derivative of the error with respect to the coefficients and set it equal to zero.

And compute the components of the coefficient matrix. The right hand side of the matrix will be the function we are modeling times a x value.

0 2

i

b

ai

dx

da

xdfxsxf

da

dE

Continuous Least Square Continuous Least Square Function ExampleFunction Example

Given the following function:

Model the function with a quadratic polynomial.

2xs x xe

2210 xaxaaxf

from 0 to 1


The integral for the error is:

122 2

0 1 2

0

12 2

0 1 20 0

1 1 1 12 2

0 1 2

0 0 0 0

2 1 0

x

x

x

E x a a x a x xe dx

dE xa a x a x xe dx

da

a dx a x dx a x dx xe dx


The integral for the components are:1

0 0

0

11

1

0

12 2

2

0

2

3

a dx a

aa x dx

aa x dx


The coefficient matrix becomes:

1

0

23

1

0

22

1

0

2

2

1

0

5

1

4

1

3

14

1

3

1

2

13

1

2

11

dxex

dxex

dxxe

a

a

a

x

x

x


The right-hand side of the equation becomes:

2986.1

5973.1

0973.2

38

1

14

1

14

1

2

2

2

1

0

23

1

0

22

1

0

2

e

e

e

dxex

dxex

dxxe

x

x

x


Continuous Least Squared Fit

0

1

2

3

4

5

6

7

8

0 0.5 1 1.5

X Value

Y V

alu

ePolynomial

Exact

The function becomes :

f(x) = 0.3328 -2.5806x

+ 9.1642x2

Continuous Least Square Continuous Least Square FunctionFunction

There are other forms of equations, which can be used to represent continuous functions. Examples of these functions are

• Legrendre Polynomials

• Tchebyshev Polynomials

• Cosines and sines.

Legendre PolynomialLegendre Polynomial

The Legendre polynomials are a set of orthogonal functions, which can be used to represent a function as components of a function.

xPaxPaxPaxf nn1100


These function are orthogonal over a range [ -1, 1 ]. This range can be scaled to fit the function. The orthogonal functions are defined as:

j i if 0

j i if #

1

1

ji dxxPxP


The Legendre functions are:

33

n nn2

n n n

3P x x x

5

1 dP x 1-x

2 n! dx

0

1

22

P x 1

P x x

1P x x

3


How would you work with a least square fit of a function.

dxxsxPaxPaxPaE21

1

221100

Legendre PolynomialLegendre PolynomialHow would you work with a least square fit of a function.

02 0

1

1

2211000

dxxPxsxPaxPaxPada

dE

1

1

0

1

1

022

1

1

011

1

1

000

dxxPxsdxxPxPa

dxxPxPadxxPxPa

Legendre PolynomialLegendre PolynomialThe coefficient a0 is determined by the orthogonality of the Legendre polynomials:

1

1

00

1

1

0

0

1

1

0

1

1

000

dxxPxP

dxxPxs

a

dxxPxsdxxPxPa

Legendre Polynomial ExampleLegendre Polynomial Example

Given a simple polynomial:

22xxs

We want to throw a loop, let’s model it from 0 to 4 with f(x):

xPaxPaxPaxf 221100

Legendre Polynomial ExampleLegendre Polynomial Example

The first step will be to scale the function:

bmux We know that at the ends are 0 and 4 for x and -1 to 1 for u so

8168

22*22

2

uuus

uus22 ux

Legendre Polynomial ExampleLegendre Polynomial ExampleThe coefficients are

6667.102

3333.21

8168

1

1

1

1

2

1

1

00

1

1

0

0

du

duuu

duuPuP

duuPus

a

8 and 16 21 aa

Legendre Polynomial ExampleLegendre Polynomial ExampleThe Legendre functions must be adjusted to handle the scaling:

Legendre Polynomial Example

0

5

10

15

20

25

30

35

0 1 2 3 4 5

X Values

Y V

alu

esLegendrePolynomial

Exact Value

12

1 xu

15.08

15.016

15.0667.10

2

1

0

xP

xP

xPxf

Tchebyshev PolynomialTchebyshev Polynomial

The Tchebyshev polynomials are another set of orthogonal functions, which can be used to represent a function as components of a function.

xTaxTaxTaxf nn1100

Tchebyshev PolynomialTchebyshev PolynomialThese function are orthogonal over a range [ -1, 1 ]. This range can be scaled to fit the function. The orthogonal functions are defined as:

1i j

21

0 if i j

if i j 01 /2 if i j 0

T x T xdx

x


The Tchebyschev functions are:

xTxT2xT

3xx4xT

1x2xT

xxT

1xT

1-nn1n

33

22

1

0

x


How would you work with a least square fit of a function.

dxxsxTaxTaxTaE21

1

221100

Tchebyshev PolynomialTchebyshev PolynomialHow would you work with a least square fit of a function.

02 0

1

1

2211000

dxxTxsxTaxTaxTada

dE

1

1

0

1

1

022

1

1

011

1

1

000

dxxTxsdxxTxTa

dxxTxTadxxTxTa

Rearrange


The coefficients are determined as:

1

1

00

1

1

0

0

1

1

0

1

1

000

dxxTxT

dxxTxs

a

dxxTxsdxxTxTa

Continuous FunctionsContinuous Functions

Other forms of orthogonal functions are sines and cosines, which are used in Fourier approximation. The advantages for the sines and cosines are that they can model large time scales.

You will need to clip the ends of the series so that it will have zeros at the ends.

Fourier Series Fourier Series

The Fourier series takes advantage of the orthogonality of sines and cosines.

i i jregioni i jregion

i j

iregioni

i iregion

sin if sin sin

0 if

sin =

sin sin

F x x dxC x x dx

F x x dxC

x x dx

Fourier Series Fourier Series

The time series or spatial series is generally clipped and the resulting coefficients are determined using Least Squared technique.

0

jj sinj

xCxF

Fast Fourier Transforms Fast Fourier Transforms

The Fast Fourier Transforms (FFT) are discrete form of the equations. It takes advantage of the power of 2 to find the coefficients in the analysis of data.

So when you hear FFT, it is technique developed by Black and Tukey in the early 60’s. To take advantage of the computer. It is a method to analysis the series.

Other Continuous Functions Other Continuous Functions

Wavelets are another form of orthogonal functions, which maintain the amplitude and phase information of the series.

The techniques are used in data compression, earthquake modeling, wave modeling, and other forms of environmental loading. Etc.

SummarySummary

• Developed a technique using Least Squared applications to nonlinear functions.

ax

a

b

or

b

ey

xy

SummarySummary

• Modeled the equations with continuous functions to describe the functions.

– Polynomials

– Legrendre Polynomials

– Tchebyshev Polynomials

– Cosines and Sines

HomeworkHomework

• Check the homework webpage

Documents

Lecture 17 - Approximation Methods CVEN 302 July 17, 2002