Numerical DifferentiationNumerical Differentiation� Estimate the derivatives (slope, curvature, etc.)

of a function by using the function values at only a set of discrete points

� Ordinary differential equation (ODE)� Partial differential equation (PDE)� Partial differential equation (PDE)� Represent the function by Taylor polynomials

or Lagrange interpolation� Evaluate the derivatives of the interpolation

polynomial at selected (unevenly distributed) nodal points

0

( ) ( )Differentiation lim

( )

( ) ( ) Difference

( )

x

y f(x)

dy f x x f x

dx x x x

y f x x f x dy

x x x x dx

∆ →

=+∆ −=+∆ −

∆ +∆ −= ≈∆ +∆ −

Difference ( )

A smaller step (or h) results in a smaller error

x x x x dx

x

= ≈∆ +∆ −

∆

xi-3 xi-2 xi-1 xi xi+1 xi+2 xi+3

hx =∆

Evenly distributed points along the x-axis

Distance between two neighboring points is the same, i.e. h.

x1 x2 x3

Unevenly distributed points along the x-axis

Distance between two neighboring points is the same, i.e. h.

Lagrange InterpolationLagrange Interpolation� 1st-order Lagrange polynomial (Two(Two--Point Point

)))()()()()()( 1

01

00

10

111001 xf

xxxx

xfxxxx

xfxLxfLxf−−−−−−−−++++

−−−−−−−−====++++====

� Second-order Lagrange polynomial (Three(Three--Point )Point )

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx()x(f

21202

10

12101

20

02010

212

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−====

General ThreeGeneral Three--Point Point FormulaFormula

�Lagrange interpolation polynomial for unequally spaced data

)())((

)()()()()()()(

1i1ii

1i1iii1i1i

xfxxxx

xfxLxfxLxfxLxf

−−−−++++

++++++++−−−−−−−−

−−−−−−−−−−−−−−−−====

++++++++====

)())((

))((

)())((

))((

)())((

1ii1i1i1i

i1i

i1ii1ii

1i1i

1i1i1ii1i

xfxxxx

xxxx

xfxxxx

xxxx

xfxxxx

++++++++−−−−++++

−−−−

++++−−−−

++++−−−−

−−−−++++−−−−−−−−

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−++++

−−−−−−−−====

Lagrange InterpolationLagrange Interpolation

L 2(x)f(x2)L 0(x)f(x0)

L 1(x)f(x1)

x0 x1 x2

General ThreeGeneral Three--Point Point FormulaFormula

�Lagrange interpolation polynomial for unequally spaced data

))((

))(())((

)())((

))(()(

)()()()()()()(

1ii1ii

1i1ii

1i1ii1i

1ii1i

1i1iii1i1i

xxxx

xxxxxxxx

xfxxxx

xxxxxf

xfxLxfxLxfxLxf

−−−−−−−−−−−−−−−−

−−−−−−−−++++−−−−−−−−

−−−−−−−−====

++++++++====

++++−−−−

++++−−−−

++++−−−−−−−−

++++−−−−

++++++++−−−−−−−−

�First derivative

))(())((

)(i1i1i1i

i1i1i xxxx

xxxxxf

−−−−−−−−−−−−−−−−++++++++−−−−++++

−−−−++++

))(()(

))(()(

))(()()(

i1i1i1i

i1i1i

1ii1ii

1i1ii

1i1ii1i

1ii1i

xxxxxxx2

xf

xxxxxxx2

xfxxxx

xxx2xfxf

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−====′′′′

++++−−−−++++

−−−−++++

++++−−−−

++++−−−−

++++−−−−−−−−

++++−−−−

Lagrange polynomial (Two(Two--Point )Point )

)()()()()()( 10

01

11001 xfxx

xfxx

xfxLxfLxf−−−−−−−−++++

−−−−−−−−====++++==== )()()()()()( 1

010

1011001 xf

xxxf

xxxfxLxfLxf

−−−−++++

−−−−====++++====

Forward difference

Backward difference

Numerical Numerical DifferentiationDifferentiation

Backward difference

Centered difference

Forward differenceForward difference

x i−−−−1 x i x i+1x

h

Backward differenceBackward difference

x i−−−−1 x i x i+1x

h

First DerivativesFirst Derivatives)x(f ′′′′

y

� Forward difference

� Backward difference

i-2 i-1 i i+1 i+2

1

1

1

1

1

1

1

1

)()()(

)()()(

−

−

−

−

+

+

+

+

−−=

−−≅′

−−=

−−≅′

ii

ii

ii

ii

ii

ii

ii

ii

xx

yy

xx

xfxfxf

xx

yy

xx

xfxfxf

x

Given find numerical approximations to the derivative , using two points and the forward difference formula.

Given find numerical approximations to the derivative , using two points and the backward difference formula.

Truncation ErrorsTruncation Errors� Uniform grid spacing

++++′′′′′′′′′′′′−−−−′′′′′′′′++++′′′′−−−−====−−−−====

++++′′′′′′′′′′′′++++′′′′′′′′++++′′′′++++====++++====

−−−−

++++

L

L

)x(f!3

h)x(f

!2h

)x(fh)x(f)hx(f)x(f

)x(f!3

h)x(f

!2h

)x(fh)x(f)hx(f)x(f

i

3

i

2

iii1i

i

3

i

2

iii1i

′′′′′′′′′′′′−−−−−−−−====′′′′

′′′′′′′′++++−−−−====′′′′

′′′′′′′′−−−−−−−−====′′′′

−−−−++++

−−−−

++++

))()()(

)(:

)()()(

)(:

)()()(

)(:

23

21i1i

i

21ii

i

1i1i

i

O(h f6h

h2xfxf

xfcentral

O(h) f2h

hxfxf

xfbackward

O(h) f2h

hxfxf

xfforward

ξξξξ

ξξξξ

ξξξξ

Example: First DerivativesExample: First Derivatives� Use forward and backward difference approximations

to estimate the first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

� Forward Difference

2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====

� Backward Difference

====−−−−====−−−−====−−−−−−−−====′′′′====

====−−−−====−−−−====−−−−−−−−====′′′′====

%.,..

....

).().().(,.

%.,..

...

).()().(,.

526 1551250

925063632813050750

50f750f50f 250h

958 45150

925020501

50f1f50f 50h

t

t

εεεε

εεεε

====−−−−====−−−−====−−−−−−−−====′′′′====

====−−−−====−−−−====−−−−−−−−====′′′′====

%.,....

..).().(

).(,.

%.,..

...

)().().(,.

721 71402501035156319250

25050250f50f

50f 250h

739 55050

219250050

0f50f50f 50h

t

t

εεεε

εεεε

Lagrange polynomial (Three(Three--Point )Point )

)x(f)xx)(xx(

)xx)(xx()x(f 0

212 −−−−−−−−

−−−−−−−−====

)x(f)xx)(xx(

)xx)(xx(

)x(f)xx)(xx(

)xx)(xx(

)xx)(xx(

21202

10

12101

20

2010

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−++++

−−−−−−−−

First DerivativesFirst Derivatives)x(f ′′′′

Parabolic curve

� 3 -point Forward difference

� 3 -point Backward difference

i-2 i-1 i i+1 i+2

h

xfxfxf

xx

xfxfxfxf

h

xfxfxf

xx

xfxfxfxf

iii

ii

iii

iii

ii

iii

2

)()(4)(3)()(4)(3)(

2

)()(4)(3)()(4)(3)(

21

2

21

21

2

21

−−

−

−−

++

+

++

+−=−

+−≅′

−+−=−

−+−≅′

First DerivativesFirst Derivatives)x(f ′′′′

Parabolic curve

� 3 - point central difference i-2 i-1 i i+1 i+2

h

yy

h

xfxf

xx

yy

xx

xfxfxf

iiii

ii

ii

ii

ii

22

)()(

)()()(

1111

11

11

11

11

−+−+

−+

−+

−+

−+

−=−≅

−−=

−−≅′

Centered differenceCentered difference

x i−−−−1 x i x i+1x

2h

Example: First DerivativeExample: First Derivative� Use central difference approximation to estimate the

first derivative of

at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)

� Central Difference

2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====

%4.2,934.05.0

10351563.163632813.0

25.075.0)25.0(f)75.0(f

)5.0(f,25.0h

%6.9,0.11

2.12.001

)0(f)1(f)5.0(f,5.0h

====−−−−====−−−−====

−−−−−−−−====′′′′====

====−−−−====−−−−====−−−−−−−−====′′′′====

t

t

εεεε

εεεε

Example: First DerivativesExample: First Derivatives� Use forward and backward difference approximations

of O(h2) to estimate the first derivative of

at x = 0.5 with h = 0.25 (exact sol. = -0.9125)

� Forward Difference

2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====

� Backward Difference

%82.5,859375.05.0

)925.0(3)6363281.0(42.0

)25.0(2)5.0(f3)75.0(f4)1(f

)5.0(f

====−−−−====−−−−++++−−−−====

−−−−++++−−−−====′′′′

t

εεεε

%77.3,878125.05.0

2.1)035156.1(4)925.0(3

)25.0(2)0(f)25.0(f4)5.0(f3

)5.0(f

====−−−−====++++−−−−====

++++−−−−====′′′′

t εεεε

Given find numerical approximations to the derivative , using three points and the forward difference formula.

Given find numerical approximations to the derivative , using three points and the backward difference formula

Given find numerical approximations to the derivative , using three points and the central difference formula.

Error For First Error For First Derivatives Derivatives ((ThreeThree--Point )Point )

Second DerivativeSecond Derivative� First Derivative for unequally spaced data

))(()(

))(()(

))(()()(

i1i1i

1ii1ii

1i1ii

1i1ii1i

1ii1i

xxxxxxx2

xf

xxxxxxx2

xfxxxx

xxx2xfxf

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−++++

−−−−−−−−−−−−−−−−====′′′′

++++−−−−++++

−−−−++++

++++−−−−

++++−−−−

++++−−−−−−−−

++++−−−−

� Second Derivative for unequally spaced data

))(( i1i1i1i xxxx −−−−−−−− ++++−−−−++++

))(()(

))(()(

))(()()(

i1i1i1i1i

1ii1iii

1i1ii1i1i

xxxx2

xf

xxxx2

xfxxxx

2xfxf

−−−−−−−−++++

−−−−−−−−++++

−−−−−−−−====′′′′′′′′

++++−−−−++++++++

++++−−−−++++−−−−−−−−−−−−

SecondSecond--DerivativesDerivatives� Taylor-series expansion

� Uniform grid spacing

++++′′′′′′′′′′′′′′′′++++′′′′′′′′′′′′−−−−′′′′′′′′++++′′′′−−−−====

++++′′′′′′′′′′′′′′′′++++′′′′′′′′′′′′++++′′′′′′′′++++′′′′++++====

−−−−

++++

L

L

)x(f!4

h)x(f

!3h

)x(f!2

h)x(fh)x(f)x(f

)x(f!4

h)x(f

!3h

)x(f!2

h)x(fh)x(f)x(f

i

4

i

3

i

2

ii1i

i

4

i

3

i

2

ii1i

� Second-order accurate O(h2)

++++′′′′′′′′′′′′′′′′++++′′′′′′′′++++====++++⇒⇒⇒⇒

++++++++−−−−++++−−−−====

−−−−++++

−−−−

L

L

)x(f!4

h)x(f

!2h

)x(f2)x(f)x(f

)x(f!4

)x(f!3

)x(f!2

)x(fh)x(f)x(f

i

4

i

2

i1i1i

iiiii1i

)(!

)()()()( ξξξξf

4h

hxfxf2xf

xf2

21ii1i

i ′′′′′′′′′′′′′′′′−−−−++++−−−−====′′′′′′′′ −−−−++++

Given find numerical approximations to the second derivative , using three points and the central difference formula.

Centered FiniteCentered Finite--Divided Divided DifferencesDifferences

Forward FiniteForward Finite--divided divided differencesdifferences

Backward finiteBackward finite--divided divided differencesdifferences

Higher DerivativesHigher Derivatives� All second-order accurate O(h2)

21ii1i

i

1i1ii

)x(f)x(f2)x(f2)x(fh

)x(f)x(f2)x(f)x(f

h2

)x(f)x(f)x(f

−−−−++++

−−−−++++

−−−−++++−−−−

++++−−−−====′′′′′′′′

−−−−====′′′′

� More nodal points are needed for higher derivatives� Higher order formula may be derived

42i1ii1i2i

i

32i1i1i2i

i

h

)x(f)x(f4)x(f6)x(f4)x(f)x(f

h

)x(f)x(f2)x(f2)x(f)x(f

−−−−−−−−++++++++

−−−−−−−−++++++++

++++−−−−++++−−−−====′′′′′′′′′′′′′′′′

−−−−++++−−−−====′′′′′′′′′′′′

Error Analysis and Optimum Step Error Analysis and Optimum Step size size

Error Analysis and Optimum Step Error Analysis and Optimum Step size size