Numerical DifferentiationNumerical Differentiation� Estimate the derivatives (slope, curvature, etc.)
of a function by using the function values at only a set of discrete points
� Ordinary differential equation (ODE)� Partial differential equation (PDE)� Partial differential equation (PDE)� Represent the function by Taylor polynomials
or Lagrange interpolation� Evaluate the derivatives of the interpolation
polynomial at selected (unevenly distributed) nodal points
0
( ) ( )Differentiation lim
( )
( ) ( ) Difference
( )
x
y f(x)
dy f x x f x
dx x x x
y f x x f x dy
x x x x dx
∆ →
=+∆ −=+∆ −
∆ +∆ −= ≈∆ +∆ −
Difference ( )
A smaller step (or h) results in a smaller error
x x x x dx
x
= ≈∆ +∆ −
∆
xi-3 xi-2 xi-1 xi xi+1 xi+2 xi+3
hx =∆
Evenly distributed points along the x-axis
Distance between two neighboring points is the same, i.e. h.
x1 x2 x3
Unevenly distributed points along the x-axis
Distance between two neighboring points is the same, i.e. h.
Lagrange InterpolationLagrange Interpolation� 1st-order Lagrange polynomial (Two(Two--Point Point
)))()()()()()( 1
01
00
10
111001 xf
xxxx
xfxxxx
xfxLxfLxf−−−−−−−−++++
−−−−−−−−====++++====
� Second-order Lagrange polynomial (Three(Three--Point )Point )
)x(f)xx)(xx(
)xx)(xx(
)x(f)xx)(xx(
)xx)(xx(
)x(f)xx)(xx(
)xx)(xx()x(f
21202
10
12101
20
02010
212
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−====
General ThreeGeneral Three--Point Point FormulaFormula
�Lagrange interpolation polynomial for unequally spaced data
)())((
)()()()()()()(
1i1ii
1i1iii1i1i
xfxxxx
xfxLxfxLxfxLxf
−−−−++++
++++++++−−−−−−−−
−−−−−−−−−−−−−−−−====
++++++++====
)())((
))((
)())((
))((
)())((
1ii1i1i1i
i1i
i1ii1ii
1i1i
1i1i1ii1i
xfxxxx
xxxx
xfxxxx
xxxx
xfxxxx
++++++++−−−−++++
−−−−
++++−−−−
++++−−−−
−−−−++++−−−−−−−−
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−++++
−−−−−−−−====
Lagrange InterpolationLagrange Interpolation
L 2(x)f(x2)L 0(x)f(x0)
L 1(x)f(x1)
x0 x1 x2
General ThreeGeneral Three--Point Point FormulaFormula
�Lagrange interpolation polynomial for unequally spaced data
))((
))(())((
)())((
))(()(
)()()()()()()(
1ii1ii
1i1ii
1i1ii1i
1ii1i
1i1iii1i1i
xxxx
xxxxxxxx
xfxxxx
xxxxxf
xfxLxfxLxfxLxf
−−−−−−−−−−−−−−−−
−−−−−−−−++++−−−−−−−−
−−−−−−−−====
++++++++====
++++−−−−
++++−−−−
++++−−−−−−−−
++++−−−−
++++++++−−−−−−−−
�First derivative
))(())((
)(i1i1i1i
i1i1i xxxx
xxxxxf
−−−−−−−−−−−−−−−−++++++++−−−−++++
−−−−++++
))(()(
))(()(
))(()()(
i1i1i1i
i1i1i
1ii1ii
1i1ii
1i1ii1i
1ii1i
xxxxxxx2
xf
xxxxxxx2
xfxxxx
xxx2xfxf
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−====′′′′
++++−−−−++++
−−−−++++
++++−−−−
++++−−−−
++++−−−−−−−−
++++−−−−
Lagrange polynomial (Two(Two--Point )Point )
)()()()()()( 10
01
11001 xfxx
xfxx
xfxLxfLxf−−−−−−−−++++
−−−−−−−−====++++==== )()()()()()( 1
010
1011001 xf
xxxf
xxxfxLxfLxf
−−−−++++
−−−−====++++====
Forward difference
Backward difference
Numerical Numerical DifferentiationDifferentiation
Backward difference
Centered difference
Forward differenceForward difference
x i−−−−1 x i x i+1x
h
Backward differenceBackward difference
x i−−−−1 x i x i+1x
h
First DerivativesFirst Derivatives)x(f ′′′′
y
� Forward difference
� Backward difference
i-2 i-1 i i+1 i+2
1
1
1
1
1
1
1
1
)()()(
)()()(
−
−
−
−
+
+
+
+
−−=
−−≅′
−−=
−−≅′
ii
ii
ii
ii
ii
ii
ii
ii
xx
yy
xx
xfxfxf
xx
yy
xx
xfxfxf
x
Given find numerical approximations to the derivative , using two points and the forward difference formula.
Given find numerical approximations to the derivative , using two points and the backward difference formula.
Truncation ErrorsTruncation Errors� Uniform grid spacing
++++′′′′′′′′′′′′−−−−′′′′′′′′++++′′′′−−−−====−−−−====
++++′′′′′′′′′′′′++++′′′′′′′′++++′′′′++++====++++====
−−−−
++++
L
L
)x(f!3
h)x(f
!2h
)x(fh)x(f)hx(f)x(f
)x(f!3
h)x(f
!2h
)x(fh)x(f)hx(f)x(f
i
3
i
2
iii1i
i
3
i
2
iii1i
′′′′′′′′′′′′−−−−−−−−====′′′′
′′′′′′′′++++−−−−====′′′′
′′′′′′′′−−−−−−−−====′′′′
−−−−++++
−−−−
++++
))()()(
)(:
)()()(
)(:
)()()(
)(:
23
21i1i
i
21ii
i
1i1i
i
O(h f6h
h2xfxf
xfcentral
O(h) f2h
hxfxf
xfbackward
O(h) f2h
hxfxf
xfforward
ξξξξ
ξξξξ
ξξξξ
Example: First DerivativesExample: First Derivatives� Use forward and backward difference approximations
to estimate the first derivative of
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
� Forward Difference
2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====
� Backward Difference
====−−−−====−−−−====−−−−−−−−====′′′′====
====−−−−====−−−−====−−−−−−−−====′′′′====
%.,..
....
).().().(,.
%.,..
...
).()().(,.
526 1551250
925063632813050750
50f750f50f 250h
958 45150
925020501
50f1f50f 50h
t
t
εεεε
εεεε
====−−−−====−−−−====−−−−−−−−====′′′′====
====−−−−====−−−−====−−−−−−−−====′′′′====
%.,....
..).().(
).(,.
%.,..
...
)().().(,.
721 71402501035156319250
25050250f50f
50f 250h
739 55050
219250050
0f50f50f 50h
t
t
εεεε
εεεε
Lagrange polynomial (Three(Three--Point )Point )
)x(f)xx)(xx(
)xx)(xx()x(f 0
212 −−−−−−−−
−−−−−−−−====
)x(f)xx)(xx(
)xx)(xx(
)x(f)xx)(xx(
)xx)(xx(
)xx)(xx(
21202
10
12101
20
2010
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−++++
−−−−−−−−
First DerivativesFirst Derivatives)x(f ′′′′
Parabolic curve
� 3 -point Forward difference
� 3 -point Backward difference
i-2 i-1 i i+1 i+2
h
xfxfxf
xx
xfxfxfxf
h
xfxfxf
xx
xfxfxfxf
iii
ii
iii
iii
ii
iii
2
)()(4)(3)()(4)(3)(
2
)()(4)(3)()(4)(3)(
21
2
21
21
2
21
−−
−
−−
++
+
++
+−=−
+−≅′
−+−=−
−+−≅′
First DerivativesFirst Derivatives)x(f ′′′′
Parabolic curve
� 3 - point central difference i-2 i-1 i i+1 i+2
h
yy
h
xfxf
xx
yy
xx
xfxfxf
iiii
ii
ii
ii
ii
22
)()(
)()()(
1111
11
11
11
11
−+−+
−+
−+
−+
−+
−=−≅
−−=
−−≅′
Centered differenceCentered difference
x i−−−−1 x i x i+1x
2h
Example: First DerivativeExample: First Derivative� Use central difference approximation to estimate the
first derivative of
at x = 0.5 with h = 0.5 and 0.25 (exact sol. = -0.9125)
� Central Difference
2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====
%4.2,934.05.0
10351563.163632813.0
25.075.0)25.0(f)75.0(f
)5.0(f,25.0h
%6.9,0.11
2.12.001
)0(f)1(f)5.0(f,5.0h
====−−−−====−−−−====
−−−−−−−−====′′′′====
====−−−−====−−−−====−−−−−−−−====′′′′====
t
t
εεεε
εεεε
Example: First DerivativesExample: First Derivatives� Use forward and backward difference approximations
of O(h2) to estimate the first derivative of
at x = 0.5 with h = 0.25 (exact sol. = -0.9125)
� Forward Difference
2.1x25.0x5.0x15.0x1.0)x(f 234 ++++−−−−−−−−−−−−−−−−====
� Backward Difference
%82.5,859375.05.0
)925.0(3)6363281.0(42.0
)25.0(2)5.0(f3)75.0(f4)1(f
)5.0(f
====−−−−====−−−−++++−−−−====
−−−−++++−−−−====′′′′
t
εεεε
%77.3,878125.05.0
2.1)035156.1(4)925.0(3
)25.0(2)0(f)25.0(f4)5.0(f3
)5.0(f
====−−−−====++++−−−−====
++++−−−−====′′′′
t εεεε
Given find numerical approximations to the derivative , using three points and the forward difference formula.
Given find numerical approximations to the derivative , using three points and the backward difference formula
Given find numerical approximations to the derivative , using three points and the central difference formula.
Error For First Error For First Derivatives Derivatives ((ThreeThree--Point )Point )
Second DerivativeSecond Derivative� First Derivative for unequally spaced data
))(()(
))(()(
))(()()(
i1i1i
1ii1ii
1i1ii
1i1ii1i
1ii1i
xxxxxxx2
xf
xxxxxxx2
xfxxxx
xxx2xfxf
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−++++
−−−−−−−−−−−−−−−−====′′′′
++++−−−−++++
−−−−++++
++++−−−−
++++−−−−
++++−−−−−−−−
++++−−−−
� Second Derivative for unequally spaced data
))(( i1i1i1i xxxx −−−−−−−− ++++−−−−++++
))(()(
))(()(
))(()()(
i1i1i1i1i
1ii1iii
1i1ii1i1i
xxxx2
xf
xxxx2
xfxxxx
2xfxf
−−−−−−−−++++
−−−−−−−−++++
−−−−−−−−====′′′′′′′′
++++−−−−++++++++
++++−−−−++++−−−−−−−−−−−−
SecondSecond--DerivativesDerivatives� Taylor-series expansion
� Uniform grid spacing
++++′′′′′′′′′′′′′′′′++++′′′′′′′′′′′′−−−−′′′′′′′′++++′′′′−−−−====
++++′′′′′′′′′′′′′′′′++++′′′′′′′′′′′′++++′′′′′′′′++++′′′′++++====
−−−−
++++
L
L
)x(f!4
h)x(f
!3h
)x(f!2
h)x(fh)x(f)x(f
)x(f!4
h)x(f
!3h
)x(f!2
h)x(fh)x(f)x(f
i
4
i
3
i
2
ii1i
i
4
i
3
i
2
ii1i
� Second-order accurate O(h2)
++++′′′′′′′′′′′′′′′′++++′′′′′′′′++++====++++⇒⇒⇒⇒
++++++++−−−−++++−−−−====
−−−−++++
−−−−
L
L
)x(f!4
h)x(f
!2h
)x(f2)x(f)x(f
)x(f!4
)x(f!3
)x(f!2
)x(fh)x(f)x(f
i
4
i
2
i1i1i
iiiii1i
)(!
)()()()( ξξξξf
4h
hxfxf2xf
xf2
21ii1i
i ′′′′′′′′′′′′′′′′−−−−++++−−−−====′′′′′′′′ −−−−++++
Given find numerical approximations to the second derivative , using three points and the central difference formula.
Centered FiniteCentered Finite--Divided Divided DifferencesDifferences
Forward FiniteForward Finite--divided divided differencesdifferences
Backward finiteBackward finite--divided divided differencesdifferences
Higher DerivativesHigher Derivatives� All second-order accurate O(h2)
21ii1i
i
1i1ii
)x(f)x(f2)x(f2)x(fh
)x(f)x(f2)x(f)x(f
h2
)x(f)x(f)x(f
−−−−++++
−−−−++++
−−−−++++−−−−
++++−−−−====′′′′′′′′
−−−−====′′′′
� More nodal points are needed for higher derivatives� Higher order formula may be derived
42i1ii1i2i
i
32i1i1i2i
i
h
)x(f)x(f4)x(f6)x(f4)x(f)x(f
h
)x(f)x(f2)x(f2)x(f)x(f
−−−−−−−−++++++++
−−−−−−−−++++++++
++++−−−−++++−−−−====′′′′′′′′′′′′′′′′
−−−−++++−−−−====′′′′′′′′′′′′
Error Analysis and Optimum Step Error Analysis and Optimum Step size size
Error Analysis and Optimum Step Error Analysis and Optimum Step size size