Truncation errors: using Taylor series to approximate functions

Let’s say we want to approximate a function !(#) with a polynomial

For simplicity, assume we know the function value and its derivatives at #% = 0 (we will later generalize this for any point). Hence,

Approximating functions using polynomials:

! # = (% + (* # + (+ #+ + (, #, + (- #- + ⋯

! 0 = (%

!/ # = (* + 2 (+ # + 3 (, #+ + 4 (- #, +⋯

!′ 0 = (*

!//(#) = 2 (+ + 3×2 (, # + (4×3)(- #+ + ⋯

!′′ 0 = 2 (+!′′′ 0 = (3×2) (,

!/5 0 = (4×3×2) (-

!///(#) = 3×2 (, + (4×3×2)(- # +⋯!/5(#) = (4×3×2)(- +⋯

or f" '

ei ! ai

L4ai=f"Yi!T

f "

fixCi - it x Ci - r ) x. . . x1) ai

Taylor SeriesTaylor Series approximation about point !" = 0

% ! =&'()

*+' !'

% ! = +" + +- ! + +. !. + +/ !/ + +0 !0 + ⋯

⇒fas÷÷÷÷TI→ approximate function values

→ approximate derivatives

→ estimating errors

Taylor Series

! " = ! "$ + !& "$ (" − "$) +!&& "$2! (" − "$),+

!&&& 03! (" − "$)/+⋯

In a more general form, the Taylor Series approximation about point "$ is given by:

! " =1234

5 ! 2 ("$)6! (" − "$)2

Example:Assume a finite Taylor series approximation that converges everywhere for a given function !(#) and you are given the following information:

! 1 = 2; !)(1) = −3; !))(1) = 4; ! - 1 = 0 ∀ 0 ≥ 3

Evaluate ! 4 _fI x) = f Go) t f'

ED G - Xo ) tf "z G - xo5tfHMake x = A and Xo = I

f- (4) = fu ) t f 'Ll) C4 - D t ft ) (4 - D'

= 2 t f - 3) ( 4- Dt 2414-D2

= 2 - 9 t 18 ⇒/fl4)=HJ

Taylor SeriesWe cannot sum infinite number of terms, and therefore we have to truncate.

How big is the error caused by truncation? Let’s write ℎ = # − #%x = ht Xo

-

ffxoth)=fGDtf'GDhtf¥hit h't -- -

t÷t"fh+÷i.

hi

exact - -

truncated part what we are neglecting(Taylor approximation error

ffx) of degree n )

thx )

Taylor series with remainderLet ! be (# + 1)-times differentiable on the interval ('(, ') with !(*)continuous on ['(, '], and ℎ = ' − '(error = exact - approximation

error = fat - tn Cx ) = ¥÷, f"f hi

ht 2

=f h

" "

+ f"o ) h + . . .

( ht2) !

-dominant term when h → o Cor x → to )

error f M h" "

or error = O I hit'

)

Taylor series with remainderLet ! be (# + 1)-times differentiable on the interval ('(, ') with !(*)continuous on ['(, '], and ℎ = ' − '(error = exact - approximation

Remainder Theorem : Rnlx) =FG ) - tncx )

=÷:S . ii

/RnH=fII,Its*1-where BE ( Xo ,

x )

}since 15-xd E h

#×

iris it'ttII¥D

Graphical representation:

tix )f f' Go)=d¥ → dy = f

'

GD ( x - Xo )

\•Ferrier t

,=f Gd t f' Go )( X - Xo ) V

A

go.ua#iIIdYerror=fCD-tiCDa--Remainder

YI errorEfM5Xg.Y-xD_EE@o.x )I I

f' GD'

i !/error=0(h#tangentatxoi

, -I I

" !>

supposeinterval is reduced by half .

% Xwhat happens to the error ?

÷ ÷ '¥¥I

Example:Given the function

! " = 1(20" − 10)

Write the Taylor approximation of degree 2 about point "* = 0

found

Given the function : FG ) = 120X - LO

write the Taylor approximation of degree 2 about to = o

¥47joking;tho ) =

. 's

G) =

-1204%1726,492=¥g÷o ,

f' 'lol = -9%7=-45

ftdxk-f-shx-EIE.TT trackf"z x'I errors OH )

-

-3- to

. .

µ-

Z10-1

-

y= ax

!error log

bogeys-

- liogcatbwlogcx)

Islope !b=bfg:fg:

= III. =3

,→ error -

- ou 's r

Example:Given the function

! " = −"% + 1

Write the Taylor approximation of degree 2 about point "( = 0

IDENT

AH = ft xo) tf'GDC x - xD tf "f ( x - * It - - -

f- ( o ) = I

f-' G) = ¥C- x 't 15

"

= - X ( I - Ej" '

→ ft Co) = o

f"

A) = C I- Ej"

EH - a - xD" "

→ f''

to ) = -

A

thx) = I - zcx5 or /tdD=I-I

! " = −"% + 1error = tz - Hx )

O

{t2

¥

"

good"

approximation close to xo = o

! '

error increases when x moves away from to

-use log-hog plot to better visualize what is happening

Close to to.

error = thx ) - far )

f- G) =NETT

- no-5

thx ) = t - %

-1-0-9IRat s

ITHI.li/⇐

Olli)

3 !

here h =X - Xo =

X

Let's get Big-0 of error from the plot !

sbr-tfggfj.bg:7?-=-.s.IL--4

→ error -

- och" )

what happened here !

→ I"

G) = o hence the next term that is not

zero is f'

*G)

Example:

A)B)C)D)E)

Demo “Taylor of exp(x) about 2”

IDEMOJ

error = e' I ¥75'ta÷5t - -- ]

t#error = Off-29 )

* ef.MG - 24as x → 2

,

e becomessmaller

* e E M x"

} all are valid options !as × → 2 ,

ears does not

show asymptoticbehavior

- this is .

tightestbound .

Finite difference approximationFor a given smooth function ! " , we want to calculate the derivative !′ " at " = 1.Suppose we don’t know how to compute the analytical expression for !′ " , but we have available a code that evaluates the function value:

Can we find an approximation for the derivative with the available information? YES !

Caldasf'

G) = ¥¥fffGth)h-f#)

Can we simply use f'G) a fCxth)h-f# and make

h small ? How do we choose h ?

what is the error ?

f : IR → IR and f is smooth I h : small change#tur baton )K

to X .

- Taylor Series :

f- ( x the) = HHtf

!x) h tfz h 't.. . Cjcest replaced to with x )

f- ( x th ) flex ) tf'

G) h . to Chi )

ffxth ) - f G)=

f'

G) to Ch ) → f'G) =fGth) - Och)

: - -

-error

he extract -

fdf¥f¥fFITTED# F. D. approx )

/errordue\to FD approx

⇒ et = 01h )-

Demo: Finite Difference

! " = $% − 2

(!$")*+ = $%

(!),,-." = $%/0 − 2 − ($%−2)ℎ

$--.-(ℎ) = )45((!$")*+ − (!),,-.")

We want to obtain an approximation for !′ 1

$--.- < !99 : ℎ2

truncation error

IE IE's

Demo: Finite Difference

! " = $% − 2

(!$")*+ = $%

(!),,-." = $%/0 − 2 − ($%−2)ℎ

$--.-(ℎ) = )45((!$")*+ − (!),,-.")

We want to obtain an approximation for !′ 1

ℎ $--.-

$--.- < !99 : ℎ2

truncation error

Demo

I I

↳ absolute error !

Demo: Finite Difference

Should we just keep decreasing the perturbation ℎ, in order to approach the limit ℎ → 0 and obtain a better approximation for the derivative?

$′ & = lim+→,$ & + ℎ − $(&)

ℎ

~ error =Och )

I

Uh-Oh!What happened here?

! " = $% − 2

!′ " = $% → !′ 1 ≈ 2.7

!′ 1 = lim1→2

! 1 + ℎ − !(1)ℎ①

-managing:Freeman.

D very small h Chiao"

)

f- ( Ith ) - to ) →starts increasing

losingprecision due to cancelation !

2) even smaller Chf to- "

) → f C Ith ) = fu ) → If = O

I machine epsilon C because we are adding

h to I.

If another FP number,

than this would

be the gap .

!""#"~%ℎ2Truncation error:

Rounding error: !""#"~2(ℎ

I

I

'

-8I 10

in

h =

Eh⇒ HIE

h -- NI

f- ( Ith ) - fu ) h =NIF

- -8h he to