An Introduction to Programming though C++cs101/lectures/Lec7.pdf · An Introduction to Programming...

An Introduction to Programming though C++

Abhiram G. RanadeCh. 8: Computing Common

Mathematical Functions

Goal: Learn methods for performing common mathematical operations

• Evaluating common mathematical functions such as– sin(x), log(x), ...

• Integrating functions numerically, i.e. when you do not know the closed form.– Intrinsically useful– Also useful for evaluating certain functions

• Finding roots of functions, i.e. find x s.t. f(x) = 0.– Intrinsically useful– Also useful for evaluating certain functions

• All the methods give approximate answers. – Exact answers cannot be produced using just finitely many arithmetic operations.– Error can be made as small as we want by doing more and more work.

• The programs will be simple, using just a single loop.

Outline

• Taylor Series• Numerical Integration• Bisection Method• Newton-Raphson Method

Introduction to Taylor series

How do we calculate sin(x), cos(x), ln(x), ...?• Can be expressed as sum of infinite number of termsExample:sin(x) = x – x3/3! + x5/5! – x7/7! + ...

– Each term is easy to evaluate– By taking the first few terms we can get fairly good answer– Need more accuracy? Take more terms.

Additional property: Error using first k terms ≤ |k+1th term|.

Let us compute sin(x) = x – x3/3! + x5/5! – ...

• Need to add together t1 = +x, t2 = - x3/3!, t3 = +x5/5!, ... • tk = (-1)k+1x2k-1/(2k-1)!• tk+1 = tk (-x2)/((2k)(2k+1)• t1 = x is given as input, available for iteration 1.• Iteration k: add tk to running sum• At the beginning of iteration k: we will have tk in variable T.• At the beginning of iteration k, we will have S = sum of first k-1

terms• At end we need: S= sum of first k terms, T = k+1th term • In iteration k: add T to S. Then multiply T by (-x2)/((2k)(2k+1).

Program

main_program{ double x; cin >> x;  cout <<"Calculating sin(x) for x = "<<x<<endl;

double s=0, t=x;   for(int k=1; k<=10; k++){    s = s + t;    t = - t * x * x/(2*k)/(2*k+1);    cout <<"No of terms: "<< k <<"  Value: "<< s <<"  Error estimate: "<< abs(t) <<endl;  }  cout <<endl<<" From library function: "<<sin(x)<<endl;}

Taylor’s theorem

Taylor’s theorem: Many interesting functions can be written as: f(x) = f(x0) + f’(x0)(x – x0) + f’’(x0)(x – x0)2 / 2! + f’’’(x0)(x – x0)3 / 3! + …if x is close to x0, i.e. |x – x0| < “radius of convergence”.

We can often choose x0 such that it is possible to easily evaluate f and its derivatives at x0

Example: f(x) = sin(x) : Choose x0 = 0. f(0) = sin(0) = 0, f’(0) = cos(0) = 1, f’’(0) = -sin(0) = 0, f’’’(0) = -cos(0) = -1...So sin(x) = 0 + x + 0 – x3/3! + 0 ... Radius of convergence happens to be infinite for sin(x) at x0 = 0.

What we discussed

• Taylor series are available for many many functions.– Exercise: construct the series and write the program to

calculate cos(x).

• Other infinite expressions have also been used for calculating math functions, e.g. infinite products, continued fractions.– See exercises in book.

♞

Numerical integration

• In many scientific calculations we need to integrate.– Closed form may not be

available.

• We can integrate numerically– Integral = area under the

curve– Approximate area by

rectangles.– The more rectangles we

use, more accurate is the answer

p q

f

Plan for writing a program

• Read p, q.• Read n = number of rectangles.• Calculate w = width of

rectangle = (q-p)/n.• Consider ith rectangle,

i=0,1,..,n-1– Begins at x = p+iw.– Height = f(x) = f(p+iw)– Area = w * f(p+iw)

• Integral = sum over all i. p q

f

The generic program

• Read p, q.• Read n = number of rectangles.• Calculate w = width of rectangle = (q-p)/n.• Consider ith rectangle, i=0,1,..,n-1

– Begins at x = p+iw.– Height = f(x) = f(p+iw)– Area = (p+iw) * f(p+iw)

• Integral = sum over all i.

Should put code to evaluate f(p+iw)Next

main_program{ double p, q; cin >>p>>q; int n; cin >> n; double w = (q-p)/n; double area = 0; for(int i=0; i<n; i++){ area = area + w*f(p+iw); } cout << area << endl;}

Numerical integration to calculate ln(x)

ln(q) = natural logarithm = area under curve f(x)=1/x from 1 to q.

main_program{ double p=1, q; cin >>q; int n; cin >> n; double w = (q-p)/n; double area = 0; for(int i=0; i<n; i++){ area = area + w*1/(p+iw); } cout << area << endl; cout << log(x) << endl;}

Analysis of the error

• Error 1: due to the gap between the rectangles and the curve.– Can be reduced by increasing the number of rectangles.

• Error 2: in area of each rectangle– Each number is expressed to precision of few digits: 7-8 for float, 16-17 for

double.– So error of 10-8 or 10-17 per rectangle.– If you add up n such areas error increases to n*10-8 or n*10-17.– So be careful in increasing n too much.

• Ways of decreasing errors:– Use trapeziums instead of rectangles, hug curve better– Set rectangle height = function value at the midpoint of its width. (See text)

Exercise

• Can you predict whether our answer will be larger or smaller than the actual value of the integral, ln(x)?

• Will the error be less if we take the height to be the function value in the middle of the rectangle rather than at the beginning?– How does this work for f(x) = x?

What we discussed

• Integration is needed in many places in scientific computing.– Numerical integration can be used if closed form solutions are not

available.

• In order to perform numerical integration we just need to be able to evaluate the given function at an arbitrary point.

• Error will depend upon how many and what kind of rectangles you use.

Next: Bisection method for finding roots♞

Bisection method for finding roots

• Root of function f: Value x such that f(x)=0.• Many problems can be expressed as finding roots.

– e.g. square root of w = root of f(x) = x2 – w.– If f(x) = 0, then x2 – w =0, i.e. x = √w.

• Requirement for bisection method:– Need to be able to evaluate f.– f must be continuous.– We must be given points xL and xR such that f(xL) and f(xR) are

not both positive or both negative. “Invariant”

Bisection method: main idea

f(xL), f(xR) have different signs or are 0.+ continuity ⇒ Root exists between xL and xR.• Basic iteration will bring xL, xR

closer, while maintaining invariant:

• We iterate till xR – xL ≤ , our 𝜀, our desired error bound

• We declare xL as the root.• Error in declared root is at most . 𝜀, our

IterationLet xM = (xL + xR)/2 midpoint of interval (xL, xR).If f(xM) has same sign as f(xL), then set xL = xM

Sign of xL did not change Sign of xL continues to remain different from sign of xR

else set xR = xM.

Sign of xR does not change... Invariant holds even here

Bisection method for finding square root of 2, i.e. root of f(x)=x2 – 2.

double xL=0, // f(xL) = 0 – 2 is negative xR=2, // f(xR) = 4 – 2 is positive xM, epsilon = 0.00001;while(xR – xL >= epsilon){ xM = (xL+xR)/2; if((xL*xL – 2 > 0 && xM*xM – 2 > 0) || (xL*xL – 2 < 0 && xM*xM – 2 < 0)) xL = xM; else xR = xM;}cout << xL << endl;// How would you choose xL, xR for finding sqrt(3)?// xL = 0, xR = 3 will work

Demo

• bisection.cpp

Remarks

• In each iteration, the interval (xL, xR) halves in size.• The size of the interval gives the error in the root.• Thus the error in the root halves in each iteration.• Thus if you want the answer correct to k bits, you should use

k iterations.• The number of calculations in each iteration can be reduced.

– See the book.

Exercise: Modify the program so that it calculates the cube root of any number w. Make sure you correctly initialize xL, xR.

What we discussed

• Bisection method is a very simple method for finding the root of a function f.

• Requirements:– Should be possible to evaluate f at any x.– f should be continuous.– Need xL and xR such that f(xL) and f(xR) dont have the same sign.– Method is simple but can be slow.

Next: Newton Raphson method for finding roots♞

Newton Raphson method

Method to find the root of f(x), i.e. x s.t. f(x)=0.

Method works if:

f(x) and derivative f '(x) can be easily calculated.

A good initial guess x0 for the root is available.

Example: To find square root of y.

use f(x) = x2 - y. f ‘ (x) = 2x.

f(x), f ‘ (x) can be calculated easily. 2,3 arithmetic

ops.

Initial guess x0 = 1 is good enough!

How to get better xi+1 given

xi

f(x)xixi+1

Point A =(xi,0) known.

A

B

C

xi+1= xi – AC = xi - AB/(AB/AC) = xi- f(xi) / f‘(xi)

Calculate f(xi ).

Point B=(xi,f(xi))

Approximate f by tangentC= intercept on x axis C=(xi+1,0)

Square root of y

xi+1 = xi- f(xi) / f‘(xi)f(x) = x2 - y, f‘(x) = 2xxi+1 = xi - (xi2 - y)/(2xi) = (xi + y/xi)/2

Starting with x0=1, we compute x1, then x2, …

We can get as close to sqrt(y) as required. Proof not part of the course.

Code

double y; cin >> y;

double xi=1; // Guess. Known to work.

repeat(10){

xi = (xi + y/xi)/2;

}

cout << xi << endl;

How to iterate until error is small

xi

root

Error

Error Estimate = |f(xi)|= |xi*xi – y|

Make |xi*xi – y| small

double y; cin >> y;

double xi=1;

while(abs(xi*xi – y) > 0.001){

xi = (xi + y/xi)/2 ;

}

cout << xi << endl;

Error Analysis

• Number of correct bits double with each iteration!• Proof not in course.

Remarks

• Babylonians used this method to find square roots 3500 years ago!

• Newton’s method is very commonly used.• Also useful in multiple dimensions. Given functions f, g, h,

… Find x, y, z, w, ... such that • f(x, y, z, w, …) = 0 • g(x, y, z, w, …) = 0• h(x, y, z, w, …) = 0.• . . .–But it is trickier too (Chapter 29).

Concluding Remarks

• If you want to find f(x), then – use Taylor series for f, if f and its derivatives can be evaluated at some point

x0 close to x.– Express f as an integral of some easily evaluable function g, and use

numerical integration– Express f as the root of some easily evaluable function g, and use bisection

or Newton-Raphson.

• All the methods are iterative, i.e. the accuracy of the answer improves with each iteration.

• The ideas of this chapter are very fundamental, and will appear later also, e.g. Chapter 19, Chapter 29.

♞♞