# Analysis of Algorithms Chapter - 02 Recurrences

• View
25

0

Tags:

• #### recurrenceskhloud shawagfeh

Embed Size (px)

DESCRIPTION

Analysis of Algorithms Chapter - 02 Recurrences. 1. Methods. Definition: A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs. The worst-case running time T(n) of an algorithm could be described as by the recurrence - PowerPoint PPT Presentation

### Text of Analysis of Algorithms Chapter - 02 Recurrences

• *Analysis of Algorithms

Chapter - 02Recurrences

Khloud Shawagfeh

• * Definition: A recurrence is an equation or inequality that describes a function in terms of its value on smaller inputs. The worst-case running time T(n) of an algorithm could be described as by the recurrenceT(n) = aT(n/b) + f(n) where a1, b>1, and f(n) is a given function. Three methods will be discussed to solve the recurrences obtaining asymptotic bounds on the solution. Substitution method: We guess a bound and then use mathematical induction to prove that our guess is correct. Recursion-tree method: It converts the recurrence into a tree whose nodes represent the cost incurred at various levels of the recursion We use techniques for bounding summations to solve the recurrence. Master method: It provides bounds for recurrences of the above form. It requires memorization of three cases. Once you memorize the cases, determining asymptotic bounds for many recurrences is easy.Methods

Khloud Shawagfeh

• *Definition of Proof: The process or an instance of establishing the validity of a statement especially by derivation from other statements in accordance with principles of reasoning.

Proof by Induction: Let Pn be a statement for all the positive integers (n = 1, 2, 3, . . .). If the following two properties hold:P1 is true.Pk+1 is true if Pk is true for each positive integer k. Then Pn is true for all n.

Why Induction Works?Let S be the set of all numbers n for which Pn is false.Let k be the minimum number in S.k > 1 since by the first property of the induction definition, P1 is true.By the minimality of k, Pk1 is true and Pk is false.A contradiction to the second property of the induction definition.Induction

Khloud Shawagfeh

• * Prove that for any integer n 1:1 + 2 + 3 + + n = n(n+1)/2 . Define: L(n) = 1 + 2 + 3 + + n. R(n) = n(n+1)/2 .

Prove that L(n) = R(n) for n 1.

Verifying the Claim

First check for small values of n:A Summation Problem

nL(n)R(n)11=11.2/2=121+2=32.3/2=331+2+3=63.4/2=6:::101+2+3++10=5510.11/2=55

Khloud Shawagfeh

• * Idea: Compute the value of 2L(n).

Example: 2(1 + 2 + 3) = (1 + 2 + 3) + (3 + 2 + 1)= (1 + 3) + (2 + 2) + (3 + 1)= 3 4= 12= 2(34/2)

Hence2L(n) = (1 + + n) + (n + + 1) = (1 + n) + (2 + (n 1)) + + (n + 1) = (n + 1) + (n + 1) + + (n + 1) = n(n + 1)

This implies that: L(n) = n(n+1)/2 = R(n).

A Direct Proof

Khloud Shawagfeh

• * For n=1, L(1) = 1 and R(1) = (1.2)/2 = 1 So, for n=1, it is true.

Let us assume that it is true for n=k, that is, L(k) = R(k) =>1+2+3+..+k = k(k+1)/2

Now, if we can prove that it is true for n=k+1 also, then it can be said that it is true for all n. For n=k+1, L(k + 1) = 1 + 2 + + k + (k + 1)= L(k) + (k + 1)= R(k) + (k + 1)= k(k + 1)/2 + (k + 1)= (k + 1)(k/2 + 1)= (k + 1)(k + 2)/2= R(k + 1)

Hence L(n) = R(n), for all integer n.A Proof by Induction

Khloud Shawagfeh

• *Substitution Method

Khloud Shawagfeh

• * The method for solving recurrences entails two steps: Guess the form of the solution Use mathematical induction to find the constants and show that the solution works. This method is powerful, but it obviously can be applied only in cases when it is easy to guess the form of the solution. This method can be used to establish either upper or lower bounds on a recurrence.

Problem: Let us determine an upper bound on the following recurrence:T(1) = 0.T(n) = 2T(n/2) + n, for n>1. Solution: Compute the solution for small powers of 2:T(2) = 2T(1) + 2 = 2.T(4) = 2T(2) + 4 = 8.T(8) = 2T(4) + 8 = 24.T(16) = 2T(8) + 16 = 64.T(32) = 2T(16) + 32 = 160.Recurrence -1

Khloud Shawagfeh

• *Guessing the Solution - Guess T(n) = n log2 n for n a power of 2.

A Proof by InductionFor n=1, T(1)=0, that is T(n)=nlog2 nSo, for n=1, it is true.

Let us assume that it is true for n=k/2, that is, T(k/2)=(k/2) log2 (k/2)

Now, if we can prove that it is true for n=k also, then it can be said that it is true for all n. For n=k,T(k) = 2T(k/2) + k = 2(k/2) log2(k/2) + k = k(log2 k 1) + k = k log2 k

Hence T(n) = n log2 n, for all integer n.Recurrence -1 (Contd.)

Khloud Shawagfeh

• *Recurrence -2 Problem: Let us determine an upper bound on the following recurrence:T(1) = a.T(n) = 2T(n/2) + bn.For some constants a, b (independent of n).

Solution: Compute the solution for small powers of 2: T(2) = 2T(1) + 2b = 2b + 2a. T(4) = 2T(2) + 4b = 8b + 4a. T(8) = 2T(4) + 8b = 24b + 8a. T(16) = 2T(8) + 16b = 64b + 16a. T(32) = 2T(16) + 32b = 160b + 32a.

Guessing the Solution Guess T(n) = bn log2 n + an, for n a power of 2

Khloud Shawagfeh

• * Verify the guess for small numbers: b 1 log2 1 + a 1 = a. b 2 log2 2 + a 2 = 2b + 2a. b 4 log2 4 + a 4 = 8b + 4a. b 8 log2 8 + a 8 = 24b + 8a. b 16 log2 16 + a 16 = 64b + 16a. b 32 log2 32 + a 32 = 160b + 32a.

A Proof by Induction For n=1, T(1)=a, that is T(n)=b.nlog2 n + a.n So, for n=1, it is true. Let us assume that it is true for n=k/2, that is, T(k/2)=b.(k/2) log2 (k/2) + a.(k/2) Now, if we can prove that it is true for n=k also, then it can be said that it is true for all n.

For n=k, T(k) = 2T(k/2) + bk = 2(b(k/2) log2(k/2) + a(k/2)) + bk = bk(log2 k 1) + ak + bk = bk log2 k + ak

Hence T(n) = b.n log2 n + a.n, for all integer n

Khloud Shawagfeh

• *Recurrence -3 Problem: Let us determine an upper bound on the following recurrence:T(1) = a.T(n) = T(n/2) + b.For some constants a, b (independent of n).

Solution: Compute the solution for small powers of 2: T(2) = T(1) + b = b + a. T(4) = T(2) + b = 2b + a. T(8) = T(4) + b = 3b + a. T(16) = T(8) + b = 4b + a. T(32) = T(16) + b = 5b + a.

Guessing the Solution Guess T(n) = b log2 n + a, for n a power of 2

Khloud Shawagfeh

• * Verify the guess for small numbers: b log2 1 + a = a. b log2 2 + a = b + a. b log2 4 + a = 2b + a. b log2 8 + a = 3b + a. b log2 16 + a = 4b + a. b log2 32 + a = 5b + a.

A Proof by Induction For n=1, T(1)=a, that is T(n)=b log2 n + aSo, for n=1, it is true.Let us assume that it is true for n=k/2, that is, T(k/2)=b log2 (k/2) + aNow, if we can prove that it is true for n=k also, then it can be said that it is true for all n. For n=k,T(k) = T(k/2) + b = (b log2(k/2) + a) + b = b(log2 k 1) + a + b = b log2 k + a

Hence T(n) = b log2 n + a, for all integer n

Khloud Shawagfeh

• T(n) = 2T(n/2) + n = O(n lg n)Thus, we need to show that T(n) c n lg n with an appropriate choice of cInductive hypothesis: assume T(n/2) c (n/2) lg (n/2)Substitute back into recurrence to show that T(n) c n lg n follows, when c 1T(n) = 2 T(n/2) + n 2 (c (n/2) lg (n/2)) + n = cn lg(n/2) + n= cn lg n cn lg 2 + n= cn lg n cn + n cn lg n for c 1= O(n lg n)for c 1Recurrence -4

Khloud Shawagfeh

• Iteration MethodIteration method:

Expand the recurrence k times

Work some algebra to express as a summation

Evaluate the summation

Khloud Shawagfeh

• Iteration Method ExampleT(n) = n + 2T(n/2)T(n) = n + 2T(n/2) = n + 2(n/2 + 2T(n/4)) = n + n + 4T(n/4) = n + n + 4(n/4 + 2T(n/8)) = n + n + n + 8T(n/8) = in + 2iT(n/2i) = kn + 2kT(1) = nlgn + nT(1) = (nlgn)Assume: n = 2kn = 2kTaking lg on both sideslg n = lg (2k )lg n = klg2lg n = k x 1lg n = k

Khloud Shawagfeh

• T(n)= c + T(n-1)=c + c + T(n-2)=2c + T(n-2)=2c + c + T(n-3)=3c + T(n-3)kc + T(n-k) = ck + T(n-k)So far for n k we have T(n) = ck + T(n-k)

To stop the recursion, we should have n - k = 0 k = nT(n) = cn + T(0) = cn

Thus in general T(n) = O(n)

Khloud Shawagfeh

• T(n) = n + T(n-1) = n + n-1 + T(n-2)= n + n-1 + n-2 + T(n-3)= n + n-1 + n-2 + n-3 + T(n-4)= = n + n-1 + n-2 + n-3 + + (n-k+1) + T(n-k)=for n k

To stop the recursion, we should have n - k = 0 k = n

Khloud Shawagfeh

• The Master MethodBased on the Master theorem.T(n) = aT(n/b) + f(n)a 1, b > 1 are constants.f(n) is asymptotically positive.

Requires memorization of three cases.

Khloud Shawagfeh

• The Master TheoremTheorem:Let a 1 and b > 1 be constants, let f(n) be a function, and Let T(n) be defined on nonnegative integers by the recurrence T(n) = aT(n/b) + f(n), where we can replace n/b by n/b or n/b. T(n) can be bounded asymptotically in three cases:If f(n) = O(nlogba) for some constant > 0, then T(n) = (nlogba).If f(n) = (nlogba), then T(n) = (nlogbalg n).If f(n) = (nlogba+) for some constant > 0, and if, for some constant c < 1 and all sufficiently large n, we have af(n/b) c f(n), then T(n) = (f(n)).

Khloud Shawagfeh

• The Master TheoremGiven: a divide and conquer algorithm

An algorithm that divides the problem of size n into a subproblems, each of size n/b

Let the cost of each stage (i.e., the work to divide the problem + combine solved subproblems) be described by the function f(n)

Khloud Shawagfeh

• The Master Theoremif T(n) = aT(n/b) + f(n) where a 1 & b > 1then

Khloud Shawagfeh

• Understanding Master TheoremIn each of the three cases, we are comparing f(n) with nlogba , the solution to the recurrence is determined by the larger of the two functions.

In case 1, if the function nlogba is the larger, then the solution T(n) = (nlogba).

In case 3, if the function f(n) is the larger, then the solution is T(n) = (f(n)).

In case 2, if the two functions are the same size, then the ##### Introduction to Algorithms · PDF file 2019-09-12 · Introduction to Algorithms 6.046J/18.401J LECTURE 2 Asymptotic Notation • O-, Ω-, and Θ-notation Recurrences • Substitution
Documents ##### 1 Recurrences Algorithms Jay Urbain, PhD [email protected] Credits: Discrete Mathematics and Its Applications, by Kenneth Rosen The Design and Analysis of
Documents ##### Parallel Algorithms Examples Examples Concepts & Definitions Concepts & Definitions Analysis of Algorithms Analysis of Algorithms
Documents ##### Algorithms and Data Structures - ece250/materials/notes/Lecture5-Solving... · PDF file Algorithms and Data Structures Solving Recurrences (Master Theorem – Case1) Materials from
Documents ##### ADA Lecture Note Updated - VSSUT Lecture 1 - Introduction to Design and analysis of algorithms Lecture 2 - Growth of Functions ( Asymptotic notations) Lecture 3 - Recurrences, Solution
Documents ##### Analysis of Multithreaded ... Analysis of Multithreaded Algorithms 2009-10-29 OUTLINE 6.172 Performance Engineering of Software Systems Lecture 13 •DivideDivide -&& -Conquer Recurrences
Documents ##### Introduction to · PDF file Day 3 Introduction to Algorithms L2.2 Solving recurrences • The analysis of merge sort from Lecture 1 required us to solve a recurrence. • Recurrences
Documents ##### Analysis of Algorithms CS 477/677 Recurrences Instructor: George Bebis (Appendix A, Chapter 4)
Documents ##### Recurrences Part 3. Recursive Algorithms Recurrences are useful for analyzing recursive algorithms Recurrence â€“ an equation or inequality that describes
Documents ##### CSC 413/513: Intro to Algorithms Solving Recurrences Continued The Master Theorem Introduction to heapsort
Documents ##### CSI 5163 (95.573) ALGORITHM ANALYSIS AND zaguia/csi5163/W14-Intro-CSI5163.pdf Solving recurrences Dr. Nejib Zaguia CSI5163-W14 21 Review: Solving Recurrences The expression: There
Documents ##### Independent Component Analysis: Algorithms and .Independent Component Analysis: Algorithms and Applications
Documents ##### CSE 2331 / 5331 Foundations II: Data Structures and Algorithms Topic 2: -- Analyzing algorithms -- Solving recurrences
Documents ##### Pathogenesis, prevention of recurrences and outcome of ... PATHOGENESIS, PREVENTION OF RECURRENCES
Documents ##### Asymptotic Notation, Recurrences: Substitution, sourav/Lecture-02.pdf Asymptotic Notation, Recurrences: Substitution, Iteration, Master Method Lecture 2 L2.2 Solving recurrences •The
Documents ##### CS 253: Algorithms Chapter 4 Divide-and-Conquer Recurrences Master Theorem Credit: Dr. George Bebis
Documents Documents