ECE250: Algorithms and Data Structures

Solving Recurrences (Master Theorem – Case1)

Materials from CLRS: Chapter 4.4

Ladan Tahvildari, PEng, SMIEEE Professor

Software Technologies Applied Research (STAR) Group

Dept. of Elect. & Comp. Eng.

University of Waterloo

Acknowledgements

v The following resources have been used to prepare materials for this course: Ø  MIT OpenCourseWare Ø  Introduction To Algorithms (CLRS Book) Ø  Data Structures and Algorithm Analysis in C++ (M. Wiess) Ø  Data Structures and Algorithms in C++ (M. Goodrich)

v Thanks to many people for pointing out mistakes, providing suggestions, or helping to improve the quality of this course over the last ten years: Ø  http://www.stargroup.uwaterloo.ca/~ece250/acknowledgment/

Lecture 5 ECE250 2

Lecture 5 ECE250

Methods for Solving Recurrences

v Recursion-Tree Method

v Repeated Substitution

v Master Method

3

Analyzing Merge Sort

Lecture 5 ECE250

MERGE-SORT A[1 . . n] 1.  If n = 1, done. 2.  Recursively sort A[ 1 . . ⎡n/2⎤ ]

and A[ ⎡n/2⎤+1 . . n ] . 3. “Merge” the 2 sorted lists

T(n) Θ(1) 2T(n/2)

Θ(n)

Sloppiness: Should be T( ⎡n/2⎤ ) + T( ⎣n/2⎦ ) , but it turns out not to matter asymptotically.

4

Lecture 5 ECE250

Master Method

The master method applies to recurrences of the form

T(n) = a T(n/b) + f (n) , where a ≥ 1, b > 1, and f is asymptotically positive.

5

Lecture 5 ECE250

f (n/b)

Idea of Master Theorem

f (n/b) f (n/b)

Τ (1)

Recursion tree:

… f (n)

a

f (n/b2) f (n/b2) f (n/b2) … a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

#leaves = ah = alogbn = nlogba

nlogbaΤ (1)

6

Lecture 5 ECE250

Master Method Intuition

v Three common cases: Ø Running time dominated by cost at leaves Ø Running time evenly distributed throughout the tree Ø Running time dominated by cost at the root

v Consequently, to solve the recurrence, we need only to characterize the dominant term

v In each case compare with ( )f n log( )b aO n

7

Lecture 5 ECE250

Master Method Strategy

1.  Extract a, b, and f(n) from a given recurrence

2.  Determine

3.  Compare f(n) and asymptotically

4.  Determine appropriate Master Theorem Case, and apply it

logb an

logb an

8

Lecture 5 ECE250

Three Common Cases

Compare f (n) with nlogba:

1.  f (n) = O(nlogba – ε) for some constant ε > 0. • f (n) grows polynomially slower than nlogba

(by an nε factor). Solution: T(n) = Θ(nlogba) .

Running time dominated by cost at leaves

9

Lecture 5 ECE250

f (n/b)

Idea of Master Theorem

f (n/b) f (n/b)

Τ (1)

Recursion tree:

… f (n)

a

f (n/b2) f (n/b2) f (n/b2) … a h = logbn

f (n)

a f (n/b)

a2 f (n/b2)

…

nlogbaΤ (1) CASE 1: The weight increases geometrically from the root to the leaves. The leaves hold a constant fraction of the total weight. Θ(nlogba)

10

Lecture 5 ECE250

Example (1)

v  T(n) = 4T(n/2) + n a = 4, b = 2 ⇒ nlogba = n2; f (n) = n. CASE 1: f (n) = O(n2 – ε) for ε = 1. ∴ T(n) = Θ(n2).

11

Example (2)

Lecture 5 ECE250 12

Consider this recursive algorithm. The input A is an array of size n. The algorithm divides the incoming array into a number of sub-arrays and then recursively calls itself with one or more of the sub-arrays. Write down the recurrence function for the running time of Alg2 and solve it.

Alg2 (A[1..n] ) if ( n <= 0 ) return; q = n/4; if (q is an even number) Alg2 (A[1..q]); Alg2 (A[q+1..2q]); else Alg2 (A[2q+1..3q]); Alg2 (A[3q+1..n]); end end