Data Structures and Algorithms(CS210/ESO207/ESO211)

Lecture 15• Methods for solving the recurrences which occur

frequently in the analysis of algorithms.

1

Commonly occurring recurrences

• T(n) = T(n/2) + cn

• T(n) = T(7n/10) + T(n/5) + cn

• T(n) = 4T(n/2) +

• T(n) = 3T(n/2) +

• T(n) = T() + cn

2

Methods for solving Common Recurrences in algorithm analysis

3

Methods for solving common Recurrences

• Unfolding the recurrence.

• Guessing the solution and then proving by induction.

• A General solution for a large class of recurrences (Master theorem)

4

Solving a recurrence by unfolding

Let T(1) = 1, T(n) = cn + 4 T(n/2) for n>1, where c is some positive constant

Solving the recurrence for T(n) by unfolding (expanding) T(n) = cn + 4 T(n/2) = cn + 2cn + T(n/) = cn + 2cn + T(n/) = cn + 2cn + = cn + 2cn + = O()

5

A geometric increasing series with log n terms and common ratio 2

Solving a recurrence by guessing and then proving by induction

T(1) = T() = 2T(/2) + Guess: T() ≤ + for some constants and . Proof by induction: Base case: holds true if ≥ Induction hypothesis: T() ≤ + for all <To prove: T() ≤ + Proof: T() = 2T(/2) + ≤ 2( + ) + // by induction hypothesis

= - + 2 + = + + ( + - ) ≤ + if ≥ +

6Hence T() ≤ ( + for all value of .

So T() = )

These inequalities can be satisfied

simultaneously by selecting

= =+

It looks similar/identical to the recurrence of merge sort. So

we guess T(n) = O(n log n)

Solving a recurrence by guessing and then proving by induction

Key points:• You have to make a right guess (past experience may help)• What if your guess is too loose ?• Be careful in the proof by induction

– To realize the last point, find the error in the following reasoning.

For the recurrence T(1) = , and T() = 2T(/2) + ,one guesses T() = O() and provides the following (wrong)proof by induction. Induction hypothesis: T() ≤ + for all T() = 2T(/2) + ≤ 2( + ) + // by induction hypothesis

= + + = O()

7

A General Method for solving a large class of Recurrences

8

Solving a large class of recurrences

T(1) = 1,T(n) = f(n) + a T(n/b)Where • a and b are constants and b >1 • f(n) is a multiplicative function: f(xy) = f(x)f(y) Example of a multiplicative function : f(n) =

AIM : To solve T(n) for n =

9

Solving a slightly general class of recurrences

T(n) = f(n) + T(n/) = f(n) + f(n/) + T(n/) = f(n) + f(n/) + f(n/) + T(n/) = … = f(n) + f(n/) + … + f(n/) + … + f(n/) +T(1)

= ( ) + … after rearranging … = + … continued to the next page …

10

T(n) = + (since is multiplicative) = + = + = + = +

= + Case 1: = , T(n) = ??

Case 2: < , T(n) = ??

11

A geometric series

𝑎𝑘(𝑘+1)

+O(1)

¿O(𝑛log𝑏𝑎 log𝑏𝑛 )

=  O ( f ( n ))

For < , the sum of this series is bounded by

= O(1)

= O)

Three cases

T(n) = +

Case 1: = , T(n) =

Case 2: < , T(n) =

Case 3: > , T(n) = ??

12

+ = )

For > , the sum of this series is equal to

Master theorem

T(1) = 1,T(n) = f(n) + a T(n/b) where f is multiplicative.There are the following solutionsCase 1: = , T(n) = Case 2: < , T(n) = Case 3: > , T(n) = )

13

Examples

14

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) =Case 2: < , T(n) = Case 3: > , T(n) = )

Example 1: T()= + 4 T(/2)

Solution: T()= ??

15

O(𝑛2)

This is case 3

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) =Case 2: < , T(n) = Case 3: > , T(n) = ) Example 2: T()= + 4 T(/2)

Solution: T()= ??

16

 O(𝑛2 log4𝑛 )

This is case 1

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) =Case 2: < , T(n) = Case 3: > , T(n) = )

Example 3: T()= + 4 T(/2)

Solution: T()= ??

17

 O(𝑛3 )

This is case 2

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) =Case 2: < , T(n) = Case 3: > , T(n) = )

Example 4: T()= 2 + 3 T(/2)

Solution: T()= ??

18

We can not apply master theorem directly since f(n) = 2 is not multiplicative. But if we define G(n)= T(n)/2, then we get

G(n) = + 3 T(/2). Now we can apply master theorem (case 3) to get G(n) = . Hence T(n) = .

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) = Case 2: < , T(n) = Case 3: > , T(n) = )

Example 6: T() = T() + c

Solution: T()= ??

19

We can not apply master theorem directly since T() <> T(/) for any constant .

Solving T() = T() + c using the method of unfolding

T() = c + T() = c + c+ T() = c + c+ c + T() = c + c+ … c + … + T(1)

= O()

20

A series which is decreasing at a rate faster than any geometric series

Can you guess the number of terms in

this series ?

⌈ log log𝑛⌉

Master theorem

If T(1) = 1, and T(n) = f(n) + a T(n/b) where f is multiplicative, then there are the following solutionsCase 1: = , T(n) =Case 2: < , T(n) = Case 3: > , T(n) = )

Example 5: T()= + 2 T(/2)

Solution: T()= ??

21

We can not apply master theorem since f(n) = is not multiplicative. Using the

method of “unfolding”, it can be shown that T(n) = .

HomeworkSolve the following recurrences systematically (if possible by various methods). Assume that T() =1 for all these recurrences. • T() = 1 + 2 T(/2)• T() = + 2 T(/2)• T() = + 7 T(/3)• T() = + 2T(/2)• T() = 1 + T(/5)• T() = + 2 T(/4)• T() = 1 + T()• T() = + T(/10)• T() = + T(/4)

22