04 Recurrences

8/2/2019 04 Recurrences

1/25

Huo Hongwei 1

Design and Analysis of Algorithms

Recurrences

Reference:CLRS Chapter 4

Topics:

Substitution method Recursion-tree method

Master method


2/25

Huo Hongwei 2

Solving recurrences

The analysis of Mergesort from Lecture 2 required us tosolve a recurrence.

Recurrences are a major tool for analysis of algorithms

Today: Learn a few methods.

Substitution method

Recursion- tree method

Master method

Divide and Conquer algorithms which are analyzable byrecurrences.


3/25

Huo Hongwei 3

Recall: Mergesort

otherwise2T(n/2) + (n) ifn = 1(1)T(n) =

MERGE-SORT(A,p,r)

1 if p < r2 then q (p+r)/23 MERGE-SORT (A, p, q)

4 MERGE-SORT (A, q+1, r)

5 MERGE (A, p, q, r)

MERGESORT


4/25

Huo Hongwei 4

Substitution method

The most general method:

Guess the form of the solution.

Verify by induction.

Solve for constants.

Ex. T(n) = 4T(n/2) + 100n

Assume that T(1) = (1). Guess O(n3). (Prove O and separately.) Assume that T(k) ck3 fork


5/25

Huo Hongwei 5

Example of substitution

T(n) = 4T(n/2) + 100n

4c(n/2)3 + 100n= (c/2)n3 + 100n

=cn3 ((c/2)n3 100n)

cn3 whenever (c/2)n3 100n 0, for example, ifc 200 andn 1.

desiredresidual

desired

residual


6/25

Huo Hongwei 6

Example (continued)

We must also handle the initial conditions/the boundaryconditions, that is, ground the induction with base cases.

Base: T(n) = (1) for alln


7/25

Huo Hongwei 7

A tighter upper bound?

We shall prove that T(n) = O(n2).

Assume that T(k) ck2 fork


8/25

Huo Hongwei 8

A tighter upper bound?

IDEA: Strengthen the induction hypothesis.

Subtract a low-order term.

Assume that T(k) c1k2 c2k fork 100.

Pickc1 big enough to handle the initial conditions.

subtleties


9/25

Huo Hongwei 9

Avoiding Pitfalls

Be careful not to misuse asymptotic notation. Forexample:

We can falsely prove T(n) = O(n) by guessing T(n) cnfor T(n) = 2T(n/2) + n

T(n) 2c n/2 + n cn + n= O(n) Wrong!

The error is that we havent proved the exact form ofthe inductive hypothesis T(n) cn.


10/25

Huo Hongwei 10

Changing Variables

Use algebraic manipulation to make an unknownrecurrence similar to what you have seen before.

Consider T(n) = 2T(n1/2) + lgn , Rename m = lgn and we have

T(2m) = 2T(2m/2) +m .

Set S(m) = T(2m) and we have

S(m) = 2S(m/2) + m S(m) = O(m lgm) . Changing back from S(m) to T(n), we have

T(n) = T(2m) = S(m) = O(m lgm) = O(lgn lg lgn) .


11/25

Huo Hongwei 11

Recursion- tree method

A recursion tree models the costs (time) of a recursiveexecution of an algorithm.

The recursion tree method is good for generatingguesses for the substitution method.

The recursion-tree method can be unreliable, just like anymethod that uses ellipses ().

The recursion-tree method promotes intuition, however.


12/25

Huo Hongwei 12

The Construction of a Recursion Tree

Solve T(n) = 3T(n/4) + (n2), we haveT(n)


13/25

Huo Hongwei 13


Solve T(n) = 3T(n/4) + (n2), we havecn2

n4

T( ) n4

T( )n4

T( )


14/25

Huo Hongwei 14


Solve T(n) = 3T(n/4) + (n2), we havecn2

n4

c( )2n4

c( )2n4

c( )2

n16T( ) n16T( )n16T( ) n16T( ) n16T( )n16T( )n16T( ) n16T( )n16T( )


15/25

Huo Hongwei 15

Construction of recursion tree

The fully expanded tree has lg4n +1 levels, i.e., it has height lg4n.

n /4k

cn2

c(n/4)2

T(1) T(1) T(1) T(1) T(1) T(1) T(1)

cn2

3/16cn2

(3/16)2cn2

Total: O(n2)

c(n/4)2 c(n/4)2

c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2 c(n/16)2

T(1) T(1) T(1) (n )log43nlog43

lg4n

geometric series


16/25

Huo Hongwei 16

Master Method

It provides a cookbook method for solving recurrencesof the form:

T(n) = a T(n/b) + f(n)

wherea 1 andb > 1 are constants andf(n) is anasymptotically positive function


17/25

Huo Hongwei 17

Idea of master theorem

Recursion tree

f(n/b)f(n/b) f(n/b)

(1)

f(n)a

f(n/b

2

)f(n/b2

) f(n/b

2

)

ah = logbn

f(n)

a f(n/b)

a2f(n/b2)

#leaves =ah

=alogbn

=nlogba

nlogba (1)


18/25

Huo Hongwei 18

Three common cases

Comparef(n) withnlogba:

1.f(n) = O(nlogba-) for some constant > 0.f(n) grows polynomially slower thannlogba (by ann

factor).

Solution: T(n)= (nlogba).


19/25

Huo Hongwei 19


Recursion tree

f(n/b)f(n/b) f(n/b)

(1)

f(n)a

f(n/b2)f(n/b2) f(n/b2)

ah = logbn

f(n)

a f(n/b)

a2f(n/b2)

CASE 1: The weight increases geometrically from

the root to the leaves. The leaves hold a

constant fraction of the total weight.

nlogba (1)(nlogba)


20/25

Huo Hongwei 20

Three common cases


2.f(n)= (nlogba lgkn) for some constantk 0. f(n) andnlogba grow at similar rates.

Solution: T(n)= (nlogba lgk+1n).


21/25

Huo Hongwei 21


Recursion tree

f(n/b)f(n/b) f(n/b)

(1)

f(n)a


ah = logbn

f(n)

a f(n/b)

a2f(n/b2)

CASE 2: (k = 0)The weight is approximately the

same on each of the logbn levels.

nlogba (1)(nlogba lgn)


22/25

Huo Hongwei 22

Three common cases


3.f(n)= (nlogba+) for some constant > 0. f(n) grows polynomially faster thannlogba (by ann

factor),

andf(n) satisfies the regularity condition that

a f(n/b) c f(n) for some constantc < 1. Solution: T(n)= (f(n)).


23/25

Huo Hongwei 23


Recursion tree

f(n/b)f(n/b) f(n/b)

(1)

f(n)a


ah = logbn

f(n)

a f(n/b)

a2f(n/b2)

CASE 3: The weight decreases geometrically from

the root to the leaves. The root holds a

constant fraction of the total weight.

nlogba (1)(f(n))


24/25

Huo Hongwei 24

Examples

T(n) = 4T(n/2) + n

a = 4, b = 2, nlogba = n2 ;f(n) = n. CASE 1:f(n) = O(n2- ) for =1. T(n) = (n2)

T(n) = 4T(n/2) + n2

a = 4, b = 2, nlogba = n2 ;f(n) = n2. CASE 2:f(n) = (n2 lg0n), that is,k = 0. T(n) = (n2 lgn)


25/25

Huo Hongwei 25

Examples

T(n) = 4T(n/2) + n3

a = 4, b = 2, nlogba = n2 ;f(n) = n3. CASE 3:f(n) = (n2+ ), for = 1 and 4(cn/2)3 cn3 (regular

cond.) forc = 1/2.

T(n) = (n3) T(n) = 4T(n/2) + n2/lgn

a = 4, b = 2, nlogba = n2 ;f(n) = n3/lgn. Master method does not apply. In particular, for every

constant > 0, we haven = (lgn).

Documents

04 Recurrences