Chapter 4.Chapter 4.

RecurrencesRecurrences

OutlineOutlineOffers Offers three methodsthree methods for solving recurrences, that for solving recurrences, that is for obtaining asymptotic bounds on the solutionis for obtaining asymptotic bounds on the solutionIn the In the substitution methodsubstitution method, we guess a bound and , we guess a bound and then use mathematical induction to prove our then use mathematical induction to prove our guess correctguess correctThe The recursion-tree methodrecursion-tree method converts the recurrence converts the recurrence into a tree whose nodes represent the costs into a tree whose nodes represent the costs incurred at various levelsincurred at various levelsThe The master methodmaster method provides bounds for the provides bounds for the recurrence of the form recurrence of the form ( ) ( / ) ( )

1, 1

T n aT n b f n

a b

TechnicalitiesTechnicalities

Assumption of Assumption of integer argumentsinteger arguments to functions to functionsBoundary conditionsBoundary conditions we usually we usually ignoreignore– The recurrences that arises from the running time of The recurrences that arises from the running time of

algorithms generally have algorithms generally have T(n)= T(n)= (1)(1) for for sufficiently small nsufficiently small n– Changing the value of Changing the value of T(1),T(1), the solution typically does not the solution typically does not

change by more than a change by more than a constant factorconstant factor, so the , so the order of order of growthgrowth is unchanged. is unchanged.

Often omit Often omit floors, ceilings and boundary conditionsfloors, ceilings and boundary conditions– We forge ahead without these details and later determine We forge ahead without these details and later determine

whether or not they matter and it is whether or not they matter and it is important to know when important to know when they do matterthey do matter..

When an algorithm contains a When an algorithm contains a recursive call to itselfrecursive call to itself, , its running time can often be described by a its running time can often be described by a recurrencerecurrenceA A recurrencerecurrence is a function ( is a function (equation or in-equalityequation or in-equality) ) defined in terms ofdefined in terms of– one or more base cases, --boundary conditionone or more base cases, --boundary condition– itself, with smaller argumentsitself, with smaller arguments..

Example:Example:– T T (n) (n) = = 1 1 if if n n = 1 = 1 ,,

T (n-T (n-11) ) +1 +1 if if n > n > 1 1 ..SolutionSolution: : T (n) T (n) = = nn..

– T (n) T (n) = = 1 1 if if n n = 1 = 1 ,,22T (n/T (n/22) ) + + n n if if n n ≥ 1 ≥ 1 ..

SolutionSolution: : T (n) T (n) = = n n lg lg n n + + nn

RecurrenceRecurrence

SummarySummary

In algorithm analysis, we usually express both the In algorithm analysis, we usually express both the recurrence and its solution using recurrence and its solution using asymptotic notationasymptotic notationExample: T(n)=2T(n/2)+Example: T(n)=2T(n/2)+(n), with solution T(n)= (n), with solution T(n)= (nlgn)(nlgn)The boundary conditions are usually expressed as “ The boundary conditions are usually expressed as “ T(n)=1 for sufficiently small n” When we desire an When we desire an exact rather than asymptoticexact rather than asymptotic solution, we need to deal with boundary conditionsolution, we need to deal with boundary conditionIn practice, we just use asymptotic notations most of In practice, we just use asymptotic notations most of the time and we ignore boundary conditionsthe time and we ignore boundary conditions

Substitution MethodSubstitution Method

The substitution method entails two stepsThe substitution method entails two steps– Guess the form of the solutionGuess the form of the solution– Use Use mathematical induction mathematical induction to find the constants and to find the constants and

show that the solution worksshow that the solution works

The method is powerful but only applied to cases The method is powerful but only applied to cases when it is easy to guess the form of he answerwhen it is easy to guess the form of he answer

The method can be used to establish either upper The method can be used to establish either upper or lower bound on a recurrence.or lower bound on a recurrence.

The Substitution methodThe Substitution methodT(n) = 2T(n/2) + n T(n) = 2T(n/2) + n

GuessGuess:: T(n) = O(n lg n)T(n) = O(n lg n)Proof: Proof:

Prove that Prove that T(n) T(n) c n lg n c n lg n for for c>0 c>0Assume T(n/2)Assume T(n/2) c(n/2)lg(n/2) for some positive constant c(n/2)lg(n/2) for some positive constant cc

then then T(n)T(n) 2(c2(c n/2 n/2 lg n/2) + n lg n/2) + n = = cn lg n – cn + ncn lg n – cn + n cn lg ncn lg n if if cc 1 1

ThereforeTherefore, , T(n) = O(n lg n)T(n) = O(n lg n)

( ) ( lg )

( ) ( lg )

T n O n n

T n n n

( ) ( lg )T n n n

SubtletiesSubtleties There are times when can correctly guess an asymptotic There are times when can correctly guess an asymptotic bound on the solution of a recurrence, but somehow the math bound on the solution of a recurrence, but somehow the math doesn’t seem to work out in the inductiondoesn’t seem to work out in the induction

Revising the guess by subtracting a lower-order term often Revising the guess by subtracting a lower-order term often permits the math to go throughpermits the math to go through

Several PointsSeveral Points

Making good guessMaking good guess

Avoiding pitfallsAvoiding pitfalls

Changing variablesChanging variables

/2

( ) 2 lg

2 lg

(2 ) 2 (2 )

( ) (2 ) ( ) 2 ( / 2)

( ) ( lg )

( ) ( ) (lg lg lg )

m

m m

m

T n T n n

n m n

T T m

S m T S m S m m

S m O m m

T n S m O n n

The Recursion-Tree The Recursion-Tree methodmethod

Each code represents the cost of a single sub-problem Each code represents the cost of a single sub-problem somewhere in the set of recursive function invocations.somewhere in the set of recursive function invocations.SumSum the costs within each level of the tree to obtain a the costs within each level of the tree to obtain a set of per-level costs.set of per-level costs.Then sum all the per-level costs to determine the total Then sum all the per-level costs to determine the total cost of all levels of the recursion.cost of all levels of the recursion.It’s useful to solve the recurrence which describes the It’s useful to solve the recurrence which describes the running time of a running time of a divide-and-conquer algorithmdivide-and-conquer algorithm..It’s used to It’s used to generate a good guessgenerate a good guess which is then which is then verified by the substitution method.verified by the substitution method.A careful drawing of a recursion tree and summing the A careful drawing of a recursion tree and summing the costs can be used as a costs can be used as a direct proofdirect proof of a solution to a of a solution to a recurrence.recurrence.

ExamplesExamples

T(n)=3T(n/4)=cnT(n)=3T(n/4)=cn22

– Substitution MethodSubstitution Method– Recursion Tree Recursion Tree

T(n)=T(n/3)+T(2n/3)+O(n)T(n)=T(n/3)+T(2n/3)+O(n)

#nodes at ith level: 3i

Size of sub-problem at ith : n/4i

=>total level: log4n

ValidationValidation

Another ExampleAnother Example

Use Substitution Method to Use Substitution Method to verifyverify

ExampleExample1.1. T (n) T (n) = 5= 5T (n/T (n/22) ) + + (n(n²²))

vs. vs. n²n²

sol.) Since sol.) Since log2 5 - log2 5 - = 2 = 2 for some constant for some constant > > 0, use Case 1 ⇒0, use Case 1 ⇒ ..

22.. T (n) T (n) = 5= 5T (n/T (n/22) ) + + (n³)(n³)

vs. vs. n³n³

sol.) sol.) Now lg 5 + Now lg 5 + = 3 for some constant = 3 for some constant > > 00

Use Case 3 ⇒Use Case 3 ⇒ T (n) T (n) = = (n³)(n³)

3. T 3. T (n) = (n) = 2727T (n/T (n/33) + ) + (n³/ (n³/ lg lg n)n)

vs. vs. for any for any k k ≥ ≥ 0.0.

sol) sol) Cannot use the master method.Cannot use the master method.

HomeworkHomework

4.1-3, 4.1-54.1-3, 4.1-5

4.2-24.2-2

4.3-44.3-4

Class ExerciseClass Exercise

Pp.85 Problem 4-1Pp.85 Problem 4-1

a, c, f, ha, c, f, h