Recurrences

Recurrences

Recurrence

A recurrence is an equality or inequality on a function thatuses values of the function on smaller inputs.

Examples:

T (n) =

1 if n = 1

T (n − 1) + 1 if n > 1

T (n) =

1 if n = 1

2T (bn/2c) + n if n > 1

T (n) =

0 if n ≤ 2

T (b√nc) + 1 if n > 2

T (n) =

1 if n = 1

T (b2n/3c) + T (bn/3c) + 1 if n > 1

Recurrences

Methods for solving recurrences

The substitution method.

The iteration method.

The master method.

Recurrences

The substitution method

In the substitution method, we guess the answer to therecurrence and then prove the correctness using induction.

Recurrences

Example 1: Guessing the exact solution

Recurrence T (n) =

1 if n = 1

2T (n/2) + n if n > 1

Guess T (n) = n log n + n.

Base For n = 1, n log n + n = 1 = T (1).

Induction step Assume T (m) = m logm + m for all m < n.

T (n) = 2T (n/2) + n

= 2(n

2log

n

2+

n

2

)+ n by the induction hypothesis

= n logn

2+ n + n

= n log n − n log 2 + n + n

= n log n − n + n + n

= n log n + n

Recurrences

Example 2: Guessing upper and lower bounds

Recurrence T (n) =

1 if n = 1

2T (n/2) + n if n > 1

Guess T (n) ≤ cn log n for n ≥ 2.

Base T (2) = 2T (1)+2 = 4 ≤ c ·2 log 2 = 2c ⇒ c ≥ 2.

Induction step Assume T (m) ≤ cm logm for all 2 ≤ m < n.

T (n) = 2T (n/2) + n

≤ 2(cn

2log

n

2

)+ n

= cn logn

2+ n

= cn log n − cn + n

≤ cn log n if −cn + n ≤ 0⇒ c ≥ 1

⇒ T (n) ∈ O(n log n).Recurrences

Lower bound

Guess T (n) ≥ dn log n for all n.

Base T (1) = 1 ≥ d · 1 log 1 = 0.

Induction step Assume T (m) ≥ dm logm for all 2 ≤ m < n.

T (n) = 2T (n/2) + n

≥ 2(dn

2log

n

2

)+ n

= dn logn

2+ n

= dn log n − dn + n

≥ dn log n if −dn + n ≤ 0⇒ d ≤ 1

⇒ T (n) ∈ Ω(n log n).SinceT (n) ∈ O(n log n), we conclude that T (n) ∈ Θ(n log n).

Recurrences

Example 3: Revising the guess

Recurrence T (n) = 8T (n/2) + n2.

Guess T (n) ≤ cn3.

Induction step

T (n) = 8T (n/2) + n2

≤ 8c(n

2

)3+ n2

= 8c · n3

8+ n2

= cn3 + n2

6≤ cn3

Recurrences

Example 3: Revising the guess

Sometimes the guess needs to be revised by subtracting alower-order term from the previous guess.

Guess T (n) ≤ cn3 − dn2.

Induction step

T (n) = 8T (n/2) + n2

≤ 8

(c(n

2

)3− d

(n2

)2)+ n2

= 8c · n3

8− 8d · n

2

4+ n2

= cn3 − 2dn2 + n2

= cn3 − dn2 − dn2 + n2

≤ cn3 − dn2 if −dn2 + n2 ≤ 0⇒ d ≥ 1

Recurrences

The iteration method

Expand the recurrence into a summation.

Evaluate the summation.

Recurrences

Example 1

Recurrence T (n) =

0 if n = 0

T (n − 1) + 1 if n > 0

Iteration

T (n) = T (n − 1) + 1

=(T (n − 2) + 1

)+ 1

= T (n − 3) + 3

...

= T (0) + n

= n

Recurrences

Example 2

Recurrence T (n) =

1 if n = 1

T (n − 1) + n if n > 1

Iteration

T (n) = T (n − 1) + n

= T (n − 2) + (n − 1) + n

= T (n − 3) + (n − 2) + (n − 1) + n

...

= 1 + 2 + · · ·+ n

=n(n + 1)

2

Recurrences

Changing free terms

If we are only interested in the asymptotic behavior, the freeterms in the recurrence can be simplified.

T (n) =

1 if n = 1

T (n − 1) + n if n > 1

T2(n) =

3 if n = 1

T2(n − 1) + 3n if n > 1

T3(n) =

1 if n = 1

T3(n − 1) + n + 2√n if n > 1

T2(n) = 3T (n) for all n (T2(n) = 3 + 6 + · · ·+ 3n).

T (n) ≤ T3(n) ≤ 3T (n) for all n.

Recurrences

Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1

log(3/2)· n log n

⇒ T (n) ∈ Θ(n log n).

Recurrences

Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1

log(3/2)· n log n

⇒ T (n) ∈ Θ(n log n).

Recurrences

Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1

log(3/2)· n log n

⇒ T (n) ∈ Θ(n log n).

Recurrences

Recurrence tree

T (n) = T (n/3) + T (2n/3) + n.

1log 3· n log n = n · log3 n ≤ T (n) ≤ n · log3/2 n = 1

log(3/2)· n log n

⇒ T (n) ∈ Θ(n log n).

Recurrences

The master method

A method for solving recurrences of the form

T (n) = aT (n/b) + f (n)

where a ≥ 1, b > 1, and f (n) is an asymptotic positivefunction.

Recurrences

The master theorem

Let T (n) = aT (n/b) + f (n) where a ≥ 1, b > 1 and f (n) isasymptotically positive.

1 If f (n) ∈ O(nlogb a−ε) for some constant ε > 0 thenT (n) ∈ Θ(nlogb a).

2 If f (n) ∈ Θ(nlogb a) then T (n) ∈ Θ(f (n) · log n).

3 f (n) ∈ Ω(nlogb a+ε) for some constant ε > 0 and ifaf (n/b) ≤ cf (n) for some constant c < 1 and allsufficiently large n, then T (n) ∈ Θ(f (n)).

T (n/b) means either bn/bc or dn/be.Case 2 can be generalized: If f (n) ∈ Θ(nlogb a logk n) forsome k ≥ 0 then T (n) ∈ Θ(f (n) · logk+1 n).

Recurrences

Example 1 (case 1)

T (n) = 9T (n/3) + n.

We have a = 9, b = 3, f (n) = n.

logb a = log3 9 = 2.

All the conditions of the first case are satisfied:

a = 9 ≥ 1, b = 3 > 1, f (n) = n ≥ 0.For ε = 1/2, f (n) ∈ O(nlogb a−ε) = O(n3/2).

Hence, T (n) ∈ Θ(nlogb a) = Θ(n2).

Recurrences

Example 2 (case 1)

T (n) = 5T (n/2) + n2.

We have a = 5, b = 2, f (n) = n2.

logb a = log2 5 ≈ 2.322.

All the conditions of the first case are satisfied:

a = 5 ≥ 1, b = 2 > 1, f (n) = n2 ≥ 0.For ε = 0.3, f (n) ∈ O(nlogb a−ε) = O(n2.022).

Hence, T (n) ∈ Θ(nlogb a) = Θ(nlog2 5).

Recurrences

Example 3 (case 2)

T (n) = T (2n/3) + 1.

We have a = 1, b = 3/2, f (n) = 1.

logb a = log3/2 1 = 0.

All the conditions of the second case are satisfied:

a = 1 ≥ 1, b = 3/2 > 1, f (n) = 1 ≥ 0.f (n) ∈ Θ(nlogb a) = Θ(1).

Hence, T (n) ∈ Θ(f (n) · log n) = Θ(log n).

Recurrences

Example 4 (case 3)

T (n) = 3T (n/4) + n log n.

We have a = 3, b = 4, f (n) = n log n.

logb a = log4 3 ≈ 0.792.

All the conditions of the third case are satisfied:

a = 3 ≥ 1, b = 4 > 1, f (n) = n log n ≥ 0.For ε = 0.1, f (n) ∈ Ω(nlogb a+ε) = Ω(n0.892).For all n, af (n/b) = 3n

4 log n4 ≤

34n log n = 3

4 f (n).

Hence, T (n) ∈ Θ(f (n)) = Θ(n log n).

Recurrences

Example 5

T (n) = 2T (n/2) + n/ log n.

We have a = 2, b = 2, f (n) = n/ log n.

logb a = log2 2 = 1.

n/ log n < n, but n/ log n /∈ O(nlogb a−ε) for every ε > 0.

Recurrences