CSE 2331 / 5331

Foundations II: Data Structures and Algorithms

Topic 2: -- Analyzing algorithms -- Solving recurrences

CSE 2331 / 5331

Example: for Loop (1)

Textbook A.1 for summation formulas

What if i (or j) starts from 1000 ?

What if i starts from n/2 ?

What if j ends with i2 ?

What if j ends with ?

CSE 2331 / 5331

Example: for Loop (2)

What if we change the first line to n > 100000 ?

CSE 2331 / 5331

Evaluating Summations

Evaluate No fixed technique Be familiar with some basics

Textbook Appendix A.1and A.2

CSE 2331 / 5331

Two Useful Techniques

Splitting summations E.g, ,

Using integral: If f is monotonically increasing, E.g,

CSE 2331 / 5331

Example: while Loop

What if we change the to ?

What if we change the to ?

CSE 2331 / 5331

Analysis of while Loop

Stops when , that is, after iterations.

IChanges to since for any

constant c.

CSE 2331 / 5331

More Examples (0)

What if we change the to ?

CSE 2331 / 5331

More Examples (1)

CSE 2331 / 5331

More Examples (2)

What if we change the to ?

CSE 2331 / 5331

More Examples (3)

CSE 2331 / 5331

Recursive Algorithms

CSE 2331 / 5331

Sorting Revisited (1)

CSE 2331 / 5331

T(n): worst case time complexity for any input of size n

Recurrence: T(n) = T(n-1) + cn

Solving recurrence: T(n) = T(n-1) + cn

= T(n-2) + c(n-1) + cn = T(n-k) + c(n-k+1) + .. + cn = T(1) + c[2 + … + n-1 + n]= Θ (n2)

Expansion into a series.

Usual assumption: T(n) is constant for small n, say n = 1, 2, 3

CSE 2331 / 5331

More examples (1)

CSE 2331 / 5331

More Examples (2)

CSE 2331 / 5331

Sorting Revisited (2)

Can we do better than ? Yes. Many ways.

We will talk about Merge-sort

CSE 2331 / 5331

Merge Sort

Divide

Conquer

?

?

Conquer Step Merge(B, C)

Input: Given two sorted arrays B and C Output: Merge into a single sorted array

CSE 2331 / 5331

16

10

8

3

7

5

4

2

22 3 4 5 7 8 10 16 2 32 3 4 2 3 4 5 2 3 4 5 7

If size of B and C are s and t:

Running time: O ( s + t )

CSE 2331 / 5331

Pseudocode

MergeSort ( A, r, s ) if ( r ≥ s) return; m = (r+s) / 2; A1 = MergeSort ( A, r, m ); A2 = MergeSort ( A, m+1, s ); Merge (A1, A2);

Recurrence relation for MergeSort(A, 1, n) T(n) = 2T(n/2) + cn

CSE 2331 / 5331

Solve Recurrence

Expansion into a series Recursion tree Substitution method

CSE 2331 / 5331

Expansion

T(n) = 2 T(n/2) + n = 2 (2 T (n/4) + n/2) + n = 4 T(n/4) + n + n = 4 (2 T(n/8) + n/8)) + n + n

……. = 2k T(n/2k) + n + … + n

Let . Then:

CSE 2331/5331

Recursion-tree Method

Solve T(n) = 2 T(n/2) + n

T(n)n

n/2 n/2

T(n/4) T(n/4) T(n/4) T(n/4)

n

n/2 n/2

n/4 n/4 n/4 n/4

(1) (1)

nn/2+n/2

4 (n/4)

n (1)

T(n) ≤ n height(tree) = n lg n

n

T(n/2) T(n/2)T(n) =

n

T(n/2) T(n/2)

CSE 2331 / 5331

Another MergeSortMergeSort ( A, r, s )

if ( r ≥ s) return; m1 = r+(s-r) / 3; m2 = r+2(s-r) / 3; A1 = MergeSort ( A, r, m1 ); A2 = MergeSort ( A, m1 +1, m2 ); A3 = MergeSort ( A, m2 +1, s ); Merge (A1, A2, A3);

Recurrence relation for MergeSort(A,1,n) T(n) = 3T(n/3) + cn

What if we change 3 to k ?

CSE 2331 / 5331

Another MergeSort

MergeSort ( A, r, s ) if ( r ≥ s) return; m = r+(s-r) / 3; A1 = MergeSort ( A, r, m ); A2 = MergeSort ( A, m+1, s ); Merge (A1, A2);

Recurrence relation for MergeSort(A,1,n) T(n) = T(n/3) + T(2n/3) + cn

CSE 2331 / 5331

Recursion Tree full levels and total

levels Total cost of each

level is at most cn Upper bound Lower bound

CSE 2331 / 5331

More Examples of Recurrences (1)

T(n) = T(n-10) + c ?

T(n) = T(n-10) + n2 + 3n ?

CSE 2331 / 5331

More Examples (2)

?

CSE 2331 / 5331

More Examples (3)

Sometimes, it is hard to get tight bound

Easy to prove that and

CSE 2331 / 5331

Summary

No good formula for solving recurrences Practice!

Expansion and recursion tree are two most intuitive These two are quite similar

Substitution method more rigorous and powerful But require good intuition and use of induction

Other methods: Master Theorem (not covered)