CS325 Winter 2014: HW1

1. 0.2 from text book. Show that, if c is a positive real number, then g(n) =1 + c+ c2 + · · ·+ cn is

(a) Θ(1) if c < 1

(b) Θ(n) if c = 1

(c) Θ(cn) if c > 1

The moral: in big-Θ terms, the sum of a geometric series is simply the firstterm if the series is strictly decreasing, the last term if the series is strictlyincreasing, of the number of terms if the series is unchanging. (Question0.2 from the text.)

Solution: First, we note that a geometric series has a closed-form solu-tion:

g(n) = 1 + c+ c2 + ...+ cn =cn+1 − 1

c− 1

If c < 1, we have:

limn→∞

g(n) =0− 1

c− 1=

1

1− c(since limn→∞ cn+1 = 0)

From this we conclude that g(n) = Θ(1) for c < 1.

If c = 1, we have g(n) = n+ 1 = Θ(n).

if c > 1, we have:

limn→∞

g(n)

cn= lim

n→∞

cn+1 − 1

cn+1 − cn

= limn→∞

c− 1cn

c− 1(divide by cn simultaneously top and bottom)

=c

c− 1(limn→∞

1cn = 0 for c > 1)

This allows us to conclude that g(n) = O(cn) for c > 1.

Similar we can show that

limn→∞

cn

g(n)=c− 1

c

This allows us to conclude that g(n) = Ω(cn) for c > 1. Thus, we haveg(n) = Θ(cn).

1

2. 0.3(a) from text book.

The Fibonacci numbers F0, F1, ..., are defined by the rule

F0 = 0, F1 = 1, Fn = Fn−1 + Fn−2

Use induction to prove that Fn ≥ 20.5n for n ≥ 6. (Question 0.3a fromthe text.)

Solution:

(Inductive assumption): Assuming that the statement is true for 6 ≤ k <n

(Base cases:)

if n = 6, we have F6 = 8 ≥ 26/2 = 8.

if n = 7, we have F7 = 13 ≥ 27/2 = 11.31.

(Inductive case:) if n ≥ 8, the inductive assumption implies that Fn−1 ≥20.5(n−1) and Fn−2 ≥ 20.5(n−2).

As such, we have: Fn+1 = Fn +Fn−1 ≥ 2n/2 + 2(n−1)/2 = 2(n−1)/2(21/2 +1) ≥ 2(n−1)/2 · 2 = 2(n+1)/2

Thus, for all n ≥ 6, we have Fn ≥ 20.5n.

3. 2.3 from text book.

In class, we saw a method for solving recurrence relations which is basedon analyzing the recursion tree and deriving a formula for the work doneat each level. Another (closely related) method is to expand out therecurrence a few times, until a pattern emerges. For instance, let’s startwith the familiar T (n) = 2T (n/2)+O(n). Think of O(n) as being ≤ cn forsome constant c, so: T (n) ≤ 2T (n/2) + cn. By repeatedly applying thisrule, we can bound T (n) in terms of T (n/2), then T (n/4), then T (n/8),and so on, at each step getting closer to the value of T () we do know,namely T (1) = O(1):

T (n) ≤ 2T (n/2) + cn

≤ 2[2T (n/4) + cn/2] + cn = 4T (n/4) + 2cn

≤ 4[2T (n/8) + cn/4] + 2cn = 8T (n/8) + 3cn

≤ 8[2T (n/16) + cn/8] + 3cn = 16T (n/16) + 4cn

· · ·

A pattern is emerging... the general term is

T (n) ≤ 2kT (n/2k) + kcn

Plugging in k = log2 n (the depth of the recursion) we get T (n) ≤ nT (1)+cn log2 n = O(n log n).

2

(a) Do the same thing for the recurrence T (n) = 3T (n/2) +O(n). Whatis the general kth term in this case? And what value of k should beplugged in to get the answer?

(b) Now try the recurrence T (n) = T (n− 1) +O(1), a case which is notcovered by the master theorem. Can you solve this too?

Solution:

(a) From the recurrence relation, T (n) ≤ 3T (n/2) + cn for some constantc. So:

T (n) ≤ 3T (n/2) + cn

≤ 3(3T (n/4) + cn/2) + cn = 32T (n/4) + (3/2 + 1)cn

≤ 32(3T (n/8) + cn/4) + (3/2 + 1)cn = 33T (n/23) + (32/22 + 3/2 + 1)cn

So the general term is:

T (n) ≤ 3kT (n/2k) + cn

k∑i=0

(3/2)i

Since n is reduced by a factor of 2 in each step, the depth of therecursion tree is log2 n. Setting k = log2 n and using the fact that∑k

i=0(3/2)i = O((3/2)k) by the Geometric Series Lemma,

T (n) ≤ 3log2 nT (1) + cn3log2 n/2log2 n

Since 3log2 n = nlog2 3 and T (1) ≤ c,

T (n) ≤ 2cnlog2 3 = O(nlog2 3)

(b) This one is easier:

T (n) ≤ T (n− 1) + c for some constant c

≤ T (n− 2) + c+ c = T (n− 2) + 2c

And the general term is:

T (n) = T (n− k) + ck

Since n is reduced by 1 in each iteration, setting k = n results in:

T (n) = O(n)

4. 2.4 from text book. The recursive relations for each of the three algo-rithms are:A: T (n) = 5T (n/2) +O(n)B: T (n) = 2T (n− 1) + c

3

C: T (n) = 9T (n/3) +O(n2)

Using master theorem to solve for A and C, we have:A: T (n) = O(nlog2 5)C: T (n) = O(n2 log n)

As for case B, we will use the technique introduced in problem 2.3:T (n) = 2T (n − 1) + c = 2(2T (n − 2) + c) + c = 4T (n − 2) + (2 + 1)c =

4(2T (n−3)+c)+(2+1)c = 8T (n−3)+(4+2+1)c = 2kT (n−k)+c∑k−1

i=0 2i

The depth k = n, thus we have T (n) = O(2n)

Among the three, algorithm C has the lowest time complexity.

5. 2.17 from text book.

Given a sorted array of distinct integers A[1, . . . , n], you want to find outwhether there is an index i for which A[i] = i. Give a divide and conqueralgorithm that runs in time O(log n).

Solution:

algo(A[1, . . . , n])1. if n == 1, return (A[n]==n)2. m = dn2 e3. if A[m] = m return true4. else if A[m] > m, return algo(A[1, . . . ,m− 1])5. else return algo(A[m+ 1, . . . , n]−m)

Each division reduce the problem size by half, and we do a constantamount of work in each function call. This gives us the following re-currence relation:

T (n) = T (n/2) +O(1)

which solves to T (n) = O(log n).

6. Proof by induction. Show that for every full binary tree, the number ofleave nodes is exactly one more than the number of the internal nodes.We showed a proof by doing induction on the number of nodes in the tree.In this assignment, please prove this statement by doing induction on thenumber of internal nodes.

Solution: The only difference between induction on the number of inter-nal nodes vs. on the number of total nodes in the tree is the base case.For internal nodes, the basecase is n = 0. For number of total nodes, thebase case is n = 1. We can use the same inductive arguments for both.

4

Sample questions The following questions are provided with solutions tohelp you review the contents.

0.1 from textbook

f(n) g(n) f = O(g)? f = Ω(g)? f = θ(g)?a n− 100 n− 200 yes yes yes =

b n1/2 n2/3 yes no no <c 100n+ log n n+ (log n)2 yes yes yes =d n log n 10n log 10n yes yes yes =e log 2n log 3n yes yes yes =f 10 log n log(n2) yes yes yes =g n1.01 n(log n)2 no yes no >h n2/ log n n(log n)2 no yes no >i n0.1 (log n)10 no yes no >j (log n)logn n/ log n no yes no >k

√n (log n)3 no yes no >

l n1/2 5log2 n yes no no <m n2n 3n yes no no <n 2n 2n+1 yes yes yes =o n! 2n no yes no >

p (log n)logn 2(log2 n)2 yes no no <q

∑ni=1 i

k nk+1 yes yes yes =

2.5 from textbook

a. T (n) = O(nlog3 2)

b. T (n) = O(nlog4 5)

c. T (n) = O(n log7 n)

d. T (n) = O(n2 log3 n)

e. T (n) = O(n3 log2 n)

f. T (n) = O(n3/2 log n)

g. T (n) = O(n)

h. T (n) = nc + (n− 1)c + ...+ 1c =∑n

i=1 ic = O(nc+1)

i. T (n) =∑n

i=1 ci = O(cn)

j. T (n) = O(2n)

k. T (n) = T (n1/2) + 1 = T (n1/4) + 2 = T (n(1/2)k

) + k Let’s assumethat it stops the recursive call if the input size is a small constant b.

Solving n1/2k

= b leads to k = O(log log n) and T (n) = O(log log n).

2.12 from textbook.

The recurrence relation is: T (n) = 2T (n/2) + c. Solving this recurrencerelation using the master theorem, we have case 3 and T (n) = Θ(nlog2 2) =Θ(n)

5