1 Recursion, Recurrences and Induction Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong

1

Recursion, Recurrences and Induction

Supplementary Notes

Prepared by Raymond WongPresented by Raymond Wong

2

e.g.1 (Page 7) Tower of Hanoi

When n = 1 (i.e., no. of disks = 1) I want to move all n disks from i to j

i j{i, j}

3

e.g.1 Tower of Hanoi


i j{i, j}

We move ONE disk from i to j.

No. of disk moves = 1

Disk move:Moving ONE disk from i to j

4



i j{i, j}

Top disk

Bottom disk

5



i j{i, j}

6



i j{i, j}

We move ONE top disk from i to {i, j}.


Disk move:Moving ONE top disk from i to {i, j}

7



i j{i, j}

We move ONE bottom disk from i to j.



Moving ONE bottom disk from i to j

2

8



i j{i, j}

We move ONE top disk from {i, j} to j.




2

Moving ONE top disk from {i, j} to j

3

9


n can be any positive integer I want to move all n disks from i to j

i j{i, j}

Top disk group

containing n-1 disks

Bottom disk

Virtual top box

10



i j{i, j}

11



i j{i, j}

We move ONE top disk group from i to {i, j}.

No. of disk moves = no. of disk moves for one top disk group

Disk move:Moving ONE top disk group from i to {i, j}

12



i j{i, j}





+ 1

13



i j{i, j}



We move ONE top disk group from {i, j} to j.

Moving ONE top disk group from {i, j} to j


+ 1

+ no. of disk moves for one top disk group

14



i j{i, j}





No. of disk moves = 2.no. of disk moves for one top disk group

+ 1

No. of disk moves for n

disks

No. of disk moves for n-1

disks

15







+ 1


disks

No. of disk moves for n-1

disks

The top disk group corresponds to top n-1 disks

top n-1 disks

top n-1 disks

The bottom disk corresponds to the largest disk

the largest disk

Let M(n) be the total number of disk moves needed for n disks.

M(n) M(n-1)

Note that M(1) = 1

16







+ 1


disks

top n-1 disks

top n-1 disks

the largest disk

M(n) M(n-1)

Why is it correct?

This is because we are implicitly using induction.We assume that we can move top n-1 disks correctly.

Let p(n) is a statement that the algorithm is correct for n.

Step 1: Prove that p(1) (i.e., the base case) is true.

Step 2: Prove that “p(n-1) p(n)” is true for all n > 1.Step 2(a): Assume that p(n-1) is true for n > 1.

Verify that p(1) is true

Step 2(b): According to p(n-1), we deduce that p(n) is true.

Inductive Hypothesis

Inductive Step

Note that M(1) = 1

17







+ 1


disks

top n-1 disks

top n-1 disks

the largest disk

M(n) M(n-1)

Why is it correct?

This is because we are implicitly using induction.We assume that we can move top n-1 disks correctly.

Let p(n) is a statement that the algorithm is correct for n.


Note that M(1) = 1

18



i j{i, j}


19



i j{i, j}

This case is obviously true.


20



i j{i, j}



21



i j{i, j}

We move ONE top disk group from i to {i, j}.



We assume that moving n-1 disks is

correct.

22



i j{i, j}




23



i j{i, j}




We assume that moving n-1 disks is

correct.

24

e.g.2 (Page 11) Given M(1) = 1 and

M(n) = 2M(n-1) + 1 for n > 1 What are the values of M(1), M(2), M(3), M(4) and M(5)?M(1) = 1

M(2) = 2M(1) + 1 = 2.1 + 1 = 3

M(3) = 2M(2) + 1 = 2.3 + 1 = 7

M(4) = 2M(3) + 1 = 2.7 + 1 = 15

M(5) = 2M(4) + 1 = 2.15 + 1 = 31

21 - 1

22 - 1

23 - 1

24 - 1

25 - 1

It “seems” that M(n) = 2n-1

25

e.g.3 (Page 12)

Illustration of “Proof by mathematical induction” (Weak Induction)

We are going to prove the following claim C: statement P(n) is true for each positive integer n, namely 1, 2, …

P(1) true

P(2)

P(3)

P(4)

P(5)

…

Step 1: Prove that P(1) (i.e., the base case) is true.

Step 2: Prove that “P(n-1) P(n)” is true for all n > 1.

true

true

true

true

true

Step 2(a): Assume that P(n-1) is true for n > 1.

Verify that P(1) is true

Step 2(b): According to P(n-1), we deduce that P(n) is true.

Inductive Hypothesis

Inductive Step

26

e.g.4 (Page 17) Let S(n) be the number of subsets of a set X of size n. What are the values of S(0), S(1), S(2) and S(3)?

When n = 0, X = {} There is a subset, namely an empty set (i.e., {})

When n = 1, X = {1} There are 2 subsets, namely {} and {1}.

Thus, S(0) = 1

Thus, S(1) = 2

When n = 2, X = {1, 2}There are 4 subsets, namely {}, {1}, {2}, {1, 2}.Thus, S(2) = 4

When n = 3, X = {1, 2, 3}There are 8 subsets, namely {},{1},{2},{3},{1, 2},{1, 3},{2, 3},{1, 2, 3}

Thus, S(3) = 8

It “seems” that S(n) = 2n

20

21

22

23

27

e.g.4Consider the 8 subsets of {1, 2, 3}

{} {1} {2} {1, 2}

{3} {1, 3} {2, 3} {1, 2, 3}

All 4 possible subsets of {1, 2}

Each set is the same as the above set by adding a number 3

All possible subsets of a set of size 2.

The total number of such subsets isequal to S(2).

All possible subsets of a set of size 2 by adding a number 3

The total number of such subsets isequal to S(2).

The total number of subsets of a set of size 3 (i.e., S(3) ) is equal to2.S(2)

In general, we derive that S(n) = 2.S(n-1)

We want to find a recursion formula for S(n).

28

e.g.5 (Page 21)

time

0 1 2 3 n-1 n…

Raymond borrows a loan with initial amount A (e.g., $100,000) from the bank.

Raymond needs to return a monthly payment M (e.g., $1000) every month.

Raymond returns a monthly payment M (e.g., $1000) to the bank.


Raymond returns M

Raymond returns M

Raymond returns M

Suppose T(n) is the amount of money in the bank at the n-th month.

T(0) T(1) T(2) T(3) T(n-1) T(n)

29

e.g.5

time

0 1 2 3 n-1 n…

Raymond borrows a loan with initial amount A (e.g., $100,000) from the bank.

Raymond needs to return a monthly payment M (e.g., $1000) every month.



Raymond returns M

Raymond returns M

Raymond returns M

Suppose T(n) is the amount of money in the bank at the n-th month.

T(0) T(1) T(2) T(3) T(n-1) T(n)=A =T(0)*(1+0.01p/12) – M

Suppose the interest rate is p% per year

Suppose the interest rate is (p/12)% per month.

Suppose the interest rate is (0.01p/12) per month.

=T(1)*(1+0.01p/12) – M

=T(n-1)*(1+0.01p/12) – M

…

30

e.g.6 (Page 24)

T(0) = bT(n) = rT(n-1) + a if n > 0

Derive a closed form for T(n) by a “top-down” approach (or called iterating recurrence)

T(0) = bT(n) = rT(n-1) + a if n > 0

31

e.g.6

T(0) = bT(n) = rT(n-1) + a if n > 0

T(n) = rT(n-1) + a = r(rT(n-2) + a) + a

= r2T(n-2) + ra + a = r2(rT(n-3) + a) + ra + a = r3T(n-3) + r2a + ra + a = r3(rT(n-4) + a) + r2a + ra + a = r4T(n-4) + r3a + r2a + ra + a

= …= rnT(n-n) + rn-1a + rn-2a + … + ra + a

= rnT(0) + a(rn-1 + rn-2 + … + r + 1)

= rnT(0) + a(rn-1 + rn-2 + … + r + r0)

= rnb + a ri

1

0

n

i

Note that

ri

1

0

n

i

=1 – rn 1 – r

if r 1= rnb + a 1 – rn

1 – r

32

e.g.7 (Page 25)

T(0) = bT(n) = rT(n-1) + a if n > 0

Derive a closed form for T(n) by a “bottom-up” approach

T(0) = bT(n) = rT(n-1) + a if n > 0

33

e.g.6

T(0) = bT(n) = rT(n-1) + a if n > 0

T(0) = b

T(1) = rT(0) + a = rb + a

T(2) = rT(1) + a = r(rb+a) + a = r2b + ra + a

T(3) = rT(2) + a = r(r2b + ra + a) + a = r3b + r2a+ ra + a

…

T(n) = rnb + rn-1a + rn-2a + … + ra + a

= rnb + a(rn-1 + rn-2 + … + r + 1)= rnb + a(rn-1 + rn-2 + … + r + r0)

= rnb + a ri

1

0

n

i

Note that

ri

1

0

n

i

=1 – rn 1 – r

if r 1= rnb + a 1 – rn

1 – r

34

e.g.7 (Page 31)

We want to prove the following corollary.Corollary 4.2: The formula for the sum of a geometric series with r 1 is

ri

1

0

n

i

=1 – rn 1 – r

35

e.g.7

Corollary 4.2: The formula for the sum of a geometric series with r 1 is

ri

1

0

n

i

=1 – rn 1 – r

Theorem 4.1 is shown as follows.

Theorem 4.1:If T(0) = b T(n) = rT(n-1) + a if n > 0then = rnb + a 1 – rn

1 – rT(n)

Let T(0) = 0

Let T(n) = ri

1

0

n

i

for n > 0

We want to show that T(n) = 1 – rn 1 – r

by using Theorem 4.1 (which is relatedto recursion).

We want to write T(n) in a recursion form. Consider T(n) = rn-1 + rn-2 + … + r + r0

= (rn-1 + rn-2 + … + r) + r0

= r(rn-2 + rn-3 + … + r0) + 1= r T(n-1) + 1 This is in a recursion

form.We know that b = 0We know that a = 1

According to Theorem 4.1, we have

= rnb + a 1 – rn 1 – r

T(n)

= rn0 + 11 – rn 1 – r

1 – rn 1 – r

=Done

36

e.g.8 (Page 32) We want to prove the following

theorem.Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series

Then, the value of the geometric series is O(t(n)).

ri

1

0

n

i

e.g.,If r = 0.5, then we have the value V of geometric series is equal to 0.50 + 0.51 + 0.52 + … + 0.5n-1

1 + 0.5 + 0.25 + … + 0.5n-1

The largest term t(n) in this series is equal to 1 (= 0.50)

e.g.,If r = 2, then we have the value V of geometric series is equal to 20 + 21 + 22 + … + 2n-1

1 + 2+ 2 + … + 2n-1

The largest term t(n) in this series is equal to 2n-1

Lemma 4.3 states that V = O(t(n)) = O(1)

Lemma 4.3 states that V = O(t(n)) = O(2n-1)

Why is it correct?

37

e.g.8

Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series


ri

1

0

n

i

Consider two case.

Case 1: r < 1

Case 2: r > 1

From our previous slide, we know that t(n) = r0 = 1

From our previous slide, we know that t(n) = rn-1

Consider Case 1

ri

1

0

n

i

We know that =1 – rn 1 – r

< 1

1 – r

Note that r is a constant. 1

1 – rThus, is a constant.

= O(1)

= O(t(n))

Note that we can show that = (t(n)) ri

1

0

n

i

38

e.g.8

Lemma 4.3: Let r 1 be a positive value independent of n.Let t(n) be the largest term in the geometric series


ri

1

0

n

i

Consider two case.

Case 1: r < 1

Case 2: r > 1

From our previous slide, we know that t(n) = r0 = 1

From our previous slide, we know that t(n) = rn-1

Consider Case 2

ri

1

0

n

i

We know that =1 – rn 1 – r

Note that r is a constant. r

1 – rThus, is a constant.

= O(rn-1)= O(t(n))

=rn – 1 r – 1

< rn

r – 1

=r

r – 1rn-1

Note that we can show that = (t(n)) ri

1

0

n

i

39

e.g.9 (Page 35) First-order Linear Recurrence

Dependent on one-step backward (i.e., T(n-1))

E.g., T(n) = f(n)T(n-1) + g(n) Second-order Linear Recurrence

Dependent on two-steps backward (i.e., T(n-2))E.g., T(n) = T(n-1) +2T(n-2)

First-order Non-Linear Recurrence Linear means the power of T(n-1) is 1. Non-linear means the power of T(n-1) is not 1. E.g., T(n) = (T(n-1))2 + 3

We are interested in this recurrence where f(n) is equal to a constant r.

40

e.g.10 (Page 37)

Prove the following Theorem 4.5 by induction.

Theorem 4.5: For any positive constants a and r, and any function g defined on the non-negative integers, the solution to the first-order linear recurrence

isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)

41

e.g.10


isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)

Step 1: Prove that P(0) (i.e., the base case) is true.

Let P(n) be “ ” rn-i

n

i 1

T(n) = rna + g(i)

T(0) = a (by definition)

= r0aWe want to show that

r0-i

10

0i

T(0) = r0a + g(i)

That is, T(0) = r0a

Thus, P(0) is true.

42

e.g.10


isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)


n

i 1

T(n) = rna + g(i)

Step 2: Prove that “P(n-1) P(n)” is true for all n > 0.Step 2(a): Assume that P(n-1) is true for n > 0.

r(n-1)-i

1

1

n

i

T(n-1) = rn-1a + g(i)That is,


Consider T(n) = rT(n-1) + g(n) (by definition) r(n-1)-i

1

1

n

i

rn-1a + g(i)= r( ) + g(n)

We want to show that

rn-i

n

i 1

T(n) = rna + g(i)

r(n-1)-i

1

1

n

i

rna + r g(i)= + g(n)

r1+(n-1)-i

1

1

n

i

rna + g(i)= + g(n)

rn-i

1

1

n

i

rna + g(i)= + rn-ng(n)

rn-i

n

i 1

rna + g(i)=

43

e.g.10


isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)


n

i 1

T(n) = rna + g(i)

Step 2: Prove that “P(n-1) P(n)” is true for all n > 0.Step 2(a): Assume that P(n-1) is true for n > 0.

r(n-1)--i

1

1

n

i

T(n-1) = rn-1a + g(i)That is,


We want to show that

rn-i

n

i 1

T(n) = rna + g(i)

Thus, P(n) is true.We prove that “P(n-1) P(n)” is true for all n > 0

By Mathematical Induction,

rn-i

n

i 1

T(n) = rna + g(i)

44

e.g.11 (Page 39)

We want to find the closed form of T(n)where

T(0) = 6 T(n) = 4T(n-1) + 2n if n > 0

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0

45

e.g.11

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0


isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)

We can write the closed form directly by using Theorem 4.5.

We have r = 4 g(n) = 2n a= 6

Thus, by using Theorem 4.5, we have

rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 4n.6 + 4n-i . 2i

n

i 1= 4n.6 +4n 4-i . 2i

46

e.g.11

T(0) = 6T(n) = 4T(n-1) + 2n if n > 0


We have r = 4 g(n) = 2n a= 6


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 4n.6 + 4n-i . 2i

n

i 1= 4n.6 +4n 4-i . 2i

n

i 1= 4n.6 +4n (22)-i . 2i

n

i 1= 4n.6 +4n 2-2i . 2i

n

i 1= 4n.6 +4n 2-i

n

i 1= 4n.6 +4n

1

2( ) i

n

i 1= 4n.6 +4n

1

2( ) i-1

1

2

1

0

n

i= 4n.6 +4n

1

2( ) i

1

2

= 4n.6 +4n . 1

2

1 – (½)n

1 – ½

= 4n.6 +4n . 1

2

1 – (½)n

½

= 4n.6 +4n . (1 – (½)n)

= 4n.6 +4n – 4n(½)n

= 4n.7 – 22n . 2-n = 4n.7 – 2n

47

e.g.12 (Page 40)

We want to find the closed form of T(n)where

T(0) = 10 T(n) = 3T(n-1) + n if n > 0

T(0) = 10T(n) = 3T(n-1) + n if n > 0

48

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


isT(n) =

rT(n-1) + g(n) if n > 0,a if n = 0,

rn-i

n

i 1

T(n) = rna + g(i) (*)


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i

49

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i

Theorem 4.6: For any real number x 1,

n

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2

Suppose that we know the following theorem.

n

i 1i . 3-i =

n

i 1i . (1/3)i

n(1/3)n+2 – (n+1)(1/3)n+1 + (1/3)

(1-1/3)2=

n.3-(n+2) – (n+1)3-(n+1) + 3-1

(2/3)2=

. (n.3-(n+2) – (n+1)3-(n+1) + 3-1)=32

22

50

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i


n

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2


n

i 1i . 3-i . (n.3-(n+2) – (n+1)3-(n+1) + 3-1)=

32

22

. (n.3-(n+2)+2 – (n+1)3-(n+1)+2 + 3-1+2)=1

22

. (n.3-n – (n+1)3-n+1 + 31)=1

4. (n.3-n – (n+1)3.3-n + 3)=

1

4

51

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i


n

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2


n

i 1i . 3-i . (n.3-n – (n+1)3.3-n + 3)=

1

4

. (n.3-n – (3n+3).3-n + 3)=1

4

. (n.3-n – 3n.3-n – 3.3-n + 3)=1

4

. (– 2n.3-n – 3.3-n + 3)=1

4

52

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i


n

i 1i . xi =

nxn+2 – (n+1)xn+1 + x

(1-x)2


n

i 1i . 3-i . (– 2n.3-n – 3.3-n + 3)=

1

4

. [(– 2n – 3).3-n + 3]=1

4

=–2n – 3

43-n +

3

4

n

i 1i . 3-i =

–2n – 3

43-n +

3

4

53

e.g.12

T(0) = 10T(n) = 3T(n-1) + n if n > 0


We have r = 3 g(n) = n a= 10


rn-i

n

i 1

T(n) = rna + g(i)

n

i 1= 3n.10 + 3n-i . i

n

i 1= 3n.10 + 3n 3-i . i

n

i 1= 3n.10 + 3n i . 3-i


n

i 1i . 3-i =

–2n – 3

43-n +

3

4

= 3n.10 + 3n (–2n – 3

43-n +

3

4)

= 3n.10 +–2n – 3

4+ 3n . 3

4

= 3n.(10 + ) – 2n + 3

4

3

4

= 3n. – 2n + 3

4

43

4

Documents

1 Recursion, Recurrences and Induction Supplementary Notes Prepared by Raymond Wong Presented by Raymond Wong