Homogeneous Recurrence Relation

7/28/2019 Homogeneous Recurrence Relation

1/30

Solving Recurrence Relations

Yen (NTUEE) Discrete Mathematics 2012 1 / 38
http://find/


2/30

Recurrence Relations

Definition

A recurrence relation for {an} is an equation that expresses an in termsof a0, . . . , an1. A sequence is called a solution of a recurrence relation ifits terms satisfy the recurrence relation.

For instance, the relation fn = fn1 + fn2 of Fibonacci numbers is a

recurrence relation.

http://find/


3/30

Tower of Hanoi

Example

(The Tower of Hanoi) Consider moving a stack of disks with different sizeswith three pegs. Initially, all disks are sorted and placed on the first peg.One can only move the topmost disk from a peg to another peg. At anytime, disks on each peg must be sorted as well. How many steps does it

take to move all disks from one peg to another?

Solution: Let Hn be the number of steps to solve the problem of n disks.Clearly, H1 = 1.

Now consider moving n disks from peg 1 to peg 2. If we can move thetopmost n 1 disks from peg 1 to peg 3, we can solve the puzzle bymoving the bottom disk from peg 1 to peg 2 and then the n 1 disksfrom peg 3 to peg 2. In other words, Hn = Hn1 + 1 + Hn1. We haveHn = 2Hn1 + 1.

http://goforward/http://find/http://goback/


4/30

Tower of Hanoi

Hence

Hn = 2Hn1 + 1= 2(2Hn2 + 1) + 1= 22Hn2 + 21 + 20

= 22(2Hn

3 + 1) + 21 + 20

= 23Hn3 + 22 + 21 + 20

= 2n1H1 + 2n2 + + 21 + 20

=

n

1

i=0

2i

=2n 12 1 = 2

n 1

http://find/


5/30

Catalan numbers

Example

Find the number of ways to parenthesize the product of n + 1 numbersx0, x1, . . . , xn.

Solution: Let Cn denote the number of ways to parenthesize the productof n + 1 numbers. For instance, C3 = 5 because

x0 (x1 (x2 x3)) x0 ((x1 x2) x3) (x0 (x1 x2)) x3 ((x0 x1) x2) x

are all the possible ways of multiplying x0 x1 x3.Clearly C0 = 1. Consider x0 x1 xn. We can compute it by(x0 xk) (xk+1 xn) for any k = 0, . . . , n. Therefore

Cn = C0Cn1 + C1Cn2 + + Cn1C0.

The sequence {Cn}, Cn = n1k=0 CkCnk1, is called Catalan numbers.

Well give a closed form for Catalan numbers later.Yen (NTUEE) Discrete Mathematics 2012 5 / 38
http://find/


6/30

Homogeneous Linear Recurrence Relations with Constant

Coefficients

DefinitionA linear homogeneous recurrence relation of degree k with constantcoefficients is of the form

an = c1an

1 + c2an

2 +

+ ckan

k,

where c1, c2, . . . , ck R with ck = 0.For instance, the recurrence for Fibonacci numbers fn = fn1 + fn2 is alinear homogeneous recurrence relation of degree two.

Theorem

Let c1, c2 R. Suppose x2 = c1x + c2 has two distinct roots r1 and r2.Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 ifand only if an = 1r

n1 + 2r

n2 for n

N and some constants1, 2.

http://find/


7/30

Solving Linear Recurrence Relations

Proof.() Suppose an = 1rn1 + 2rn2 . We want to verify an = c1an1 + c2an2.

c1an1 = c11rn11 + c12rn12

c2an2 = c21rn

2

1 + c22rn

2

2c1an1 + c2an2 = 1rn21 (c1r1 + c2) + 2r

n22 (c1r2 + c2)

= 1rn21 r

21 + 2r

n22 r

22

= 1rn1 + 2r

n2

= an

(Contd on Next Page)

http://find/http://goback/


8/30


Proof.

(Contd) () From the first part of proof, we know an = 1rn

1 + 2rn

2satisfies the recurrence relation an = c1an1 + c2an2. It remains to showan = 1r

n1 + 2r

n2 satisfies initial conditions for some 1, 2. The theorem

then follows from the uniqueness of solution to linear homogeneousrecurrence relation.

To see an = 1rn1 + 2rn2 satisfies the initial conditions for some 1, 2.Consider a1 = 1r1 + 2r2 and a0 = 1 + 2.This is a linear system of two variables 1 and 2. The solutions are

1 =a1 a0r2

r1 r2,

and

2 =a0r1 a1

r1 r2 .

http://find/


9/30

Fibonacci numbers

Example

Recall the recurrence relation for Fibonacci numbers fn = fn1 + fn2 withf0 = 0 and f1 = 1. Find an explicit formula for the Fibonacci numbers.

Solution: The solutions to x2 = x + 1 are 1

52 . Hence

fn = 1

1 + 52

n+ 2

1 5

2

n

for some 1 and 2. Solving 0 = 1 + 2, and 1 = 11+

52 + 2

1

52 , we

have 1 =1

5and 2 =

1

5. Therefore

fn =1

5

1 +

5

2

n 1

5

1 5

2

n.

http://find/


10/30

Characteristic Equations with Multiple Roots

Theorem

Let c1, c2 R with c2 = 0. Suppose x2

= c1x + c2 has only one root r.Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 ifand only if an = 1r

n + 2nrn for n N and some constants1, 2.

Proof.

() 2r = c1 and c1r + 2c2 = 0. Let an = 1rn + 2nrn. Thenc1an1 = c11rn1 + c12(n 1)rn1c2an2 = c21rn2 + c22(n 2)rn2

c1an1 + c2an2 = 1rn

2

(c1r + c2) + 2rn

2

(c1(n 1)r + c2(n 2))= 1r

n2r2 + 2rn2(c1nr c1r + c2n 2c2)= 1r

n + 2rn2(c1nr + c2n)

= 1rn + 2r

n2n(c1r + c2)

= 1rn

+ 2nrn

2

r2

n nYen (NTUEE) Discrete Mathematics 2012 10 / 38
http://find/


11/30


Proof.

(Contd)(

) It remains to show there are 1 and 2 satisfying the initial

conditions. We have a0 = 1. Therefore

2 =a1 a0r

r.

http://find/


12/30

An Example

Example

What is the solution of the recurrence relation

an = 6an

1

9an

2

with a0 = 1 and a1 = 6?

Solution: Since 3 is the multiple root of x2 = 6x 9, we havean = 13

n + 2n3n. Moreover, 1 = a0 = 1 and 2 = (6

1

3)/3 = 1.

We have an = 3n + n3n.

http://find/


13/30

Characteristic Equations with Distinct Roots

TheoremLet c1, c2, . . . , ck R. Suppose the characteristic equation

rk c1rk1 ck = 0

has k distinct roots r1, . . . , rk. Then {an} is a solution of the recurrencerelation

an = c1an1 + c2an2 + + ckankif and only if

an = 1rn1 + 2rn2 + + krnkfor n N and constants1, 2, . . . , k.

http://find/


14/30


Theorem

Let c1, c2, . . . , ck R. Suppose the characteristic equationrk c1rk1 ck = 0

has t distinct roots r1, . . . , rt with multiplicities m1, . . . ,mt respectively

such that mi 1 and m1 + m2 + + mk = k. Then {an} is a solution of the recurrence relation an = c1an1 + c2an2 + + ckank if and only if

an = (1,0 + 1,1n + + 1,m11nm11)rn1+ (2,0 + 2,1n +

+ 2,m2

1n

m21)rn2+ + (t,0 + t,1n + + t,mt1nmt1)rnt

for n N and constantsi,j.

http://find/


15/30

Solving Nonhomogeneous Linear Recurrence Relations

Definition

A linear nonhomogeneous recurrence relation with constantcoefficients is of the form

an = c1an1 + c2an2 + + ckank + F(n),where c1, c2, . . . , ck R and F : N Z. The recurrence relation

an = c1an1 + c2an2 + + ckankis called the associated homogeneous recurrence relation.

http://find/


16/30


Theorem

Let{a(p)n } be a particular solution of

an = c1an1 + c2an2 + + ckank + F(n),Then every solution is of the form {a(p)n + a(h)n }, where{a(h)n } is a solutionof

an = c1an1 + c2an2 + + ckank.

(Contd on Next Page)


S l i N h Li R R l i


17/30


Proof.

Since {a(p)n } is a particular solution, we have

a(p)n = c1a

(p)n1 + c2a

(p)n2 + + cka(p)nk + F(n).

Suppose{

bn}

be any solution of the nonhomogeneous recurrence relationsuch that

bn = c1bn1 + c2bn2 + + ckbnk + F(n).Then

bn a(p)n = c1(bn1 a(p)n1) + c2(bn2 a(p)n2) + + ck(bnk a(p)nk).

That is, {a(p)n bn} is a solution of the associated homogeneousrecurrence relation. Thus, bn = a

(p)n + a

(h)n .


A E l
http://find/


18/30

An Example

Example

Find all solutions of an = 3an1 + 2n with a1 = 3.

Solution: Let us guess a(p)n = cn + d. Then

cn + d = 3(c(n 1) + d) + 2n. We have cn + d = (3c+ 2)n + (3d 3c).Solve c = 3c + 2 and d = 3d 3c. We obtain c = 1 and d =

3

2 .Therefore a

(p)n = n 32 is a particular solution.

By Theorem 9, we have a(h)n = 3n as solutions to the associated

homogeneous recurrence relation. Therefore all solutions are of the forman =

n

3

2

+

3n where is a constant.Finally, 3 = a1 = 1 32 + 3. = 116 . Therefore

an = n 32

+11

63n.


A E l


19/30

An Example

ExampleFind solutions of an = 5an1 6an2 + 7n.

Solution: Let us guess a(p)n = c 7n. Then

c

7n = 5c

7n1

6c

7n2 + 7n. Hence 49c = 35c

6c + 49, c = 4920 .

a(p)n =4920 7n is a particular solution.

By Theorem 9, we have a(h)n = 12

n + 23n as solutions of the associated

homogeneous recurrence relation. Therefore

an = 12n + 23n + 4920 7n

are all solutions.


S l i P ti l S l ti
http://find/


20/30

Solving Particular Solutions

Theorem

Suppose{

an}

satisfies an

= c1

an1

+ c2

an2

+

+ ck

ank

+ F(n), wherec1, c2, . . . , ck R and

F(n) = (btnt + bt1nt1 + + b1n + b0)sn,

where s, b0, b1, . . . , btR.

When s is not a root of the characteristic equation of the associated linear

homogeneous recurrence relation, there is a particular solution of the form

(ptnt + pt1nt1 + + p1n + p0)sn.

When s is a root of the characteristic equation and its multiplicity is m,

there is a particular solution of the form

nm(ptnt + pt1nt1 + + p1n + p0)sn.


S l i N h Li R R l ti
http://find/


21/30


Example

Solve an = an1 + 3n2

3n + 1. with a0 = 0.Solution: Since an = an1 + 3n2 3n + 1, F(n) = (3n2 3n + 1)1n.By Theorem 9, we have a

(h)n = 1n = as homogeneous solutions.

By Theorem 15, we let a(p)n = n(p2n

2 + p1n + p0) as a particular solution.

Substitute a(p)n in the recurrence relation, we have

n(p2n2 + p1n + p0) = (n 1)(p2(n 1)2 + p1(n 1) + p0) + 3n2 3n + 1.

Simplify both sides of the equation and compare their coefficients:

p2 = p2

p1 = 3p2 + p1 + 3p0 = 3p2 2p1 + p0 3

0 = p2 + p1 p0 + 1


http://find/


22/30


(Contd)Solve the linear system and get p0 = 0, p1 = 0, p2 = 1. Hence

a(p)n = n(1 n2 + 0 n + 0) = n3 is a particular solution.

Thus, an = n3 + are all solutions. Since a0 = 0, we have = 0. And we

conclude an = n3.

Notice that n3 (n 1)3 = 3n2 3n + 1. Hencenk=1 3k

2 3k + 1 = n3. Alternatively, note that

an =

nk=1 3k

2

3k + 1,

you can solve it by computing the summation.


Solving Recurrence Relations by Domain Transformation
http://find/


23/30

Solving Recurrence Relations by Domain Transformation

Example

T(n) = 2T(n/2) + nlog2n, n > 1

where n is a power of 2.

Solutions: Replace n by 2k, we have

T(2k) = 2T(2k1) + k2k

Let tk = T(2k). The above can be rewritten as

tk = 2tk

1 + k2k

Hence,tk = c12

k + c2k2k + c3k22k

T(n) = c1n + c2nlogn + c3nlog2n


Solving Recurrence Relations by Range Transformation
http://find/


24/30

Solving Recurrence Relations by Range Transformation

Example

an = 3a2n1 subject to a0 = 1

Solution: Let bn = log an. Then the above can be rewritten as

bn = 2bn1 + log 3 subject to b0 = 0.

Hencebm = (2

n

1)log 3

an = 2(2n1)log3 = 32

n1


Divide and Conquer Algorithms and Recurrence Relations
http://find/


25/30

Divide-and-Conquer Algorithms and Recurrence Relations

Recall the merge sort algorithm. The algorithm divides the sequence inhalves, solves each half recursively, and then merges sorted results to getthe solution. This type of algorithms are called divide-and-conqueralgorithms. In the complexity analysis of divide-and-conquer algorithms,

we obtain the recurrence relation of the form

f(n) = af(n

b) + g(n),

where f(n) is the number of steps needed in solving an instance of size n.

We will prove a theorem to help us solve the recurrence relation.


http://find/


26/30


Theorem

Let f be a non-decreasing function satisfying

f(n) = af(n

b) + c

whenever b

|n, where a

1, b

Z

+, b> 1, and cR

+. Then

f(n) =

O(nlogba) if a > 1O(lg n) if a = 1

Furthermore, when n = bk and a > 1, kZ

+,

f(n) = C1nlogba + C2,

where C1 = c/(a 1) + f(1) and C2 = c/(a 1).


http://find/


27/30


Proof: We first consider n = bk. Then

f(n) = af(bk

b ) + c

= af(bk1) + c= a(af(bk2) + c) + c= a2f(bk2) + ac + c= = akf(1) + ak1c + ak2c + + ac + c

= akf(1) + ck1

i=0

ai

When a = 1, we have f(n) = f(1) + ck = f(1) + clogbn. Furthermore, ifbk < n < bk+1, we have f(bk) f(n) f(bk+1) and bk+1 < nb. Thus,f(n) f(bk+1) = f(1)+c(k+1) < f(1)+c(1+logbn) = f(1)+c+clogbn.

Therefore f(n) = O(lg n) when a = 1.Yen (NTUEE) Discrete Mathematics 2012 27 / 38

http://find/


28/30


(Contd)When a > 1 and n = bk, then nlogba = bklogba = ak. We have

f(n) = akf(1) + cak 1a 1

= ak

f(1) +c

a

1

c

a

1

= nlogbaC1 + C2

For bk < n < bk+1, we have

f(n) f(bk+1)= C1ak+1 + C2

= aC1ak + C2

aC1nlogba + C2

Hence, f(n) = O(n

logba

).Yen (NTUEE) Discrete Mathematics 2012 28 / 38

An Example
http://find/


29/30

An Example

Example

Let f(n) = 5f(n/2) + 3. Estimate f(n).

Solution: By Theorem 19, we have f(n) = O(n

log2 5

) = O(n

lg 5

) sincea = 5 and b = 2.

For merge sort algorithm, we have T(n) = 2T(n/2) + (n). Theorem 19is not applicable. The following theorem will be proved in your algorithm

class:


http://find/


30/30


Theorem

Let f be a non-decreasing function satisfying

f(n) = af(n

b) + cnd

whenever n = bk, k Z+, b Z+ with a 1, b> 1, and c, d R withc> 0, d 0. Then

f(n) =

O(nd) if a < bd

O(nd lg n) if a = bd

O(nlogba) if a > bd

Therefore, the complexity of merge sort algorithm is O(n lg n).

http://find/

Documents

Homogeneous Recurrence Relation