Recurrence Relations. Outline Recurrence relationsSolving recurrence relationsRecurrence and Divide-and-conquer algorithmsGenerating functions 2301233

Recurrence Relations

OutlineRecurrence relations

Solving recurrence relations

Recurrence and Divide-and-conquer algorithms

Generating functions

2301233 Recurrence Relation 2

RECURRENCE RELATIONS


What is a recurrence relation?• A recurrence relation for the sequence {an} is an

equation that expresses an in terms of one or more of the previous terms of the sequence, namely a0, a1,…, an-1, for all integers n n0, where n0 is a non-negative integer.

• A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.

• Example: Given the recurrence relation an=2an-1 -an-2.• 3, 5, 7, 9, 11, … satisfies the recurrence relation.• 2, 3, 4, 5, 6, … also satisfies the recurrence relation.


Compound interest : ExampleLet Pn be the amount of money in the account after n years. If you initially deposit P0=1000 in your account and the bank gives k % per year with interest compounded annually.

Pn = (1+k/100) Pn-1

P1 = (1+k/100) P0 = (1+k/100) P0

P2 = (1+k/100) P1 = (1+k/100)2 P0

P3 = (1+k/100) P2 = (1+k/100)3 P0

Pn = (1+k/100) Pn-1 = (1+k/100)n P0


Binary strings with no “00” : Example


Strings with length n

Starting with 0

Starting with 00 Starting with 01

Starting with 1

00000 00001 00010 00011 00100 00101 00110 00111

01000 01001 01010 01011 01100 01101 01110 01111

10000 10001 10010 10011 10100 10101 10110 10111

11000 11001 11010 11011 11100 11101 11110 11111

All strings of length n-1 with no 00

All strings of length n-2 with no 00none

Binary strings with no “00” : ExampleLet an be the number of n-bit binary strings containing no “00”.

an = an-1 + an-2

a0 = 1, a1 = 2, a2 = 32301233 Recurrence Relation 7

Strings with length n

Starting with 0

Starting with 00 Starting with 01

Starting with 1

All strings of length n-1 with no 00

All strings of length n-2 with no 00none

Rabbit breeding & Fibonacci Numbers • A pair of rabbits is placed on an island. After 2 months

old, a pair of rabbits produces another pair each month. How many rabbits are there after n months?

month 1-month rabbits >=2-month rabbits Total (fn)

1 1 0 12 0 1 13 1 1 24 1 2 35 2 3 56 3 5 87 5 8 138 8 13 21


fn =fn-1 + fn-2

Tower of Hanoi


Let H(n) be the number of move to solve tower of Hanoi with n discs.

H(n) = 2H(n-1) + 1 H(1) = 1

SOLVING RECURRENCE RELATIONS


Verify solutions of recurrence relations


an = (-4)n

an = 1

an = 3n

an = 2n + 1an = 4an-2 – 3 an-1

(-4)n = 4(-4)n-2 – 3(-4)n-1 ?

4(-4)n-2+12(-4)n-2 = 16(-4)n-2

4(-4)n-2+12(-4)n-2 = (-4)n

3n = 4(3(n-2)) – 3 (3(n-1)) ?

12n – 24 – 9n +9 = 3n – 15

3n

1= 4 (1) – 3(1) ?

1= 1

2n+1=4(2(n-2)+1) –3(2(n-1)+1)?

8n – 12 – 6n +3 = 2n – 9

2n +1

If no initial condition is given, there can be more than one

solution of a recurrence relation.

Types of recurrence relations


k, the number of terms referred back

• Order or degree

Function g

• linear or non-linear• The coefficients of terms are constants or variables

Function F

• some constant (Homogeneous)• some function of n (Non-homogeneous)

an= g(an-1 , an-2 ,…,an-k ) + F(n)

Example: Recurrence relations

• linear, homogeneous, degree 1, with constant coefficient

an = 1.2 an-1

• linear, homogeneous, degree 3, with constant coefficient

an = 3an-3

• Non-linear, homogeneous, degree 2, with constant coefficient

an = a2n-1 + an-2

• Linear, homogeneous, degree 2, with variable coefficient

an = nan-1 - 2an-2


• Linear, non-homogeneous, degree 1, with constant coefficient

an = an-1 + 2n

• Linear, homogeneous, degree 1, with constant coefficient

hn = 2hn-1 + 1

• Non-linear, non-homogeneous, degree 1, constant coefficient

an = a2n-1 + 2n

• Linear, non-homogeneous, degree 1, variable coefficient

an = n2 an-1 + n

How to solve recurrence relations

Guess and verify

Unfold

Variable transformation

Using formula

Generating function


Guess and verifyFind the closed form formula of an = 2an-1 +1 for n >0, a0=0

< a0 , a1, a2, a3 , a4, a5,…> = <0, 1, 3, 7, 15, 31, …>

Guess: an = 2n -1, for n 0.

Verify: • a0 = 20 -1 = 0

• an = 2an-1 +1 = 2(2n-1 -1) +1 = 2n - 2 +1 = 2n – 1

So, our guess is correct. • an = 2n -1, for n 0. 2301233 Recurrence Relation 15

Difficult to make a correct guess

Guess and VerifyFind the closed form formula of an = an-1 +n for n >0, a0=1

a0 = 1, a1= 1+1, a2 = 1+1+2, a3 =1+1+2+3, a4 = 1+1+2+3+4, a5 = 1+1+2+3+4+5

Guess: an = 1+ 0in

i , for n 0.

an = 1+n(n+1)/2 = (n2+n+2)/2Verify: a0 = (02+0+2)/2 = 1

an = an-1+n =[(n-1)2+(n-1)+2]/2 +n = (n2+n+2)/2

So, our guess is correct. • an = (n2+n+2)/2 , for n 0. 2301233 Recurrence Relation 16

UnfoldFind the closed form formula of an = an-1 +n for n >0, a0=1

an = an-1 +n

= (an-2 + n-1)+n

= ((an-3 + n-2) + n-1)+n

…

= ((…(a0+1)… + n-2) + n-1)+n

= 1+1… + n-2 + n-1+n

=1+n(n+1)/2

= (n2+n+2)/22301233 Recurrence Relation 17

Recurrence in Tower of HanoiH(1) = 1

H(n) = 2H(n-1) + 1

= 2(2H(n-2) +1) +1

= 4H(n-2) + 2 + 1

= 4(2(H(n-3) + 1) + 2 + 1

= 8H(n-3) + 4 + 2 + 1

= 8(2H(n-4) +1) + 4 + 2 + 1

= 16H(n-4) + 8 + 4 + 2 + 1

= 2n-1 + 2n-2 + 2n-3 + … + 1

= 2n - 1

Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1

x + 1=(2n-1 + 2n-2 + … + 4 + 2 + 1)+1

x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2

x + 1 = 2n-1 + 2n-2 + … + 4 + 4

x + 1 = 2n-1 + 2n-2 + … + 8

x + 1 = 2n

x = 2n - 1


Variable TransformationFind the closed form formula of a(n) = a(n/2) +1 for n =2k, k 0 a1=1

Let b(k) = a(2k) .From a(1)=1 a(n) = a(n/2) +1

We get b(0)=1 b(k) = b(k-1) +1

The closed form formula of b(k) is 1+ k.

b(k) = 1+k = 1+ log2 n= a(n).

Thus, a(n) = 1+ log2 n.2301233 Recurrence Relation 19

Variable TransformationFind the closed form formula of an = nan-1 +n! for n >0, a0=2.

Divide the recurrence relation by n!, and we have an/n! = an-1/(n-1)! + 1.

Let bn = an/n!.

Then, b0=2 bn = bn-1 +1.

The closed form formula of bn is 2+ n.

bn = an/n! = 2+n .

Thus, an = n!(2 + n).2301233 Recurrence Relation 20

Variable TransformationFind the closed form formula of an = (an-1)2/an-2 for n >1, a0=1, a1=2.

Divide the recurrence relation by an-1, and we have an/an-1 = an-1/an-2.

Let bn = an/an-1.

Then, bn = bn-1 b1 = a1/a0 =2.

The closed form formula of bn is 2.

bn = an/an-1 = 2.

Thus, an = 2an-1, and an = 2n.2301233 Recurrence Relation 21

Using formulaLinear homogeneous recurrence relation

• The sequence {an} is a solution of the recurrence relation an = c1 an-1 + c2 an-2 iff an = 1 rn

1 + 2 rn2 ,

where :– r1 and r2 are two distinct roots of r2 - c1r - c2 = 0,

– a0 = 1 + 2 and a1 = 1 r1 + 2 r2

(or

– 1 = (a1 – a0 r2)/(r1 - r2 ) and 2 = (a0 r1 –a1)/(r1 - r2 ) )


Where the formula comes from

Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.

Let and be constants.

Since r1 and r2 are roots of r2 - c1r - c2= 0,

r12 - c1r1 - c2 = 0 (r1

2 = c1r1 + c2)

and r22 - c1r2 - c2= 0 (r2

2 = c1r2 + c2).

c1an-1 + c2an-2 = c1(1r1n-1+2r2

n-1) + c2(1r1n-2+2r2

n-2)

= c11r1n-1+ c1 2r2

n-1 + c21r1n-2+ c2 2r2

n-2

= c11r1n-1+ c21r1

n-2 +c1 2r2n-1 + c2 2r2

n-2

= 1r1n-2 (c1r1 + c2) + 2r2

n-2 (c1r2 + c2)

= 1r1n-2 r1

2 + 2r2n-2 r2

2 = 1 r1n + 2 r2

n = an2301233 Recurrence Relation 23


The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn

1 + 2rn

2 +…+ k rnk , where :

– r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,

– a0 = 1 + 2 +…+ k

– a1 = 1 r1 + 2 r2 +…+ k rk

– a2 = 1 r21 + 2 r2

2 +…+ k r2k

– …


ExampleFind the solution of an = an-1 + an-2 with a0=0 a1=1 ?

We have c1 = 1 and c2 = 1.

The characteristic equation is r2 - r - 1 = 0.

The roots r1= (1+5)/2 and r2= (1-5)/2 .

From a0 = 1 + 2 and a1 = 1 r1 + 2 r2

0 = 1 + 2 and 1 = 1 r1 + 2 r2

1 = 1 r1 - 1 r2 = 1(r1 - r2) = 15 1 = 1/5

2= - 1 = -1/5 2= -1/5

an = 1 rn1 + 2 rn

2 = ((1+5)/2)n /5 - ((1-5)/2)n /5

an = ( (1+5)n - (1-5)n )/(52n )2301233 Recurrence Relation 25

ExampleWhat is the solution of an = an-1 + 2an-2

with a0=2, a1=7 ?

We have c1 = 1 and c2 = 2.

The characteristic equation is r2 - r - 2 = 0.

The roots r1= 2 and r2= -1 .

1 = (a1 – a0r2 )/(r1 - r2 ) = (7+2)/(2+1) = 3

2 = (a0r1 – a1)/(r1- r2 ) = (4-7)/(2+1) = -1

an = 1 rn1 + 2 rn

2 = 32n - (-1)n


ExampleWhat is the solution of an=6an-1-11an-2+6an-3

with a0=2, a1=5, a2=15 ?

The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with roots r1= 1, r2= 2 and r3= 3 .

a0 = 1 + 2 + 3

a1 = 1 r1 + 2 r2 + 3 r3 = 1 + 22 + 33

a2 = 1 r12 + 2 r2

2 + 3 r32 = 1 + 42 + 93

1 = 1, 2 = -1, 3 = 2

an = 1 – 2n + 23n



The sequence {an} is a solution of the recurrence relation an = c1an-1 + c2an-2 +…+ ckan-k iff

an = (1,0 + 1,1n + …+ 1,m1nm1-1

) rn1 +

(2,0 + 2,1n + …+ 2,m2nm2-1

) rn2 +…+

(t,0 + t,1n + …+ t,mtnmt-1

) rnt , where :

r1, r2 ,…, rt are distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0, with multiplicities m1, m2 ,…, mt


ExampleWhat is the solution of an= 6an-1 -9an-2

with a0=1, a1=6 ?

We have c1 = 6 and c2 = -9.

The characteristic equation is r2 - 6r + 9 = 0.

The root r0= 3.

1 = a0 = 1

2 = a1/r0 – a0 = 6/3-1 = 1

an = 1rn0 + 2nrn

0 = 3n + n 3n = 3n (1+ n).


Using formulaLinear non-homogeneous recurrence relation

If the sequence {an(p)} is a particular solution of

the recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k + F(n) then every solution is of the form

{an(p) + an

(h)} , where {an(h)} is a solution of the

associated homogeneous recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k.


ExampleWhat are the solutions of an= 3an-1+ 2n with a1=3 ?

We have c1 = 3.

The associated characteristic eqn is r-3=0, with root r = 3.

Then, an(h) = 3n .

Let an(p) = cn +d.

Then, from an= 3an-1+ 2n,

cn +d = 3(c(n-1)+d ) + 2n.

= (3cn + 2n) - 3c + 3d

= (3c + 2)n - 3c + 3d

c = 3c + 2, d = -3c + 3d .

Thus, c = -1 and d = -3/2. That is, an

(p) = -n -3/2

an= an(h) + an

(p) = 3n -n -3/2.

From a1= 3 = 3 -1 - 3/2,

= 11/6.

Solution:an= (11/6)3n -n -3/2.


TheoremLet {an} be a solution of the linear non-homogeneous recurrence relation

an = c1an-1 + c2an-2 +…+ ckan-k + F(n),

where c1, c2, …, ck are real numbers, and

F(n) = (btnt + bt-1nt -1+…+ b1n + b0) sn,

where b0, b1,…, bt and s are real numbers.

When s is not a root of the characteristic equation of the associated linear homogeneous recurrence relation, there is a particular solution of the form

(ptnt + pt-1nt -1+…+ p1n + p0) sn.2301233 Recurrence Relation 32

TheoremLet {an} be a solution of the linear non-homogeneous recurrence relation

an = c1an-1 + c2an-2 +…+ ckan-k + F(n),

where c1, c2, …, ck are real numbers, and

F(n) = (btnt + bt-1nt -1+…+ b1n + b0) sn,

where b0, b1,…, bt and s are real numbers.

When s is a root of the characteristic equation with multiplicity m, there is a particular solution of the form

nm (ptnt + pt-1nt -1+…+ p1n + p0) sn.


Example: SummationWhat are the solutions of an= an-1+ n with a1=1 ?

We have c1 = 1.

The associated characteristic eqn is r-1=0, with root r = 1.

Then, an(h) = 1n = .

Let an(p) = n(cn +d) = cn2 +dn.

Then, from an= an-1+ n, cn2 +dn = c(n-1)2+d (n-1) + n.

Thus, cn2 +dn = cn2-2cn+c +d n - d + n , c-d + n(1-2c)=0.

c-d =0 and 1-2c=0. That is, c=d=1/2.

an= an(h) + an

(p) = + n(n+1)/2. From a1= 1 = +1, = 0.

The solution is an= n(n+1)/2.2301233 Recurrence Relation 34

DIVIDE-AND-CONQUER ALGORITHMSRECURRENCE AND


Divide-and-conquer and Recurrence

Let f(n) be the number of operations required to solve a problem of size n.

f (n) = a f (n/b) + g(n)


XX1 X2 X3 Xa...n/b n/b n/b n/b

n elements

Binary Search• To search for an element a from a sorted list X of

size n elements.

– If a=m, then stop– If a<m, then search for a from X1.

– If a>m, then search for a from X2.

Let f (n) be the number of comparisons in binary search within n elements.

f (n) = f ( n/2 ) + 2.2301233 Recurrence Relation 37

XX1 X2m

Master MethodLet f be an eventually non-decreasing function that satisfies the recurrence relation

f (n) = a f (n/b) + cnd

whenever n n0, a 1, b > 1, c > 0, d 0, n=n0bk for any integer k > 0.

Let n=n0bk . Then, f (n0bk) = a f (n0bk-1) + cn0dbkd.

Let hi = f (n0bi ) . Then, hi = a hi-1 + c (bd)i, where c =cn0d.

hi( h) = ai

When abd : hi( p) = p bdi

When a=bd : hi( p) = pi ai


Master MethodWhen abd : hi

( p) = p bdi

p bdi = a p bd(i-1) + c (bd)i

p = a p/bdi + c

p (1 – a/bd ) = c p = c /(1 – a/bd )

hi = hi( h) + hi

( p)

= ai + c /(1 – a/bd ) bdi

f (n) = f (n0bi ) = hi (i=logb n/n0)

= ai + c /(1 – a/bd ) bdi

(ai = (n/n0)logb a , bdi = (n/n0)d )

= (n/n0)logb a+c /(1–a/bd ) (n/n0)d

= c1 nlogb a + (c/(1–a/bd )) nd

When a=bd : hi( p) = pi ai

pi ai = a p(i-1) a(i-1) + c ai

p ai = c ai

p = c = cn0d.

hi = hi( h) + hi

( p)

= ai + c i ai

f (n) = f (n0bi ) = hi (i=logb n/n0)

= ( + c i)ai

(ai = bd logb n/n0 = (n/n0)d)

= ( + c i )(n/n0)d

= c1 nd + cnd logb (n/n0) 2301233 Recurrence Relation 39

Master MethodLet f be an eventually non-decreasing function that satisfies the recurrence relation

f (n) = a f (n/b) + cnd

whenever n n0, a 1, b > 1, c > 0, d 0, n = n0bk for any positive integer k.

When a=bd, f(n) c1nd logbn.

When a>bd, f(n) c2nlogba

When a<bd, f(n) c3nd.2301233 Recurrence Relation 40

Binary Search• If a=m, then stop• If a<m, then search for a from X1.

• If a>m, then search for a from X2.

Let f(n) be the # of comparison in binary search within n elements.

f(n) = f(n/2) + 2.

Because a=1, f(n) k log n.


Merge sortprocedure msort(L: list of a1, a2, …, an)if n>1 then { m := n/2 L1 := list of a1, a2, …, am

L2 := list of am+1, am+2, …, an

merge(msort(L1), msort(L2))}

f(n) = 2f(n/2) + n. a = 2, b = 2, c = 1, d = 1

Since a=bd, f(n) kn log n.

procedure merge(A, B: list of n

elements)i := 1 j := 1 k := 1while i<n & j<n{ if A[i]<B[j] then C[k++] := A[i++]

else C[k++] := B[j++]

}


Fast Multiplication for Large Integers a = 2n A1 + A0. b = 2n B1 + B0.

A1 A0 B1 B0

a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.

Let f (n) be the number of operations needed to multiply two n-bit integers.

f (2n) = 3f(n) + Cn.a = 3, b = 2, c = C, d = 1. Because a>bd, f(n) k nlog23.


10010 00011 10000 10011

GENERATING FUNCTION


Generating function: Definition• The generating function for the sequence

a0, a1, …, ak, … of real numbers is the infinite series

G(x) = a0 + a1x + … + akxk + … =

k =0 ak xk

Examples:• The generating function for 1, 2, 4, ,…2kxk, … is 1

+ 2x + 4x2 + … + 2kxk + …• The generating function for 1, -1, 1, ,…(-1)k xk, …

is 1 - x + x2 - … + x2k - x2k+1 +…2301233 Recurrence Relation 45

Power Series

Let f (x) =

k =0 ak xk and g(x) =

k =0 bk xk.

Then, f (x) + g(x) =

k =0 (ak +bk)xk and

f (x) g(x) =

k =0(k

j=0 aj bk-j) xk .


Extended Binomial TheoremLet u be a real number and k be a nonnegative integer.

The extended binomial coefficient C(u, k) or ( u k ) is

u(u-1)…(u-k+1)/k! if k > 0. 1 if k = 0.

Let x be a real number with | x |<1 and u be a real number.

Then, (1+x)u = xn( u

k ) xk.


Useful Generating Functions

G(x) ak

(1 + x)n = 1 + C(n,1)x + C(n,2)x2 + …+ xn . C(n, k)

(1 + ax)n = 1 + C(n,1)ax + C(n,2)a2x2 + …+ anxn C(n, k) ak

(1 + xr)n = 1 + C(n,1)xr + C(n,2)x2r + …+ xnr C(n, k/r) if r | k0 otherwise

1/(1-x) =1 + x + x2 + … . 1

1/(1-ax) =1 + ax + a2x2 + … . ak

(1-xn+1)/(1-x) =1 + x + x2 + …+ xn . 1 if k n0 otherwise


ExampleSolve the recurrence relation ak= 3ak-1 with a0=2.

Let G(x) = a0 + a1x + a2x2 + a3x3 + … be the generating function for the sequence {ak}.

G(x) = a0 + a1x + a2x2 + a3x3 + …

3x G(x) = 3a0x + 3a1x2 + 3a2x3 + …

(1-3x)G(x) = a0 +(a1-3a0)x+(a2-3a1)x2+(a3-3a2) x3+ ..

(1-3x)G(x) = a0 = 2, and G(x) = 2/(1-3x)

From 1/(1-ax) =1 + ax + a2x2 +…, G(x)/2 = 1/(1-3x).

Thus, a = 3, and ak= 2(3k)-1 2301233 Recurrence Relation 49

ExampleSolve the recurrence relation an= 8an-1 + 10n-1 with a0=1.

Multiply the recurrence relation by xn , and get an xn = 8an-1 xn

+ 10n-1 xn

Let G(x) =

n=0 anxn be the generating function for

the sequence {an}.

G(x) =

n=0 anxn = a0 +

n=1 anxn

G(x) - 1 = 8

n=1 an-1 xn

+

n=1 10n-1 xn2301233 Recurrence Relation 50

Example

G(x) - 1 = 8

n=1 an-1 xn

+

n=1 10n-1 xn

G(x) - 1 = 8x

n=1 an-1 xn-1

+ x

n=1 10n-1 xn-1

G(x) - 1 = 8x

n=0 an xn

+ x

n=0 10n xn

G(x) - 1 = 8x G(x) + x /(1 - 10x)

(1 - 8x) G(x) = 1+ x /(1 - 10x)

(1 - 8x) G(x) = (1 - 9x) /(1 - 10x)

G(x) = (1 - 9x) /((1 - 10x) (1 - 8x))2301233 Recurrence Relation 51

ExampleG(x) = (1 - 9x) /((1 - 10x) (1 - 8x))

Using partial fraction, we get

G(x) = ½(1 /(1 - 10x) + 1/(1 - 8x))

From the known power series,

G(x) = ½(

n=0 10n xn +

n=0 8n xn )

G(x) =

n=0 ½(10n + 8n) xn

Thus, we know that an = ½(10n + 8n) 2301233 Recurrence Relation 52


Aren’t you glad it’s done?

Documents

Recurrence Relations. Outline Recurrence relationsSolving recurrence relationsRecurrence and Divide-and-conquer algorithmsGenerating functions 2301233