Recurrence Relation

Outline

What is a recurrence relation ?

Solving linear recurrence relations

Divide-and-conquer algorithms and recurrence relations

What is a recurrence relation ?

Definition

A recurrence relation for the sequence {an} is an equation that expresses an in terms of one or more of the previous terms of the sequence, namely a0, a1, …, an-1, for all integers n n0, where n0 is a non-negative integer.

A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation.

Example: Given the recurrence relation an=2an-1 -an-2.

3, 5, 7, 9, 11, … satisfies the recurrence relation.

2, 3, 4, 5, 6, … also satisfies the recurrence relation.

Example: Compound interest

Let Pn be the amount of money in the account after n years. If you initially deposit Pn=1000 in your account and the bank gives k % per year with interest compounded annually.

Pn = (1+k/100) Pn-1

P1 = (1+k/100) P0 = (1+k/100) P0

P2 = (1+k/100) P1 = (1+k/100)2 P0

P3 = (1+k/100) P2 = (1+k/100)3 P0

Pn = (1+k/100) Pn-1 = (1+k/100)n P0

Example: Fibonacci numbers

A pair of rabbits is placed on an island. After 2 months old, a pair of rabbits produces another pair each month. How many rabbits are there after n months?

fn =fn-1 + fn-2

month 1-month rabbits

>=2-month rabbits

Totalfn

1 1 0 1

2 0 1 1

3 1 1 2

4 1 2 3

5 2 3 5

6 3 5 8

7 5 8 13

8 8 13 21

Tower of Hanoi

Example: Tower of Hanoi

H(1) = 1

H(n) = 2H(n-1) + 1

H(n) = 2(2H(n-2) +1) +1

= 4H(n-2) + 2 + 1

= 4(2(H(n-3) + 1) + 2 + 1

= 8H(n-3) + 4 + 2 + 1

= 8(2H(n-4) +1) + 4 + 2 + 1

= 16H(n-4) + 8 + 4 + 2 + 1

= 2n-1 + 2n-2 + 2n-3 + … + 1

= 2n - 1

Let x = 2n-1 + 2n-2 + … + 4 + 2 + 1

x +1 = (2n-1 + 2n-2 + … + 4 + 2 + 1) +1

x + 1 = 2n-1 + 2n-2 + … + 4 + 2 + 2

x + 1 = 2n-1 + 2n-2 + … + 4 + 4

x + 1 = 2n-1 + 2n-2 + … + 8

x + 1 = 2n

x = 2n - 1

Linear recurrence relation

Homogeneous and non-homogeneous

What is a linear recurrence relation ?

Linear recurrence relation of degree k

an= c1an-1 + c2an-2 +…+ ckan-k + F(n)

ci is a constant, for i = 1, 2, …, k

ck 0

Linear homogeneous recurrence relation

F(n) = 0

Linear non-homogeneous recurrence relation

F(n) 0

Examples

Linear homogeneous

an = 1.2 an-1 : degree 1

fn = fn-1 + fn-2 : degree 2

an = 3an-3 : degree 3

Non-linear homogeneous

an = a2n-1 + an-2

an = nan-1 - 2an-2

Linear non-homogeneous

an = an-1 + 2n

hn = 2hn-1 + 1

an = 3an-1 + n

Non-linear non-homogeneous

an = a2n-1 + 2n

an = n2 an-1 + n

Solving linear homogeneous recurrence relation

Theorem

The sequence {an} is a solution of the recurrence

relation an = c1 an-1 + c2 an-2 iff an = 1 rn1 + 2

rn2 , where :

• r1 and r2 are two distinct roots of r2 - c1r - c2 = 0,

• a0 = 1 + 2

• a1 = 1 r1 + 2 r2

• [ 1 = (a1 – a0 r2)/(r1 - r2 ) 2 = (a0 r1 –a1)/(r1 - r2 ) ]

Informal proof

Recurrence relation: an = c1 an-1 + c2 an-2

Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.

Let and be constants.

Show if an = 1 rn1 + 2 rn

2 , the sequence {an} is a

solution of the recurrence relation.

Show if the sequence {an} is a solution of the

recurrence relation, then an = 1 rn1 + 2 rn

2 .

Informal proof

Show if an = 1 rn1 + 2 rn

2 , the sequence {an} is a

solution of the recurrence relation.

Let an = c1 an-1 + c2 an-2 be a recurrence relation.

Let r1 and r2 be two distinct roots of r2 – c1r – c2= 0.

Let and be constants.

Since r1 and r2 are roots of r2 - c1r - c2= 0,

r12 - c1r1 - c2 = 0 (r1

2 = c1r1 + c2) and r22 - c1r2 - c2= 0 (r2

2 = c1r2 + c2).

Informal proof

r12 = c1r1 + c2 and r2

2 = c1r2 + c2.

c1an-1 + c2an-2

= c1(1r1n-1+2r2

n-1) + c2(1r1n-2+2r2

n-2)

= c11r1n-1+ c1 2r2

n-1 + c21r1n-2+ c2 2r2

n-2

= c11r1n-1+ c21r1

n-2 +c1 2r2n-1 + c2 2r2

n-2

= 1r1n-2 (c1r1 + c2) + 2r2

n-2 (c1r2 + c2)

= 1r1n-2 r1

2 + 2r2n-2 r2

2 = 1 r1n + 2 r2

n = an

Informal proof

Show if the sequence {an} is a solution of the recurrence relation, then there are 1 and 2 such that an = 1 rn

1 + 2 rn

2 .

From the initial conditions a0 and a1 ,

a0 = 1 + 2 and a1 = 1 r1 + 2 r2

2 = a0 - 1

1r1 = a1 - 2r2 = a1 - (a0-1) r2 = a1 – a0r2 + 1r2

1 (r1 - r2 ) = a1 – a0r2 >> 1 = (a1 – a0r2 )/(r1 - r2 )

2 =a0-1 =a0-(a1–a0r2 )/(r1-r2 ) = a0r1-a0r2 -a1+a0r2 /(r1-r2 )

2 = (a0r1-a1) / (r1-r2 )

Example

What is the solution of an = an-1 + 2an-2 with a0=2 a1=7 ?

We have c1 = 1 and c2 = 2.

The characteristic equation is r2 - r - 2 = 0, with roots r1= 2 and r2= -1 .

1 = (a1 – a0r2 )/(r1 - r2 ) = (7+2)/(2+1) = 3

2 = (a0r1 – a1)/(r1- r2 ) = (4-7)/(2+1) = -1

an = 1 rn1 + 2 rn

2 = 32n - (-1)n

Example: Fibonacci number

What is the solution of an = an-1 + an-2 with a0=0 a1=1 ?

We have c1 = 1 and c2 = 1.

The characteristic equation is r2 - r - 1 = 0, with roots r1= (1+5)/2 and r2= (1-5)/2 .

1 = (a1 – a0r2 )/(r1 - r2 ) = (1-0)/5 = 1/5

2 = (a0r1 – a1)/(r1- r2 ) = (0-1)/5 = -1/5

an = 1 rn1 + 2 rn

2 = ((1+5)/2)n /5 - ((1-5)/2)n /5

an = ( (1+5)n - (1-5)n )/(52n )

Theorem

The sequence {an} is a solution of the recurrence

relation an = c1 an-1 + c2 an-2 iff an = 1rn0 + 2nrn

0 , where :

• r0 is the only root of r2 - c1r - c2 = 0,

• a0 = 1

• a1 = 1 r0 + 2 r0 = r0(1 + 2)

• [ 1 = a0 2 = a1/r0 – a0 ]

Example

What is the solution of an= 6an-1 -9an-2 with a0=1 a1=6 ?

We have c1 = 6 and c2 = -9.

The characteristic equation is r2 - 6r + 9 = 0, with roots r0= 3.

1 = a0 = 1

2 = a1/r0 – a0 = 6/3-1 = 1

an = 1rn0 + 2nrn

0 = 3n + n 3n = 3n (1+ n).

Theorem

The sequence {an} is a solution of the recurrence

relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn1

+ 2rn2 +…+ k rn

k , where :

• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,

• a0 = 1 + 2 +…+ k

• a1 = 1 r1 + 2 r2 +…+ k rk

• a2 = 1 r21 + 2 r2

2 +…+ k r2k

• …

Theorem

The sequence {an} is a solution of the recurrence

relation an = c1an-1 + c2an-2 +…+ ckan-k iff an = 1rn1

+ 2rn2 +…+ k rn

k , where :

• r1, r2 ,…, rk are k distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0,

• a0 = 1 + 2 +…+ k

• a1 = 1 r1 + 2 r2 +…+ k rk

• a2 = 1 r21 + 2 r2

2 +…+ k r2k

• …

Theorem

The sequence {an} is a solution of the recurrence

relation an = c1an-1 + c2an-2 +…+ ckan-k iff

an = (1,0 + 1,1n + …+ 1,m1nm1-1

) rn1 +

(2,0 + 2,1n + …+ 2,m2nm2-1

) rn2 +…+

(t,0 + t,1n + …+ t,mtnmt-1

) rnt , where :

• r1, r2 ,…, rt are t distinct roots of rk -c1rk-1 - c2rk-2 -…- ck= 0, with multiplicities m1, m2 ,…, mt

Example

What is the solution of an=6an-1-11an-2+6an-3 with a0=2, a1=5, a2=15 ?

We have c1 = 6, c2 = -11 and c3 = 6.

The characteristic equation is r3 - 6r2 + 11r - 6 = 0, with roots r1= 1, r2= 2 and r3= 3 .

a0 = 1 + 2 + 3

a1 = 1 r1 + 2 r2 + 3 r3 = 1 + 22 + 33

a2 = 1 r12 + 2 r2

2 + 3 r32

= 1 + 42 + 93

1 = 1, 2 = -1, 3 = 2

an = 1 – 2n + 23n

Solving linear non-homogeneous recurrence relation

Theorem

If the sequence {an(p)} is a particular solution of the

recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k + F(n) then every solution is of the form {an

(p) + an(h)}

, where {an(h)} is a solution of the associated

homogeneous recurrence relation an = c1 an-1 + c2 an-2 +…+ ck an-k.

Example

What are the solutions of an= 3an-1+ 2n with a1=3 ?

We have c1 = 3.

The associated characteristic eqn is r-3=0, with root r = 3.

Then, an(h) = 3n .

Let pn = cn +d.

Then, from an= 3an-1+ 2n, cn +d = 3(c(n-1)+d ) + 2n.

Thus, c = -1 d = -3/2, and an(p) = -n -3/2

an= an(h) + an

(p) = 3n -n -3/2.

From a1= 3 = 3 -1 - 3/2, = 11/6.

The solution is an= (11/6)3n -n -3/2.

Example: summation from 1 to n

What are the solutions of an= an-1+ n with a1=1 ?

We have c1 = 1.

The associated characteristic eqn is r-1=0, with root r = 1.

Then, an(h) = 1n = .

Let pn = n(cn +d) = cn2 +dn.

Then, from an= an-1+ n, cn2 +dn = c(n-1)2+d (n-1) + n.

Thus, cn2 +dn = cn2-2cn+c +d n - d + n , c-d + n(1-2c)=0.

c-d =0 and 1-2c=0. That is, c=d=1/2.

an= an(h) + an

(p) = + n(n+1)/2. From a1= 1 = +1, = 0.

The solution is an= n(n+1)/2.

Divide-and-conquer Algorithms

Divide-and-Conquer Concept

To solve a problem of size n, we solve a sub-problems of size n/b, and combine solutions from sub-problems.

XX1 X2 X3 Xa...n/b n/b n/b n/b

n elements

Binary Search

To search for an element a from a sorted list X of size n elements.

• If a=m, then stop

• If a<m, then search for a from X1.

• If a>m, then search for a from X2.

XX1 X2m

Fast multiplication of large integers

a = 2n A1 + A0. b = 2n B1 + B0.

A1 A0 B1 B0

a b = 22nA1B1 + 2n(A0B1+ A1B0) +A0 B0.

a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.

a b = 22n A1B1 + 2nA1B1 + 2nA1 B0 –2nA0B0 +2nA0B1

– 2nA1B1 + 2n A0 B0 + A0 B0.

10010 00011 10000 10011

Recurrence Relations

In Divide-and-conquer algorithms

Divide-and-Conquer Recurrence Relations

Let f(n) be the number of operations required to solve a problem of size n.

f(n) = a f(n/b) + g(n).

XX1 X2 X3 Xa...n/b n/b n/b n/b

n elements

Binary Search

To search for an element a from a sorted list X of size n elements.

• If a=m, then stop

• If a<m, then search for a from X1.

• If a>m, then search for a from X2.

Let f(n) be the # of comparison in binary search within n elements.

f(n) = f(n/2) + 2.

XX1 X2m

Fast multiplication of large integers

a = 2n A1 + A0. b = 2n B1 + B0.

A1 A0 B1 B0

a b = (22n + 2n) A1B1 + 2n(A1-A0)(B0 – B1) + (2n+1) A0 B0.

Let f(n) be the number of operations needed to multiply two n-bit integers.

f(2n) = 3f(n) + Cn.

10010 00011 10000 10011

Theorem

Let f be an increasing function that satisfies the recurrence relation

f(n) = a f(n/b) + c

whenever n is divisible by b, b is an integer greater than 1, a1, c is a positive real number.

When n=bk, f(n) = t nlogba + s, for some t and s.

When a>1, f(n) is O(nlogba ).

When a=1, f(n) is O(log n).

Examples

Binary Search: f(n) = f(n/2) + 2.

Because a=1, f(n) is O(log n).

Find Max. : f(n) = 2f(n/2) + 2.

Because a>1, f(n) is O(nlog11 ) = O(n) .

Master Theorem

Let f be an increasing function that satisfies the recurrence relation

f(n) = a f(n/b) + cnd

whenever n=bk for some positive integer k, b is an integer greater than 1, a1, c is a positive real number, d is a non-negative real number.

When a<bd, f(n) is O(nd ).

When a=bd, f(n) is O(nd log n).

When a>bd, f(n) is O(nlogba).

Examples

Merge sort: f(n) = 2f(n/2) + n.

a = 2, b = 2, c = 1, d = 1

Because a=bd, f(n) is O(n log n).

Integer multiplication: f(n) = 3f(n/2) + Cn

a = 3, b = 2, c = C, d = 1

Because a>bd, f(n) is O(nlog23).