2008. 2. 12 : 1 Chapter 3. Growth of function Chapter 4. Recurrences

한양대학교 정보보호 및 알고리즘 연구실

2008. 2. 12이재준

담당교수님 : 박희진 교수님1

Chapter 3. Growth of functionChapter 4. Recurrences

Contents of Table

1. Technicalities for abbreviation

2. Recurrence solution methodsThe substitution method

2

2. Notation of asymptotically larger/smaller

o-notation ω-notation

3. Relationship between this notations

A. Growth of function

B. Recurrence

f(n) is asymptotically smaller than g(n) f(n) is asymptotically smaller than g(n)

• o-notation

Asymptotically smaller

- Definition for all n, n ≥ n0 , any positive

constant c, c>0, 0 ≤ f(n)< cg(n), the function f(n) is smaller to g(n) to within constant factor.

3

2. Asymptotically larger/smaller notation

f(n)

o(g(n))

n0

4

2. Asymptotically larger/smaller notation

• o-notation

- Definitiono(g(n)) = f(n) : for any positive constant c>0, there exist a constant

n0 >0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0

0)(

)(lim

ng

nf

n

this limit shows the function f(n) becomes insignificant relative to g(n) as n approaches infinity

• o-notation Example

1. Asymptotic notation

Use the definition of o-notation to prove the following property.

f(n) = o(n2)


f(n) = o(n2)

nnf 2)(

6


• o-notation Solution to example

2n = o(n2)

If f(n) = o((g(n)) then

0)(

)(lim

ng

nf

n

f(n) is asymptotically larger than g(n) f(n) is asymptotically larger than g(n)

• ω -notation

Asymptotically larger

- Definition for all n, n ≥ n0 , any positive

constant c, c>0, 0 ≤ cg(n) < f(n), the function f(n) is larger to g(n) to within constant factor.

7


f(n)

ω(g(n))

8


• ω-notation

- Definition ω(g(n)) = f(n) : for any positive constant c>0, there exist a constant

n0 >0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0

)(

)(lim ng

nf

n

if the limit exists. That is, f(n) becomes arbitrarily large relative to g(n) as n approaches infinity.

• ω -notation

- We use ω-notation to denote a lower bound that is not asymptotically tight.

- We use o-notation to denote an upper bound that is not asymptotically tight.

- f(n) ω(∈ g(n)) if and only if g(n) ∈ o(f(n)).

9


10


• ω -notation Example


f(n) = ω(n)


f(n) = ω(n)

2)(

2nnf

11


• ω -notation Solution to example

=ω(n)

If f(n) = ω((g(n)) then

2n

)(

)(lim ng

nf

n

12


f(n)

O(g(n))

o(g(n))

ω(g(n))

Ω(g(n))

θ(g(n))

O(g(n)) Ω(g(n))

ω(g(n)) o(g(n))

13


f(n)

O(g(n))

o(g(n))

ω(g(n))

Ω(g(n))

• Transitivity

• Transpose symmetry

f(n) = Θ(g(n)) and g(n) = Θ(h(n)) imply f(n) = Θ(h(n)) ,f(n) = O(g(n)) and g(n) = O(h(n)) imply f(n) = O(h(n)) , f(n) = Ω(g(n)) and g(n) = Ω(h(n)) imply f(n) = Ω(h(n)) ,f(n) = o(g(n)) and g(n) = o(h(n)) imply f(n) = o(h(n)) ,f(n) = ω(g(n)) and g(n) = ω(h(n)) imply f(n) = ω(h(n)).

f(n) = O(g(n)) if and only if g(n) = Ω(f(n)),f(n) = o(g(n)) if and only if g(n) = ω(f(n)).

14


• Reflexivity

f(n) = Θ(f(n))f(n) = O(f(n))f(n) = Ω(f(n))

• Symmetry

f(n) = Θ(g(n)) if and only if g(n) = Θ(f(n)).

θ(g(n))

O(g(n)) Ω(g(n))

Recurrences

15

• Recurrence

When should we use the recurrence?

When algorithm contains a recursive call to itself, Its running time can often be described by a recurrence.

Recurrences

16

• Recurrence

- Definition Equation or inequality that describes a function in terms

of its value on smaller inputs.

Recurrences

17

• Recurrence Example

Worst-case running time of T(n) could be described by the recurrence

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

Merge-Sort procedure

Show that the solution of T(n) = Θ(n log n)

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Recurrences

18

• Recurrence Solution method to example

- The substitution method

guess a bound and prove our guess correct for small input.

- The recursion-tree method

convert the recurrence into a tree whose nodes represent the costs incurred at various level of recursion

- The master method

determining asymptotic bounds for many simple recurrences of the form

)()/()( nfbnaTnT ( a ≥ 1, b ≥ 1, and f(n) is given function )


• Technicalities

- assumption of integerThe running time T(n) of algorithm is only defined when

n is an integer

- floors / ceilingsWe often omit floors and ceilings

- boundary conditionThe running time of an algorithm on a constant-sized

input is a constant that we typically ignore.


• Technicalities ExampleThe recurrence describing the worst-case running time of Merge-Sort

Omit assumption of integer

Omit boundary condition

)()2/(2

)1()(

nnTnT

if n=1

if n>1

if n=1

if n>1

)()2/()2/(

)1()(

nnTnTnT

)()2/(2)( nnTnT

)()2/(2

)1()(

nnTnT

if n=1

if n>1

Omit floors and ceilings

2. Recurrence solution methods

21

• Substitution Method

- DefinitionWhen recurrence is easy to guess form of the

answer withthe inductive hypothesis is applied to smaller

values,substitution of the guessed answer for the function

.

- It can be used to establish either upper or lower bounds on a recurrence.


22

• Substitution Method Steps

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.

1. Guess the form of the solution

2. Use mathematical induction to find the constant and show that the solution works.


23

• Substitution Method - Making a good guess

prove loose upper and lower bounds on the recurrence

and then reduce the range of uncertainty.

- Guessing a solution takes : experience, occasionally, creativity

- we can use recursion tree


24

• Substitution Method - Subtleties

we can prove something stronger for given value by assuming something stronger for a smaller values.

1)2/()2/(2)( nTnTnTWe guess that the solution is O(n), and show that T(n) ≤ cn

12/2/)( ncncnT

1cnOvercome difficulty by subtracting a lower-order term from our previous guess

bcnnT )(

bcn

bcn

12

( b≥1, c must be chosen large enough )


25

• Substitution Method - Avoid pitfalls

nncnT )2/(2)(

)(

2

nO

ncn

)()2/(2)( nnTnT In recurrence we can falsely prove T(n)= O(n) by guessing T(n) ≤ cn


26

• Substitution Method Example

Let us determine an upper bound on the recurrence


nnTnT )2/(2)(


27

• Substitution Method Solution to example

1. guess that the solution is 2. prove that ( c > 0 ) 1. guess that the solution is 2. prove that ( c > 0 )

)lg()( nnnT ncnnT lg)(

- Assume that this bound holds for ⌊n/2 , ⌋ that is, T (⌊n/2 ) ≤ ⌋ c ⌊n/2 ⌋lg(⌊n/2 ).⌋

T(n) ≤ 2(c ⌊n/2 lg(⌋ ⌊n/2 )) + ⌋ n

≤ cn lg(n/2) + n ( because, ⌊n/2 < ⌋ n/2 )

= cn lg n - cn lg 2 + n = cn lg n - cn + n ≤ cn lg n (as long as c ≥ 1)

We need to find constant c and n0


28


- Find the constant n0

- we can take advantage of asymptotic notation,we can replace base case

T(1) ≤ cn lg n T(1) = 1 but, c1 lg1 = 0

- So, we only have to prove T (n) = cn lg n for n ≥ n0 for n (n0 =2)

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.

Replace T(1) by T(2) and T(3) as the base cases by letting n0 =2.


29


- Find the constant c

- choosing c large enough

T (2) = 4 and T (3) = 5.T (2) = 4 ≤ c ( 2 lg 2 ) T (3) = 5 ≤ c ( 3 lg 3 ).

Any choice of c ≥ 2 suffices for base case of n=2 and n =3 Any choice of c ≥ 2 suffices for base case of n=2 and n =3


30

• Substitution Method Example



nnTnT lg)(2)(


31


- Simplify this recurrence by changing variable

nnTnT lg)(2)(

mTT mm )2(2)2( 2/

mmSmS )2/(2)(

Renaming m = lg n

S(m) = T(2m)

This new recurrence has same solution : S(m) = O(m log m)


32


- Simplify this recurrence by changing variable mmSmS )2/(2)(

This new recurrence has same solution : S(m) = O(m log m)

Changing back from S(m) to T(n)

T(n) = T(2m) = S(m)= O(m lg m)= O( lg n lg(lg n) )

Documents

2008. 2. 12 : 1 Chapter 3. Growth of function Chapter 4. Recurrences