CMPT 438 AlgorithmsChapter 3Asymptotic Notations

Programming for Fun (PFF)When: Tuesday, September 22 3:30PMWhere: RLC 105Team based: one, two or three people per teamLanguages: C/C++ and JavaIDEs: Visual Studio, Eclipse, NetBeans

Interested?

Event Schedule3:30 – 5:30 pm – Contest5:30 pm – Award Ceremony5:30 pm – Pizza Party

Register your team online or in RLC 203 (Find the registration list on the bulletin board)Contact Dr. Tina Tian for questions.

3

Some Simple Summation Formulas• Arithmetic series:

• Geometric series:

Special case: x < 1:

• Harmonic series:

• Other important

formulas:

2

)1( nn

n

k

nk1

...21

11

11

xx

xn

nn

k

k xxxx ...1 2

0

x11

0k

kx

nln

n

k nk1

1...

2

11

1

n

k

k1

lg nn lg

1

1

1

pn

p

n

k

pppp nk1

...21

Mathematical Induction

• InductionUsed to prove a sequence of statements (S(1),

S(2), … S(n)) indexed by positive integers.•Proof:

Basis step: prove that the statement is true for n = 1

Inductive step: assume that S(n) is true and prove that S(n+1) is true for all n ≥ 1

•The key to proving mathematical induction is to find case n “within” case n+1

Example

•Prove that: 2n + 1 ≤ 2n for all n ≥ 3

More Examples

)1(121

2

nnin

i

Asymptotic Notations

•How we indicate running times of algorithms

•Describe the running time of an algorithm as n grows to

• O notation: asymptotic “less than”: f(n)

“≤” g(n)

• notation: asymptotic “greater than”: f(n) “≥”

g(n)

• notation: asymptotic “equality”: f(n) “=” g(n)

Asymptotic notations

• -notation

• Intuitively (g(n)) = the set of

functions with the same order

of growth as g(n)

Examplesn2/2 –n/2 = (n2)

½ n2 - ½ n ≤ ½ n2 n ≥ 0 c2= ½

½ n2 - ½ n ≥ ½ n2 - ½ n * ½ n ( n ≥ 2 ) = ¼ n2

c1= ¼

n ≠ (n2): c1 n2 ≤ n ≤ c2 n2

only holds for: n ≤ 1/c1

10

Examples

6n3 ≠ (n2): c1 n2 ≤ 6n3 ≤ c2 n2

only holds for: n ≤ c2 /6

11

Asymptotic notations (cont.)•O-notation

• Intuitively: O(g(n)) = the

set of functions with a

smaller or same order of

growth as g(n)

Examples

2n2 = O(n3):

n2 = O(n2):

1000n2+1000n = O(n2):

n = O(n2):

2n2 ≤ cn3 2 ≤ cn c = 1 and n0= 2

n2 ≤ cn2 c ≥ 1 c = 1 and n0= 1

• 1000n2+1000n ≤ 1000n2+ 1000n2 = 2000n2 c=2000 and n0 = 1

n ≤ cn2 cn ≥ 1 c = 1 and n0= 1

13

Asymptotic notations (cont.)• - notation

• Intuitively: (g(n)) = the set of

functions with a larger or

same order of growth as g(n)

Examples

5n2 = (n)

100n + 5 ≠ (n2)

n = (2n), n3 = (n2), n = (logn)

c, n0 such that: 0 cn 5n2 cn 5n2 c = 1 and n0 = 1

c, n0 such that: 0 cn2 100n + 5

100n + 5 100n + 5n ( n 1) = 105ncn2 105n n(cn – 105) 0

Since n is positive cn – 105 0 n 105/c

contradiction: n cannot be smaller than a constant

•TheoremFor any two functions f(n) and g(n), we

have f(n) = θ(g(n)) if and only if f(n) = O(g(n)) and f(n) = (g(n))

f(n) = (g(n)) f = O(g(n)) and f = (g(n))

Asymptotic notations (cont.)

•o – notation• Used to denote an upper bound that is not

asymptotically tight

• 2n = o(n2) ?• 2n2 = o(n2) ?

Asymptotic notations (cont.)

•w – notation• Used to denote an lower bound that is not

asymptotically tight

• n2/2= w(n) ?• n2/2 = w(n2) ?

Comparisons of Functions• Transitivity:

f(n) = (g(n)) and g(n) = (h(n)) f(n) = (h(n)) Same for O and

• Reflexivity: f(n) = (f(n)) Same for O and

• Symmetry: f(n) = (g(n)) if and only if g(n) = (f(n))

• Transpose symmetry: f(n) = O(g(n)) if and only if g(n) = (f(n))

Floors and Ceilings

21

Logarithms

• In algorithm analysis we often use the notation

“log n” without specifying the base

nn 2loglg

yxlog

Binary logarithm

)log(logloglog

)(loglog

nn

nn kk

xy log

xylog yx loglog

y

xlog yx loglog

xalog xb ba loglog

abx logxba log

Polylogarithm

•lgkn

•Rule #1: Exponential functions grow faster than polynomial functions.

•Rule #2: Any positive polynomial function grows faster than any polylogarithmic function.

• For each of the following pairs of functions, either f(n) is O(g(n)), f(n) is Ω(g(n)), or f(n) is Θ(g(n)). Determine which relationship is correct.

f(n) = log n2; g(n) = log n + 5f(n) = n; g(n) = log n2

f(n) = log log n; g(n) = log nf(n) = n; g(n) = log2 nf(n) = n log n + n; g(n) = log nf(n) = 10; g(n) = log 10f(n) = 2n; g(n) = 10n2

f(n) = 2n; g(n) = 3n

f(n) = (g(n))

f(n) = (g(n))

f(n) = O(g(n))

f(n) = (g(n))

f(n) = (g(n))

f(n) = (g(n))

f(n) = (g(n))

f(n) = O(g(n))

Readings

•Chapter 4

•Homework 2Programming assignment

Due before the last class O(logn) Algorithm description, run time analysis,

source code and output•Quiz?