LECTURE 3 Asymptotic Notation Basic Data Structuresads22/courses/cse565/F08/www/lec...• Asymptotic...

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Adam Smith

Algorithm Design and Analysis

LECTURE 3 • Asymptotic Notation • Basic Data Structures

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Asymptotic notation

We write f(n) = O(g(n)) if there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0.

O-notation (upper bounds):

EXAMPLE: 2n2 = O(n3)

functions, not values

(c = 1, n0 = 2) funny, “one-way” equality

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Set Definition

O(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ f(n) ≤ cg(n) for all n ≥ n0 }

EXAMPLE: 2n2 ∈ O(n3) (Logicians: λn.2n2 ∈ O(λn.n3), but it’s convenient to be sloppy, as long as we understand what’s really going on.)

Examples

•  106 n3+ 2n2 -n +10 = O(n3)

•  n½ + log(n) = O(n½)

•  n (log(n) + n½) = O(n3/2)

•  n = O(n2)

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Ω-notation (lower bounds)

Ω(g(n)) = { f(n) : there exist constants c > 0, n0 > 0 such that 0 ≤ cg(n) ≤ f(n) for all n ≥ n0 }

EXAMPLE: (c = 1, n0 = 16)

O-notation is an upper-bound notation. It makes no sense to say f(n) is at least O(n2).

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Ω-notation (lower bounds) • Be careful: “Any comparison-based sorting algorithm requires at least O(n log n) comparisons.”

– Meaningless! – Use Ω for lower bounds.

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Θ-notation (tight bounds)

Θ(g(n)) = O(g(n)) ∩ Ω(g(n))

EXAMPLE:

Polynomials are simple: ad nd + ad–1nd–1 + + a1n + a0 = Θ(nd)

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

o-notation and ω-notation

o(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ f(n) < cg(n) for all n ≥ n0 }

EXAMPLE: (n0 = 2/c)

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.

2n2 = o(n3)

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

o-notation and ω-notation

ω(g(n)) = { f(n) : for any constant c > 0, there is a constant n0 > 0 such that 0 ≤ cg(n) < f(n) for all n ≥ n0 }

EXAMPLE: (n0 = 1+1/c)

O-notation and Ω-notation are like ≤ and ≥. o-notation and ω-notation are like < and >.

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Common Functions: Asymptotic Bounds

• Polynomials. a0 + a1n + … + adnd is Θ(nd) if ad > 0. • Polynomial time. Running time is O(nd) for some constant d independent of the input size n. • Logarithms. log a n = Θ(log b n) for all constants a, b > 0.

For every x > 0, log n = O(nx).

• Exponentials. For all r >1 and all d > 0, nd = O(rn). • Factorial.

grows faster than every exponential

can avoid specifying the base log grows slower than every polynomial

Every exponential grows faster than every polynomial

n! = (!

2!n)!n

e

"n(1 + o(1)) = 2!(n log n)

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Sort by asymptotic order of growth

a)  n log(n) b)  c)  log(n) d)  n2 e)  2n f)  n g)  n! h)  n1,000,000

i)  n1/log(n) j)  log(n!)

1.  n1/log(n) 2.  log(n) 3.  4.  n 5.  n log(n) = Θ(log(n!)) 6.  7.  n2 8.  n1,000,000

9.  2n 10.  n!

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Conventions for formulas

Convention: A set in a formula represents an anonymous function in the set.

f(n) = n3 + O(n2) means f(n) = n3 + h(n) for some h(n) ∈ O(n2) .

EXAMPLE: (right-hand side)

9/15/08 S. Raskhodnikova and A. Smith. Based on notes by E. Demaine and C. Leiserson

Convention for formulas

Convention: A set in a formula represents an anonymous function in the set.

n2 + O(n) = O(n2) means for any f(n) ∈ O(n): n2 + f(n) = h(n) for some h(n) ∈ O(n2) .

EXAMPLE: (left-hand side)

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Properties

• Transitivity. – If f = O(g) and g = O(h) then f = O(h). – If f = Ω(g) and g = Ω(h) then f = Ω(h). – If f = Θ(g) and g = Θ(h) then f = Θ(h).

• Additivity. – If f = O(h) and g = O(h) then f + g = O(h). – If f = Ω(h) and g = Ω(h) then f + g = Ω(h). – If f = Θ(h) and g = O(h) then f + g = Θ(h).

Exercise: Show that log(n!) = Θ(n log n)

•  Stirling’s formula:

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

n! = (!

2!n)!n

e

"n(1 + o(1))

log(n!) = log(!

2!n) + n log(n)" n log(e) + log(1 + o(1))! "# $=log(1)+o(1)

since log is continuous

= n%

log(n)" log(e) +log(2!n)

n! "# $constant+o(1)

&+ o(1)

= n(log(n)"O(1)) + o(1)= n log n(1"O( 1

log n )) + o(1)

= n log n(1± o(1)) = !(n log n)

8/29/2008 A. Smith; based on slides by E. Demaine, C. Leiserson, S. Raskhodnikova, K. Wayne

Overview Notation … means … Think… E.g. Lim f(n)/g(n)

f(n)=O(n) 9 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)

Upper bound

100n2 = O(n3)

If it exists, it is < 1

f(n)=Ω(g(n)) 9 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)

Lower bound

n100 = Ω(2n)

If it exists, it is > 0

f(n)=Θ(g(n)) both of the above: f=Ω(g) and f = O(g)

Tight bound log(n!) = Θ(n log n)

If it exists, it is > 0 and < 1

f(n)=o(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · f(n) < cg(n)

Strict upper bound

n2 = o(2n) Limit exists, =0

f(n)=ω(g(n)) 8 c>0, n0>0, 8 n > n0 : 0 · cg(n) < f(n)

Strict lower bound

n2 = ω(log n)

Limit exists, =1