O-Notation

April 23, 2003Prepared by Doug Hogan

CSE 260

O-notation: The Idea

Big-O notation is a way of ranking about how much time it takes for an algorithm to executeHow many operations will be done when the program is executed? Find a bound on the running time, i.e. functions that are on the same order. We care about what happens for large

amounts of data asymptotic order.

O-notation: The Idea

Use mathematical tools to find asymptotic order. Real functions to approximate integer

functions. Depends on some variable, like n or

X, which is usually the size of an array or how much data is going to be processed

O-notationThe complicated math behind it all…

Given f and g, real functions of variable x…First form: g provides an upper bound for f ≡

graph of f lies closer to the x axis than g

More general form: g provides an upper bound for f ≡

graph of f lies closer to the x axis than some positive multiple (M) of g after some minimum value of x (x0).

O-notation: A graphical view

So what does “closer to the x-axis” MEAN?

-M ∙ g(x) ≤ f(x) ≤ M ∙ g(x)

But that’s absolute value…

|f(x)| ≤ M ∙ |g(x)|

Another graphical view

x0

g

M ∙g

f

x

y

Before x0, nothing claimed about f’s growth

After x0, |f(x)| ≤ M ∙ |g(x)|

Formal Definition

Let f and g be real-valued functions defined on the same set of reals. f is of order g, written f(x) = O(g(x)), iff there exists a positive real number M (multiple) a real number x0 (starting point)

such that for all x in the domain of f and g, |f(x)| ≤ M ∙ |g(x)| when x > x0

Example

Use the definition of O-notation to express|17x6 – 3x3 + 2x + 8| ≤ 30|x6| for all x > 1

M = 30x0 = 117x6 – 3x3 + 2x + 8 is O(x6)

Graphically…17x6 – 3x3 + 2x + 8 is O(x6); M = 30; x0 = 1

30x6

x6

17x6 – 3x3 + 2x + 8

Problem

Use the definition of O-notation to express for all x > 6

M = 45x0 = 6

x

x

xx45

1

9215

xOxx

xx is 45

1

9215

Graphically…M = 45; x0 = 6 xOx

x

xx is 45

1

9215

1

9215

x

xx

x45

x

Another graphical example

f(x) = 7x3 - 2x + 3

12x3

x3

M = 12, x0 = 1 7x3 - 2x + 3 is O(x3)

Using O-notation…

Order of Power Functions:For any rational numbers r and s, if r < s,

xr is O(xs)

Order of Polynomial Functions: If a0, a1,…, an are real numbers and an ≠ 0

anxn + an-1xn-1 +… + a1x + a0 is O(xm)

for all m ≥ n

Examples

Example: Find an order for

f(x) = 7x5 + 5x3 – x + 4 (all reals x)

O(x5)

Is that the only answer?No…But it’s the “best”

Showing that a function is NOT Big-O of another…

Show that x2 is not O(x).[Arguing by contradiction.] Suppose not, that x2 is O(x). By definition of O(…), then there exista positive real number Ma real number x0

such that |x2| ≤ M ∙ |x| for all x > x0 (1)

Showing that a function is NOT Big-O of another…,

ctd.Let x be a positive real number greater than both M and x0, i.e. x>M and x>x0.Then by multiplying both sides of x>M by x, x∙x>M∙x.Since x is positive, |x2|>M∙|x|. So there is a real number x>x0 s.t. |x2|>M∙|x|. This contradicts (1) above. So, the supposition is false and thus x2 is not O(x). □

Generalization

If a0, a1,…, an are real numbers and an ≠ 0

anxn + an-1xn-1 +… + a1x + a0 is NOT

O(xm) for all m < n

Best approximation

Definition Suppose S is a set of functions from a subset of R to R and f is a R->R function.Function g is a best big-O approximation for f in S iff f(x) is O(g(x)) for any h in S, if f(x) is O(h(x)), then g(x) is

O(h(x)).

Problem

Find best big-O approx for f(x) = 5x3 – 2x + 1By thm. on polynomial orders,f(x) is O(xn) for all n ≥ 3

By previous property, f(x) is NOT O(xm) for all m < 3

So O(x3) is the best approximation.

O-Arithmetic

Let f and g be functions and k be a constant.

1. O(k*f) = O(f)2. O(f *g) = O(f) * O(g)3. O(f/g) = O(f) / O(g)4. O(f) ≥ O(g) iff f dominates* g5. O(f + g) = Max[O(f), O(g)]

(Headington 546)

DominanceLet f and g be functions and a,b,m,n be

constants. 1. xx dominates x!2. x! dominates ax

3. ax dominates bx if a > b4. ax dominates xn if n > m5. x dominates logax if a > 1

6. logax dominates logbx if b > a > 1

7. logax dominates 1 if a > 1(Headington 547)

Works Cited

Epp, Susanna. Discrete Mathematics with Applications. 2nd Ed. Belmont, CA: Brooks, 1995.

Headington, Mark A., and David Riley. Data Abstraction and Structures using C++. Lexington, MA: Heath, 1994.