Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

UCT Algorithm Circle: Basic AsymptoticAnalysis

Kosie van der Merwe

10 March 2011

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

What is Asymptotic Analysis?

Algorithms take time to run.The time an algorithm runs in is affected by:

What algorithm it is.How it’s implemented.The hardware it’s run on.The input.

Note: even algorithms that do the exact same thing canrun in differing times.We use Asymptotic Analysis to get a rough idea of theperformance of an algorithm as we change the size of theinput.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Why should we care?

Why should we care how fast an algorithm runs?Shouldn’t faster hardware make having to use a fasteralgorithm unnecessary?No, and for a few reasons:

The algorithm might need to be fast right now.Faster hardware only gives a constant speedup.A faster algorithm can give more than a constant speedup.

So a faster algorithm is almost always better.Now we need to be able to tell what algorithm is better.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

How do we calculate time?

We could calculate time as a function of the input inseconds . . .But this has a major problem.Namely, the running time is affected by implementation,language and hardware.So we need a way to capture the gist of the running time.We do this by using an abstract model of a computer.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Our abstract computer

This abstract computer has one main property:All instructions take the same amount of time.

Then to calculate the running time we count the numberinstructions.We just need to be careful that we don’t count functions asa single instruction.Finally, we want to find the worst case running time, for agiven size of inputs.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

First Example

Consider the following code:

a = Na = a * 5a = a + 1

There are 3 assignments, 1 multiplication and 1 addition.This means the total running time is

T1(N) = 3 + 1 + 1 = 5

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Second Example

Consider the following code:

a = 0for i=1 to N:

a = a + 1

First line has 1 assignment and is run once.The second line does 1 assignment every loop. So it doesN assignments total.Similarly the third line does N additions and Nassignments.This yields a total running time of

T2(N) = 1 + N + N + N = 3N + 1

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Third Example

Consider the following code:

a = 0for i=1 to N:

for j=i to N:a = a + 1

The first and second line do 1 and N operationsrespectively.The third line doesN + (N − 1) + (N − 2) + · · · + 2 + 1 = N(N+1)

2 assignments.Similarly, the last line does2(N + (N − 1) + (N − 2) + · · · + 2 + 1) = N(N + 1)operations.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Third Example

This yields a total running time of

T3(N) = 1 + N +N(N + 1)

2+ N(N + 1) =

32

N2 +52

N + 1

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Final Example

Consider this function to search an array:

function search(arr, val):for i=0 to length(arr)-1:

if arr[i] == val:return i

return -1

In the best case you only have to check the first element.But we only care about the worst case.In the worst case we’d check all N elements and still notfind val .So we get

T4(N) = 3 + N + N + N + 1 = 3N + 4

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Why Big O?

Calculating the “exact” time has a few problems:It quickly becomes messy.Time also varies per implementation.It does not quickly give you the gist of the behaviour.

This is why we use Big O notation.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Big O?

Big O is related to the worst case running time.With Big O Notation you discard:

Constant factors.Lower order terms.

So we’d write Big O’s of the examples as:O(T1(N)) = O(5) = O(1)

O(T2(N)) = O(3N + 1) = O(3N) = O(N)

O(T3(N)) = O( 32 N2 + 5

2 N + 1) = O( 32 N2) = O(N2)

O(T4(N)) = O(3N + 4) = O(N)

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Why lower order terms are unimportant

Compare the graph of N2 + 7N (blue) with N2 (red), notethe difference is negligible:

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Example Big O’s

In the order from fastest to slowest, with alternate names inbrackets:

O(1) (Constant time)O(log N) (Logarithmic time)O(N) (Linear time)O(N log N)O(N2) (Quadratic time)O(N3) (Cubic time)

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Alternate notation

Big O is actually just an upper bound.For our linear search, O(N4), O(NN) and O(N log N) areall valid Big O’s as our function will not run slower than this.In academia some other notation is also used:

Big Theta (Θ(f (N))): Is an exact function representing theworst case running time. So for the linear search this isΘ(N).Big Omega (Ω(f (N))): Is a lower bound on the worst caserunning time. For the linear search Ω(1), Ω(log N) andΩ(N) are all valid Big Omegas.

For Big O and Big Omega we want to try and get thebounds as tight as possible.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Some final points

You are not restricted to only using Big O on time.Sometimes, Big O is also used to analyse memory usageJust a few things to watch out for:

Use the easiest algorithm that is fast enough for your inputs.For small inputs, don’t ignore the constant factors.

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Outline

1 Introduction

2 Calculating time

3 Big O Notation

4 Alternate notation

5 Conclusion

6 Problems

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Question 1 - What’s the big O of the following?

function fib(N):prev = 0cur = 1for i = 1 to N:

temp = curcur = cur + prevprev = temp

O(N)

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Question 2 - What’s the big O of the following?

function sum(arr, cur = 0):if length(arr) == cur:

return 0return arr[cur] + sum(arr, cur+1)

O(N)

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Question 3 - What’s the big O of the following?

function sort(arr):for i = 0 to length(arr)-1:

mini = 1for j = i to length(arr)-1:if arr[mini] > arr[j]:

mini = jt = arr[mini]arr[mini] = arr[i]arr[i] = t

O(N2)

Kosie van der Merwe Basic Asymptotic Analysis

Introduction Calculating time Big O Notation Alternate notation Conclusion Problems

Question 4 - What’s the big O of the following?

function min(arr):res = []for i = 0 to length(arr)-1:

j = length(res)-1while j >= 0 and res[j] > arr[i]:del res[j]j = j-1

res.append(arr[i])return res[0]

O(N)Because each element in arr only gets added to andpossibly removed from res at most once.Note: deleting from an arbitrary position in an array isnormally O(N). But from the end it’s O(1).

Kosie van der Merwe Basic Asymptotic Analysis