L-03 Complexity Analysis 2.ppt

7/29/2019 L-03 Complexity Analysis 2.ppt

http://slidepdf.com/reader/full/l-03-complexity-analysis-2ppt 1/24

Data Structures (FALL-2011)

Program Efficiency

&

Complexity Analysis




Efficient Algorithm

A program that looks up a specific entry in a sorted list of size

n. Suppose this program were implemented on Computer A, a

state-of-the-art machine, using a linear search algorithm, and

on Computer B, a much slower machine, using a binary search

algorithm. Benchmark testing on the two computers running

their respective programs might look something like the

following:

http://en.wikipedia.org/wiki/Collation

http://en.wikipedia.org/wiki/List_%28computing%29

http://en.wikipedia.org/wiki/Linear_search

http://en.wikipedia.org/wiki/Binary_search

http://en.wikipedia.org/wiki/Benchmark_%28computing%29

http://en.wikipedia.org/wiki/Benchmark_%28computing%29

http://en.wikipedia.org/wiki/Binary_search

http://en.wikipedia.org/wiki/Linear_search

http://en.wikipedia.org/wiki/List_%28computing%29

http://en.wikipedia.org/wiki/Collation




n (list size)Computer A run-time

(in nanoseconds)

Computer B run-time

(in nanoseconds)

15 7 ns 100,000 ns

65 32 ns 150,000 ns

250 125 ns 200,000 ns

1,000 500 ns 250,000 ns

http://en.wikipedia.org/wiki/Nanosecond







Based on these metrics, it would be easy to

jump to the conclusion that Computer A is

running an algorithm that is far superior in

efficiency to that of Computer B. However, if

the size of the input-list is increased to a

sufficient number, that conclusion is

dramatically demonstrated to be in error:




n

(list size)

Computer A run-time

(in nanoseconds)

Computer B run-time

(in nanoseconds)

15 7 ns 100,000 ns

65 32 ns 150,000 ns

250 125 ns 200,000 ns

1,000 500 ns 250,000 ns

... ... ...

1,000,000 500,000 ns 500,000 ns

4,000,000 2,000,000 ns 550,000 ns

16,000,000 8,000,000 ns 600,000 ns

... ... ...

63,072 × 1012 31,536 × 1012 ns,

or 1 year

1,375,000 ns,

or 1.375 milliseconds










Big Oh Notation [1]

If f(N) and g(N) are two complexity functions, we say

f(N) = O(g(N))

(read "f(N) is order g(N)", or "f(N) is big-O of g(N)")

if there are constants c and N0 such that

f(N) ≤ c * g(N)

for all sufficiently large N.




Example (1)

• Consider

f(n)=2n2+3

and g(n)=n2

Is f(n)=O(g(n))? i.e. Is 2n2+3 = O(n2)?

Proof:

2n2+3 ≤ c * n2

Assume N0 =1 and c=1?

Assume N0

=1 and c=2?

Assume N0 =1 and c=3?




Comparing Functions

• As inputs get larger, any algorithm of a smaller

order will be more efficient than an algorithm

of a larger order

T

i m e ( s t e p s )

Input (size)

3N = O(N)

0.05 N2 = O(N2)

N = 60




Big-Oh Notation

• Even though it is correct to say “7n - 3 is O(n3)”,

a better statement is “7n - 3 is O(n)”, that is, one

should make the approximation as tight as possible

• Simple Rule:

Drop lower order terms and constant factors

7n-3 is O(n)

8n2log n + 5n2 + n is O(n2log n)




Asymptotic notation




Size does matter[1]

What happens if we double the input size N?

N log2N 5N N log2N N2 2N

8 3 40 24 64 256

16 4 80 64 256 65536

32 5 160 160 1024 ~109

64 6 320 384 4096 ~1019

128 7 640 896 16384 ~1038

256 8 1280 2048 65536 ~1076




Size does matter[2]

• Suppose a program has run time O(n!) and therun time for

n = 10 is 1 second

For n = 12, the run time is 2 minutes

For n = 14, the run time is 6 hours

For n = 16, the run time is 2 monthsFor n = 18, the run time is 50 years

For n = 20, the run time is 200 centuries




Standard Analysis Techniques

• Constant time statements

• Analyzing Loops

• Analyzing Nested Loops

• Analyzing Sequence of Statements

• Analyzing Conditional Statements

D S (FA 2011)




Constant time statements

• Simplest case: O(1) time statements

• Assignment statements of simple data typesint x = y;

• Arithmetic operations:x = 5 * y + 4 - z;

• Array referencing:A[j] = 5;

• Array assignment:

j, A[j] = 5;• Most conditional tests:

if (x < 12) ...

D t St t (FALL 2011)




Analyzing Loops[1]

• Any loop has two parts: – How many iterations are performed?

– How many steps per iteration?

int sum = 0,j;for (j=0; j < N; j++)

sum = sum +j;

– Loop executes N times (0..N-1) – 4 = O(1) steps per iteration

• Total time is N * O(1) = O(N*1) = O(N)





Analyzing Loops[2]

• What about this for loop?

int sum =0, j;

for (j=0; j < 100; j++)

sum = sum +j;

• Loop executes 100 times

• 4 = O(1) steps per iteration

• Total time is 100 * O(1) = O(100 * 1) = O(100)= O(1)





Analyzing Nested Loops[1]

• Treat just like a single loop and evaluate each level of nesting as needed:

int j,k;

for (j=0; j<N; j++)

for (k=N; k>0; k--)sum += k+j;

• Start with outer loop: – How many iterations? N

– How much time per iteration? Need to evaluate inner loop• Inner loop uses O(N) time

• Total time is N * O(N) = O(N*N) = O(N2)





Analyzing Nested Loops[2]

• What if the number of iterations of one loopdepends on the counter of the other?

int j,k;

for (j=0; j < N; j++)for (k=0; k < j; k++)

sum += k+j;

• Analyze inner and outer loop together:• Number of iterations of the inner loop is:

• 0 + 1 + 2 + ... + (N-1) = O(N2)

Data Str ct res (FALL 2011)




Analyzing Sequence of Statements

• For a sequence of statements, compute their complexity functions individually and addthem up

for (j=0; j < N; j++)for (k =0; k < j; k++)

sum = sum + j*k;

for (l=0; l < N; l++)

sum = sum -l;cout<<“Sum=”<<sum;

Total cost is O(N2) + O(N) +O(1) = O(N2)SUM RULE

O(N2

)

O(N)

O(1)

Data Structures (FALL 2011)




Analyzing Conditional Statements

What about conditional statements such as

if (condition)

statement1;

elsestatement2;

where statement1 runs in O(N) time and statement2 runs inO(N2) time?

We use "worst case" complexity: among all inputs of size N, thatis the maximum running time?

The analysis for the example above is O(N2)





int CompareSmallestNumber (int array[ ])

{

int x, curMin;

// set smallest value to first item in array

curMin = array[0];

// iterate through array to find smallest valuefor (x = 1; x < 10; x++)

{

if( array[x] < curMin)

{ curMin = array[x]; }}

// return smallest value in the array return curMin;

}





int CompareToAllNumbers (int array[ ])

{

bool is Min;

int x, y;

// iterate through each

for (int x = 0; x < 10; x++)

{ isMin = true;

for (int y = 0; y < 10; y++)

{/* compare the value in array[x] to the other values if we find that array[x]

is greater than any of the values in array[y] then we know that the value in array[x] is notthe minimum

remember that the 2 arrays are exactly the same, we are just taking out one

value with index 'x' and comparing to the other values in the array with index 'y' */

if( array[x] > array[y])

isMin = false;

}

if(isMin)

break;

}

return array[x];

}





Exercise

• f(n) = 3x2 + 2x + 2

• f(n) = 10n + 5n2 + n3

• f(n) = 10n + 5 and g(n) = n

SHOW f(n) is O(g(n))• f(n)=10n3+6n2+5

Documents

L-03 Complexity Analysis 2.ppt