Analysis of Algorithms Rate of Growth of functions Prof. Muhammad Saeed

Analysis of AlgorithmsAnalysis of Algorithms

Rate of Growth of Rate of Growth of functionsfunctions

Prof. Muhammad SaeedProf. Muhammad Saeed

Analysis of Algorithms 2

•Rate of Growth of Rate of Growth of functionsfunctions

Function Growth Rate Comparison

0

100

200

300

400

500

600

0 100 200 300 400 500

n

Func

tion

n

sqrt(n)

n^1.5

n^2

nlogn

logn

nloglogn

nlog^2n

nlogn^2

2/n

2^n

2^n/2

n^2logn

n^3


Functions Growth Rate Comparison

-10000

0

10000

20000

30000

40000

50000

0 200 400 600 800 1000 1200

n

Func

tions

nlog(n)

nlog(log(n))

n(logn)^2

nlog(n^2)

n^1.5

n


n n(log(n)) n(log(log(n))) n(log^2(n)) n(log(n^2)) n^1.5 n

1 0 0 0 1 1

1 0 -2 0 1 2 1

2 1 -1 1 2 2 2

2 2 0 2 4 4 2

3 4 1 5 8 6 3

4 7 2 10 13 9 4

6 11 4 20 22 15 6

8 17 6 36 34 23 8

11 26 10 64 53 37 11

15 40 15 109 80 57 15

20 60 22 181 121 90 20

27 90 32 296 179 141 27

37 132 47 475 264 222 37

49 193 67 753 386 348 49

67 281 96 1179 561 546 67

90 406 136 1827 812 856 90

122 584 191 2806 1169 1343 122

164 838 268 4277 1677 2106 164

222 1198 374 6473 2397 3304 222

299 1708 521 9736 3415 5182 299

404 2426 725 14564 4853 8129 404

546 3440 1005 21677 6879 12750 546

737 4865 1391 32117 9729 19999 737

995 6866 1922 47389 13731 31370 995Analysis of Algorithms 5

• Assume N = 100,000 and processor Assume N = 100,000 and processor speed is 1,000,000 operations per speed is 1,000,000 operations per secondsecond

Function Running Time

2N over 100 years

N3 31.7 years

N2 2.8 hours

N*N1/2 31.6 seconds

N log N 1.2 seconds

N 0.1 seconds

N1/2 3.2 x 10-4 seconds

log N 1.2 x 10-5 seconds

Running TimesRunning Times



•Series and Series and AsymptoticsAsymptotics

Series ISeries I


Series IISeries II


Infinite SeriesInfinite Series


Fundamental DefinitionsFundamental DefinitionsAsymptotics Asymptotics

• T(n) = O(f(n))T(n) = O(f(n)) if there are constants c and n if there are constants c and n0 0

such that such that T(n) ≤ cf(n)T(n) ≤ cf(n) when n when n n n00

• T(n) = T(n) = (g(n))(g(n)) if there are constants c and n if there are constants c and n0 0

such that such that T(n) T(n) cg(n) cg(n) when n when n n n00

• T(n) = T(n) = (h(n))(h(n)) if and only if if and only if T(n) = O(h(n))T(n) = O(h(n)) and and T(n) = T(n) = (h(n))(h(n))

• T(n) = o(p(n))T(n) = o(p(n)) if if T(n) = O(p(n))T(n) = O(p(n)) and and

T(n) T(n) (p(n))(p(n)) Analysis of Algorithms 11


T(n) = O(f(n)) T(n) = (g(n)) T(n) = (h(n))

AsymptoticsAsymptotics

Analysis Type

MathematicalExpression

Relative Rates of Growth

Big O T(N) = O( F(N) ) T(N) < F(N)

Big T(N) = ( F(N) ) T(N) > F(N)

Big T(N) = ( F(N) ) T(N) = F(N)

Relative Rates of GrowthRelative Rates of Growth


Relative Growth Rate ofRelative Growth Rate ofTwo FunctionsTwo Functions

Relative Growth Rate ofRelative Growth Rate ofTwo FunctionsTwo Functions

Compute Compute using L’Hopital’s Rule using L’Hopital’s Rule

Limit=0: Limit=0: f(n)=o(g(n))f(n)=o(g(n)) Limit=cLimit=c0:0: f(n)=f(n)=(g(n))(g(n)) Limit=Limit=:: g(n)=o(f(n))g(n)=o(f(n))


3n3 = O(n3)3n3 + 8 = O(n3)8n2 + 10n * log(n) + 100n + 1020 = O(n2)3log(n) + 2n1/2 = O(n1/2)2100 = O(1)TlinearSearch(n) = O(n)TbinarySearch(n) = O(log(n))

Important RulesImportant Rules


Important RulesImportant Rules

Rule 1:Rule 1:

If TIf T11(n) = O(f(n)) and T(n) = O(f(n)) and T22(n) = O(g(n)), then(n) = O(g(n)), then

a)a) T T11(n) + T(n) + T22(n) = max(O(f(n)), O(g(n)))(n) = max(O(f(n)), O(g(n)))

b)b) T T11(n) * T(n) * T22(n) = O(f(n)*g(n))(n) = O(f(n)*g(n))

Rule 2:Rule 2:

If T(x) is a polynomial of degree n, then If T(x) is a polynomial of degree n, then T(x)=T(x)=(x(xnn))

Rule 3:Rule 3:

loglogkk n = O(n) for any constant k. n = O(n) for any constant k.Analysis of Algorithms 16

General RulesGeneral Rulesforfor

• LoopsLoops

• Nested LoopsNested Loops

• Consecutive statementsConsecutive statements

• if-then-elseif-then-else

• RecursionRecursion



int gcd(int a, int b) int gcd(int a, int b) { {

int t; int t; while (b != 0) while (b != 0) { {

t = b; t = b; b = a % b; b = a % b; a = t; a = t;

} } return a; return a;

} }

Euclid’s GCDEuclid’s GCD


Binary SearchBinary Search

function BinarySearch(a, value, left, right) function BinarySearch(a, value, left, right) whilewhile left ≤ right left ≤ right

mid := floor((right+left)/2) mid := floor((right+left)/2) if a[mid] = value if a[mid] = value

return mid return mid if value < a[mid] if value < a[mid]

right := mid-1 right := mid-1 else else

left := mid+1left := mid+1endwhileendwhile return not found return not found


• Insertion SortInsertion Sort

A caseA case


Insertion Sort

Best Case: T(n)=O(n)Best Case: T(n)=O(n)

Worst Case: T(n)=Worst Case: T(n)= O(n2)


•Maximum Maximum Subsequence SumSubsequence Sum

A caseA case

int int MaxSubsequenceSum( const int A[], const unsigned int N) MaxSubsequenceSum( const int A[], const unsigned int N) {{

int Sum=0, MaxSum=0;int Sum=0, MaxSum=0;for( for( i = 0; i<N; i++)i = 0; i<N; i++)

for(for( j = i; j<N; j++) j = i; j<N; j++){{

Sum = 0; Sum = 0; for(for( k = i; k<=j; k++) k = i; k<=j; k++)

Sum += A[ k ]; Sum += A[ k ];

if (Sum > MaxSum) if (Sum > MaxSum) MaxSum = Sum; MaxSum = Sum;

} } return MaxSum;return MaxSum;

}}

Maximum Subsequence SumMaximum Subsequence SumAlgorithm 1Algorithm 1

Analysis of AlgorithmsAnalysis of Algorithms 2323



int Sum=0, MaxSum=0;int Sum=0, MaxSum=0;for( for( i = 0; i<N; i++)i = 0; i<N; i++)

Sum = 0; Sum = 0; for(for( j = i; j<N; j++) j = i; j<N; j++){{

Sum += A[ k ]; Sum += A[ k ];


} } return MaxSum;return MaxSum;

}}




int Sum = 0, MaxSum = 0, Start = 0, End = 0;int Sum = 0, MaxSum = 0, Start = 0, End = 0;for( for( End = 0; End<N; End++End = 0; End<N; End++)){{

Sum += A[ End ]; Sum += A[ End ];


elseelseif (Sum < 0)if (Sum < 0){{

Start= end+1;Start= end+1;Sum=0;Sum=0;

}} } } return MaxSum;return MaxSum;

}} Analysis of Algorithms 25

END


Documents

Analysis of Algorithms Rate of Growth of functions Prof. Muhammad Saeed