CS 2133: Data Structures Mathematics Review and Asymptotic Notation

• View
220

4

Embed Size (px)

Text of CS 2133: Data Structures Mathematics Review and Asymptotic Notation

• CS 2133: Data StructuresMathematics Review andAsymptotic Notation

• Arithmetic Series Review1 + 2 + 3 + . . . + n = ?Sn = a + a+d + a+2d + a+3d + . . . a+(n-1)dSn= 2a+(n-1)d + 2a+(n-2)d + 2a+(n-3)d + + a

2Sn = 2a+(n-1)d + 2a+(n-1)d + 2a+(n-1)d . . . + 2a+(n-1)dSn = n/2[2a + (n-1)d]

consequently 1+2+3++n = n(n+1)/2

• ProblemsFind the sum of the following

1+3+5+ . . . + 121 = ?

The first 50 terms of -3 + 3 + 9 + 15 +

1 + 3/2 + 2 + 5/2 + . . . 25=?

• Geometric Series Review1 + 2 + 4 + 8 + . . . + 2n 1 + 1/2 + 1/4 + . . . + 2-nTheorem: Sn= a + ar + ar2 + . . . + arnrSn= ar + ar2 + . . . + arn + arn+1

Sn-rSn = a - arn+1 What about the case where -1< r < 1 ?

• Geometric ProblemsWhat is the sum of 3+9/4 + 27/16 + . . .

1/2 - 1/4 + 1/8 - 1/16 + . . .

• Harmonic SeriesThis is Eulers constants

• Just an Interesting QuestionWhat is the optimal base to use in the representation of numbers n?

Example: with base x we have _ _ _ _ _ _ _ _ _

X valuesslotsWe minimize

• Logarithm ReviewLn = loge is called the natural logarithmLg = log2 is called the binary logarithm

How many bits are required to represent the number n in binary

• Logarithm RulesThe logarithm to the base b of x denoted logbx is defined to that number y such that by = xlogbx > 0 if x > 1logbx = 0 if x = 1logbx < 0 if 0 < x < 1logb(x1*x2) = logb x1 + logb x2

logb(x1/x2) = logb x1 - logb x2

logb xc = c logbx

• Additional RulesFor all real a>0, b>0 , c>0 and n

logb a = logca / logc b logb (1/a) = - logb a

logb a = 1/ logab a logb n = n logb a

• Asymptotic PerformanceIn this course, we care most about the asymptotic performance of an algorithm.How does the algorithm behave as the problem size gets very large?Running timeMemory requirementsComing up: Asymptotic performance of two search algorithms,A formal introduction to asymptotic notation

• Input SizeTime and space complexityThis is generally a function of the input sizeE.g., sorting, multiplicationHow we characterize input size depends:Sorting: number of input itemsMultiplication: total number of bitsGraph algorithms: number of nodes & edgesEtc

• Running TimeNumber of primitive steps that are executedExcept for time of executing a function call most statements roughly require the same amount of timey = m * x + bc = 5 / 9 * (t - 32 )z = f(x) + g(y)We can be more exact if need be

• AnalysisWorst caseProvides an upper bound on running timeAn absolute guaranteeAverage caseProvides the expected running timeVery useful, but treat with care: what is average?Random (equally likely) inputsReal-life inputs

• An Example: Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}

• Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}What is the preconditionfor this loop?

• Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}How many times will this loop execute?

• Insertion SortStatement EffortInsertionSort(A, n) {for i = 2 to n { c1nkey = A[i]c2(n-1)j = i - 1;c3(n-1)while (j > 0) and (A[j] > key) {c4TA[j+1] = A[j]c5(T-(n-1))j = j - 1c6(T-(n-1))}0A[j+1] = keyc7(n-1)}0}T = t2 + t3 + + tn where ti is number of while expression evaluations for the ith for loop iteration

• Analyzing Insertion SortT(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10What can T be?Best case -- inner loop body never executedti = 1 T(n) is a linear functionWorst case -- inner loop body executed for all previous elementsti = i T(n) is a quadratic functionT=1+2+3+4+ . . . n-1 + n = n(n+1)/2

• AnalysisSimplificationsIgnore actual and abstract statement costsOrder of growth is the interesting measure:Highest-order term is what countsRemember, we are doing asymptotic analysisAs the input size grows larger it is the high order term that dominates

• Upper Bound NotationWe say InsertionSorts run time is O(n2)Properly we should say run time is in O(n2)Read O as Big-O (youll also hear it as order)In general a functionf(n) is O(g(n)) if there exist positive constants c and n0 such that f(n) c g(n) for all n n0FormallyO(g(n)) = { f(n): positive constants c and n0 such that f(n) c g(n) n n0

• Big O exampleShow using the definition that 5n+4 O(n)Where g(n)=nFirst we must find a c and an n0We now need to show that f(n) c g(n) for every n n0

clearly 5n + 5 6n whenever n 6

Hence c=6 and n0=6 satisfy the requirements.

• Insertion Sort Is O(n2)ProofSuppose runtime is an2 + bn + c If any of a, b, and c are less than 0 replace the constant with its absolute valuean2 + bn + c (a + b + c)n2 + (a + b + c)n + (a + b + c) 3(a + b + c)n2 for n 1Let c = 3(a + b + c) and let n0 = 1QuestionIs InsertionSort O(n3)?Is InsertionSort O(n)?

• Big O FactA polynomial of degree k is O(nk)Proof:Suppose f(n) = bknk + bk-1nk-1 + + b1n + b0Let ai = | bi |f(n) aknk + ak-1nk-1 + + a1n + a0

• Lower Bound NotationWe say InsertionSorts run time is (n)In general a functionf(n) is (g(n)) if positive constants c and n0 such that 0 cg(n) f(n) n n0Proof:Suppose run time is an + bAssume a and b are positive (what if b is negative?)an an + b

• Asymptotic Tight BoundA function f(n) is (g(n)) if positive constants c1, c2, and n0 such that c1 g(n) f(n) c2 g(n) n n0

Theoremf(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))Proof: someday

• Notation(g) is the set of all functions f such that there exist positive constants c1, c2, and n0 such that

0 c1g(n) f(n) c2 g(n) for every n > nc

c1g(n)f(n)c2 g(n)

• Growth Rate Theorems1. The power n is in O(n) iff (with ,>0) and n is in o(n) iff

2. logbn o(n ) for any b and

3. n o(cn) for any >0 and c>1

4. logbn O(logbn) for any a and b

5. cn O(dn) iff cd and cn o(dn) iff c

• Big O Relationships1. o(f) O(f)

2. If fo(g) then O(f) o(g)

3. If f O(g) then o(f) o(g)

4. If f O(g) then f(n) + g(n) O(g)

5. If f O(f `) and g O(g`) then

f(n)* g(n) O(f `(n) * g`(n))

• Theorem: log(n!)(nlogn)Case 1 nlogn O(log(n!))log(n!) = log(n*(n-1)*(n-2) * * * 3*2*1)= log(n*(n-1)*(n-2)**n/2*(n/2-1)* * 2*1=> log(n/2*n/2* * * n/2*1 *1*1* * * 1)= log(n/2)n/2 = n/2 log n/2 O(nlogn)Case 2 log(n!) O(nlogn)log(n!) = logn + log(n-1) + log(n-2) + . . . Log(2) + log(1)< log n + log n + log n . . . + log n= nlogn

• The Little o Theorem: If log(f)o(log(g)) andlim g(n) =inf as n goes to inf then f o(g)Note the above theorem does not apply to big O for log(n2) O(log n) but n2 O(n)Application: Show that 2n o(nn)Taking the log of functions we have log(2n)=nlog22and log( nn) = nlog2n.

Hence

Implies that 2n o(nn)

• Theorem:

• Practical Complexity

Chart3

100112

212484

31.58496250074.75488750229278

428166416

52.321928094911.60964047442512532

62.584962500715.50977500433621664

72.807354922119.651484454449343128

832464512256

93.169925001428.52932501381729512

103.321928094933.219280948910010001024

113.459431618638.05374780512113312048

123.584962500743.019550008714417284096

133.700439718148.105716335816921978192

143.807354922153.3029689088196274416384

153.906890595658.6033589341225337532768

16464256409665536

174.087462841369.48686830132894913131072

184.169925001475.0586500263245832262144

194.247927513480.71062275543616859524288

204.321928094986.438561897740080001048576

f(n) = n

f(n) = log(n)

f(n) = n log(n)

f(n) = n^2

f(n) = n^3

f(n) = 2^n

Chart1

111002

248124

39271.58496250074.75488750228

416642816

5251252.321928094911.609640474432

6362162.584962500715.509775004364

7493432.807354922119.6514844544128

864512324256

9817293.169925001428.529325013512

1010010003.321928094933.21928094891024

1112113313.459431618638.0537478052048

1214417283.584962500743.01955000874096

1316921973.700439718148.10571633588192

1419627443.807354922153.302968908816384

1522533753.906890595658.603358934132768

16256409646465536

1728949134.087462841369.4868683013131072

1832458324.169925001475.058650026262144

1936168594.247927513480.7106227554524288

2040080004.321928094986.43856189771048576

Sheet1

1100112

2212484

331.58496250074.75488750229278

4428166416

552.321928094911.60964047442512532

662.584962500715.50977500433621664

772.807354922119.651484454449343128

8832464512256

993.169925001428.52932501381729512

10103.321928094933.219280948910010001024

11113.459431618638.05374780512113312048

12123.584962500743.019550008714417284096

13133.700439718148.105716335816921978192

14143.807354922153.3029689088196274416384

15153.906890595658.6033589341225337532768

1616464256409665536

17174.087462841369.48686830132894913131072

Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents
Documents