CS 2133: Data Structures Mathematics Review and Asymptotic Notation

  • View
    220

  • Download
    4

Embed Size (px)

Text of CS 2133: Data Structures Mathematics Review and Asymptotic Notation

  • CS 2133: Data StructuresMathematics Review andAsymptotic Notation

  • Arithmetic Series Review1 + 2 + 3 + . . . + n = ?Sn = a + a+d + a+2d + a+3d + . . . a+(n-1)dSn= 2a+(n-1)d + 2a+(n-2)d + 2a+(n-3)d + + a

    2Sn = 2a+(n-1)d + 2a+(n-1)d + 2a+(n-1)d . . . + 2a+(n-1)dSn = n/2[2a + (n-1)d]

    consequently 1+2+3++n = n(n+1)/2

  • ProblemsFind the sum of the following

    1+3+5+ . . . + 121 = ?

    The first 50 terms of -3 + 3 + 9 + 15 +

    1 + 3/2 + 2 + 5/2 + . . . 25=?

  • Geometric Series Review1 + 2 + 4 + 8 + . . . + 2n 1 + 1/2 + 1/4 + . . . + 2-nTheorem: Sn= a + ar + ar2 + . . . + arnrSn= ar + ar2 + . . . + arn + arn+1

    Sn-rSn = a - arn+1 What about the case where -1< r < 1 ?

  • Geometric ProblemsWhat is the sum of 3+9/4 + 27/16 + . . .

    1/2 - 1/4 + 1/8 - 1/16 + . . .

  • Harmonic SeriesThis is Eulers constants

  • Just an Interesting QuestionWhat is the optimal base to use in the representation of numbers n?

    Example: with base x we have _ _ _ _ _ _ _ _ _

    X valuesslotsWe minimize

  • Logarithm ReviewLn = loge is called the natural logarithmLg = log2 is called the binary logarithm

    How many bits are required to represent the number n in binary

  • Logarithm RulesThe logarithm to the base b of x denoted logbx is defined to that number y such that by = xlogbx > 0 if x > 1logbx = 0 if x = 1logbx < 0 if 0 < x < 1logb(x1*x2) = logb x1 + logb x2

    logb(x1/x2) = logb x1 - logb x2

    logb xc = c logbx

  • Additional RulesFor all real a>0, b>0 , c>0 and n

    logb a = logca / logc b logb (1/a) = - logb a

    logb a = 1/ logab a logb n = n logb a

  • Asymptotic PerformanceIn this course, we care most about the asymptotic performance of an algorithm.How does the algorithm behave as the problem size gets very large?Running timeMemory requirementsComing up: Asymptotic performance of two search algorithms,A formal introduction to asymptotic notation

  • Input SizeTime and space complexityThis is generally a function of the input sizeE.g., sorting, multiplicationHow we characterize input size depends:Sorting: number of input itemsMultiplication: total number of bitsGraph algorithms: number of nodes & edgesEtc

  • Running TimeNumber of primitive steps that are executedExcept for time of executing a function call most statements roughly require the same amount of timey = m * x + bc = 5 / 9 * (t - 32 )z = f(x) + g(y)We can be more exact if need be

  • AnalysisWorst caseProvides an upper bound on running timeAn absolute guaranteeAverage caseProvides the expected running timeVery useful, but treat with care: what is average?Random (equally likely) inputsReal-life inputs

  • An Example: Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}

  • Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}What is the preconditionfor this loop?

  • Insertion SortInsertionSort(A, n) { for i = 2 to n { key = A[i] j = i - 1; while (j > 0) and (A[j] > key) { A[j+1] = A[j] j = j - 1 } A[j+1] = key }}How many times will this loop execute?

  • Insertion SortStatement EffortInsertionSort(A, n) {for i = 2 to n { c1nkey = A[i]c2(n-1)j = i - 1;c3(n-1)while (j > 0) and (A[j] > key) {c4TA[j+1] = A[j]c5(T-(n-1))j = j - 1c6(T-(n-1))}0A[j+1] = keyc7(n-1)}0}T = t2 + t3 + + tn where ti is number of while expression evaluations for the ith for loop iteration

  • Analyzing Insertion SortT(n) = c1n + c2(n-1) + c3(n-1) + c4T + c5(T - (n-1)) + c6(T - (n-1)) + c7(n-1) = c8T + c9n + c10What can T be?Best case -- inner loop body never executedti = 1 T(n) is a linear functionWorst case -- inner loop body executed for all previous elementsti = i T(n) is a quadratic functionT=1+2+3+4+ . . . n-1 + n = n(n+1)/2

  • AnalysisSimplificationsIgnore actual and abstract statement costsOrder of growth is the interesting measure:Highest-order term is what countsRemember, we are doing asymptotic analysisAs the input size grows larger it is the high order term that dominates

  • Upper Bound NotationWe say InsertionSorts run time is O(n2)Properly we should say run time is in O(n2)Read O as Big-O (youll also hear it as order)In general a functionf(n) is O(g(n)) if there exist positive constants c and n0 such that f(n) c g(n) for all n n0FormallyO(g(n)) = { f(n): positive constants c and n0 such that f(n) c g(n) n n0

  • Big O exampleShow using the definition that 5n+4 O(n)Where g(n)=nFirst we must find a c and an n0We now need to show that f(n) c g(n) for every n n0

    clearly 5n + 5 6n whenever n 6

    Hence c=6 and n0=6 satisfy the requirements.

  • Insertion Sort Is O(n2)ProofSuppose runtime is an2 + bn + c If any of a, b, and c are less than 0 replace the constant with its absolute valuean2 + bn + c (a + b + c)n2 + (a + b + c)n + (a + b + c) 3(a + b + c)n2 for n 1Let c = 3(a + b + c) and let n0 = 1QuestionIs InsertionSort O(n3)?Is InsertionSort O(n)?

  • Big O FactA polynomial of degree k is O(nk)Proof:Suppose f(n) = bknk + bk-1nk-1 + + b1n + b0Let ai = | bi |f(n) aknk + ak-1nk-1 + + a1n + a0

  • Lower Bound NotationWe say InsertionSorts run time is (n)In general a functionf(n) is (g(n)) if positive constants c and n0 such that 0 cg(n) f(n) n n0Proof:Suppose run time is an + bAssume a and b are positive (what if b is negative?)an an + b

  • Asymptotic Tight BoundA function f(n) is (g(n)) if positive constants c1, c2, and n0 such that c1 g(n) f(n) c2 g(n) n n0

    Theoremf(n) is (g(n)) iff f(n) is both O(g(n)) and (g(n))Proof: someday

  • Notation(g) is the set of all functions f such that there exist positive constants c1, c2, and n0 such that

    0 c1g(n) f(n) c2 g(n) for every n > nc

    c1g(n)f(n)c2 g(n)

  • Growth Rate Theorems1. The power n is in O(n) iff (with ,>0) and n is in o(n) iff

    2. logbn o(n ) for any b and

    3. n o(cn) for any >0 and c>1

    4. logbn O(logbn) for any a and b

    5. cn O(dn) iff cd and cn o(dn) iff c

  • Big O Relationships1. o(f) O(f)

    2. If fo(g) then O(f) o(g)

    3. If f O(g) then o(f) o(g)

    4. If f O(g) then f(n) + g(n) O(g)

    5. If f O(f `) and g O(g`) then

    f(n)* g(n) O(f `(n) * g`(n))

  • Theorem: log(n!)(nlogn)Case 1 nlogn O(log(n!))log(n!) = log(n*(n-1)*(n-2) * * * 3*2*1)= log(n*(n-1)*(n-2)**n/2*(n/2-1)* * 2*1=> log(n/2*n/2* * * n/2*1 *1*1* * * 1)= log(n/2)n/2 = n/2 log n/2 O(nlogn)Case 2 log(n!) O(nlogn)log(n!) = logn + log(n-1) + log(n-2) + . . . Log(2) + log(1)< log n + log n + log n . . . + log n= nlogn

  • The Little o Theorem: If log(f)o(log(g)) andlim g(n) =inf as n goes to inf then f o(g)Note the above theorem does not apply to big O for log(n2) O(log n) but n2 O(n)Application: Show that 2n o(nn)Taking the log of functions we have log(2n)=nlog22and log( nn) = nlog2n.

    Hence

    Implies that 2n o(nn)

  • Theorem:

  • Practical Complexity

    Chart3

    100112

    212484

    31.58496250074.75488750229278

    428166416

    52.321928094911.60964047442512532

    62.584962500715.50977500433621664

    72.807354922119.651484454449343128

    832464512256

    93.169925001428.52932501381729512

    103.321928094933.219280948910010001024

    113.459431618638.05374780512113312048

    123.584962500743.019550008714417284096

    133.700439718148.105716335816921978192

    143.807354922153.3029689088196274416384

    153.906890595658.6033589341225337532768

    16464256409665536

    174.087462841369.48686830132894913131072

    184.169925001475.0586500263245832262144

    194.247927513480.71062275543616859524288

    204.321928094986.438561897740080001048576

    f(n) = n

    f(n) = log(n)

    f(n) = n log(n)

    f(n) = n^2

    f(n) = n^3

    f(n) = 2^n

    Chart1

    111002

    248124

    39271.58496250074.75488750228

    416642816

    5251252.321928094911.609640474432

    6362162.584962500715.509775004364

    7493432.807354922119.6514844544128

    864512324256

    9817293.169925001428.529325013512

    1010010003.321928094933.21928094891024

    1112113313.459431618638.0537478052048

    1214417283.584962500743.01955000874096

    1316921973.700439718148.10571633588192

    1419627443.807354922153.302968908816384

    1522533753.906890595658.603358934132768

    16256409646465536

    1728949134.087462841369.4868683013131072

    1832458324.169925001475.058650026262144

    1936168594.247927513480.7106227554524288

    2040080004.321928094986.43856189771048576

    Sheet1

    1100112

    2212484

    331.58496250074.75488750229278

    4428166416

    552.321928094911.60964047442512532

    662.584962500715.50977500433621664

    772.807354922119.651484454449343128

    8832464512256

    993.169925001428.52932501381729512

    10103.321928094933.219280948910010001024

    11113.459431618638.05374780512113312048

    12123.584962500743.019550008714417284096

    13133.700439718148.105716335816921978192

    14143.807354922153.3029689088196274416384

    15153.906890595658.6033589341225337532768

    1616464256409665536

    17174.087462841369.48686830132894913131072

Related documents