Algorithm Design Slides

  • View
    14

  • Download
    1

Embed Size (px)

DESCRIPTION

grrgrgrgr

Text of Algorithm Design Slides

  • Analysis of Algorithms

    AlgorithmInput Output

    An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.

    Analysis of Algorithms 2

    Running Time (1.1) w Most algorithms transform

    input objects into outputobjects.w The running time of an

    algorithm typically growswith the input size.w Average case time is often

    difficult to determine.w We focus on the worst case

    running time.n Easier to analyzen Crucial to applications such as

    games, finance and robotics

    0

    20

    40

    60

    80

    100

    120

    Ru

    nn

    ing

    Tim

    e

    1000 2000 3000 4000

    Input Size

    best case

    average caseworst case

    Analysis of Algorithms 3

    Experimental Studies ( 1.6)

    w Write a programimplementing thealgorithmw Run the program with

    inputs of varying size andcompositionw Use a method like

    System.currentTimeMillis() toget an accurate measureof the actual running timew Plot the results 0

    1000

    2000

    3000

    4000

    5000

    6000

    7000

    8000

    9000

    0 50 100

    Input Size

    Tim

    e (

    ms)

    Analysis of Algorithms 4

    Limitations of Experiments

    w It is necessary to implement thealgorithm, which may be difficultw Results may not be indicative of the

    running time on other inputs not includedin the experiment.w In order to compare two algorithms, the

    same hardware and softwareenvironments must be used

    Analysis of Algorithms 5

    Theoretical Analysis

    w Uses a high-level description of thealgorithm instead of an implementationw Characterizes running time as a

    function of the input size, n.w Takes into account all possible inputsw Allows us to evaluate the speed of an

    algorithm independent of thehardware/software environment

    Analysis of Algorithms 6

    Pseudocode (1.1)w High-level description

    of an algorithmw More structured than

    English prosew Less detailed than a

    programw Preferred notation for

    describing algorithmsw Hides program design

    issues

    Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of A

    currentMax A[0]for i 1 to n - 1 do

    if A[i] > currentMax thencurrentMax A[i]

    return currentMax

    Example: find maxelement of an array

  • Analysis of Algorithms 7

    Pseudocode Details

    w Control flown if then [else ]n while do n repeat until n for do n Indentation replaces braces

    w Method declarationAlgorithm method (arg [, arg])

    Input Output

    w Method callvar.method (arg [, arg])

    w Return valuereturn expression

    w ExpressionsAssignment

    (like = in Java)= Equality testing

    (like == in Java)n2 Superscripts and other

    mathematicalformatting allowed

    Analysis of Algorithms 8

    The Random Access Machine(RAM) Model

    w A CPU

    w An potentially unboundedbank of memory cells,each of which can hold anarbitrary number orcharacter

    012

    w Memory cells are numbered and accessingany cell in memory takes unit time.

    Analysis of Algorithms 9

    Primitive Operationsw Basic computations

    performed by an algorithmw Identifiable in pseudocodew Largely independent from the

    programming languagew Exact definition not important

    (we will see why later)w Assumed to take a constant

    amount of time in the RAMmodel

    w Examples:n Evaluating an

    expressionn Assigning a value

    to a variablen Indexing into an

    arrayn Calling a methodn Returning from a

    method

    Analysis of Algorithms 10

    Counting PrimitiveOperations (1.1)w By inspecting the pseudocode, we can determine the

    maximum number of primitive operations executed byan algorithm, as a function of the input size

    Algorithm arrayMax(A, n) # operationscurrentMax A[0] 2for i 1 to n - 1 do 2 + n

    if A[i] > currentMax then 2(n - 1)currentMax A[i] 2(n - 1)

    { increment counter i } 2(n - 1)return currentMax 1

    Total 7n - 1

    Analysis of Algorithms 11

    Estimating Running Timew Algorithm arrayMax executes 7n - 1 primitive

    operations in the worst case. Define:a = Time taken by the fastest primitive operationb = Time taken by the slowest primitive operation

    w Let T(n) be worst-case time of arrayMax. Thena (7n - 1) T(n) b(7n - 1)

    w Hence, the running time T(n) is bounded by twolinear functions

    Analysis of Algorithms 12

    Growth Rate of Running Time

    w Changing the hardware/ softwareenvironmentn Affects T(n) by a constant factor, butn Does not alter the growth rate of T(n)

    w The linear growth rate of the runningtime T(n) is an intrinsic property ofalgorithm arrayMax

  • Analysis of Algorithms 13

    Growth Rates

    w Growth rates offunctions:n Linear nn Quadratic n2

    n Cubic n3

    w In a log-log chart,the slope of the linecorresponds to thegrowth rate of thefunction

    1E+01E+21E+41E+61E+8

    1E+101E+121E+141E+161E+181E+201E+221E+241E+261E+281E+30

    1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n

    T(n

    )

    Cubic

    Quadratic

    Linear

    Analysis of Algorithms 14

    Constant Factors

    w The growth rate isnot affected byn constant factors orn lower-order terms

    w Examplesn 102n + 105 is a linear

    functionn 105n2 + 108n is a

    quadratic function1E+01E+21E+41E+61E+8

    1E+101E+121E+141E+161E+181E+201E+221E+241E+26

    1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n

    T(n

    )

    Quadratic

    Quadratic

    LinearLinear

    Analysis of Algorithms 15

    Big-Oh Notation (1.2)w Given functions f(n) and

    g(n), we say that f(n) isO(g(n)) if there arepositive constantsc and n0 such that

    f(n) cg(n) for n n0w Example: 2n + 10 is O(n)

    n 2n + 10 cnn (c - 2) n 10n n 10/(c - 2)n Pick c = 3 and n0 = 10

    1

    10

    100

    1,000

    10,000

    1 10 100 1,000n

    3n

    2n+10

    n

    Analysis of Algorithms 16

    Big-Oh Example

    w Example: the functionn2 is not O(n)n n2 cnn n cn The above inequality

    cannot be satisfiedsince c must be aconstant

    1

    10

    100

    1,000

    10,000

    100,000

    1,000,000

    1 10 100 1,000n

    n^2

    100n

    10n

    n

    Analysis of Algorithms 17

    More Big-Oh Examplesn 7n-2

    7n-2 is O(n)need c > 0 and n 0 1 such that 7n-2 cn for n n0this is true for c = 7 and n0 = 1

    n 3n3 + 20n2 + 53n3 + 20n2 + 5 is O(n3)need c > 0 and n 0 1 such that 3n3 + 20n 2 + 5 cn3 for n n0this is true for c = 4 and n0 = 21

    n 3 log n + log log n3 log n + log log n is O(log n)need c > 0 and n 0 1 such that 3 log n + log log n clog n for n n0this is true for c = 4 and n0 = 2

    Analysis of Algorithms 18

    Big-Oh and Growth Ratew The big-Oh notation gives an upper bound on the

    growth rate of a functionw The statement f(n) is O(g(n)) means that the growth

    rate of f(n) is no more than the growth rate of g(n)w We can use the big-Oh notation to rank functions

    according to their growth rate

    YesYesSame growthYesNof(n) grows moreNoYesg(n) grows more

    g(n) is O(f(n))f(n) is O(g(n))

  • Analysis of Algorithms 19

    Big-Oh Rules

    w If is f(n) a polynomial of degree d, then f(n) isO(nd), i.e.,

    n Drop lower-order termsn Drop constant factors

    w Use the smallest possible class of functionsn Say 2n is O(n) instead of 2n is O(n2)

    w Use the simplest expression of the classn Say 3n + 5 is O(n) instead of 3n + 5 is O(3n)

    Analysis of Algorithms 20

    Asymptotic Algorithm Analysisw The asymptotic analysis of an algorithm determines

    the running time in big-Oh notationw To perform the asymptotic analysis

    n We find the worst-case number of primitive operationsexecuted as a function of the input size

    n We express this function with big-Oh notation

    w Example:n We determine that algorithm arrayMax executes at most

    7n - 1 primitive operationsn We say that algorithm arrayMax runs in O(n) time

    w Since constant factors and lower-order terms areeventually dropped anyhow, we can disregard themwhen counting primitive operations

    Analysis of Algorithms 21

    Computing Prefix Averagesw We further illustrate

    asymptotic analysis withtwo algorithms for prefixaveragesw The i-th prefix average of

    an array X is average of thefirst (i + 1) elements of X:

    A[i] = (X[0] + X[1] + + X[i])/( i+1)

    w Computing the array A ofprefix averages of anotherarray X has applications tofinancial analysis

    0

    5

    10

    15

    20

    25

    30

    35

    1 2 3 4 5 6 7

    XA

    Analysis of Algorithms 22

    Prefix Averages (Quadratic)w The following algorithm computes prefix averages in

    quadratic time by applying the definition

    Algorithm prefixAverages1(X, n)Input array X of n integersOutput array A of prefix averages of X #operations A new array of n integers nfor i 0 to n - 1 do n

    s X[0] nfor j 1 to i do 1 + 2 + + (n - 1)

    s s + X[j] 1 + 2 + + (n - 1)A[i] s / (i + 1) n

    return A 1

    Analysis of Algorithms 23

    Arithmetic Progression

    w The running time ofprefixAverages1 isO(1 + 2 + + n)w The sum of the first n

    integers is n(n + 1) / 2n There is a simple visual

    proof of this fact

    w Thus, algorithmprefixAverages1 runs inO(n2) time 0

    1

    2

    3

    4

    5

    6

    7

    1 2 3 4 5 6

    Analysis of Algorithms 24

    Prefix Averages (Linear)w The following algorithm computes prefix averages in

    linear time by keeping a running sum

    Algorithm prefixAverages2(X, n)Input array X of n integersOutput array A of prefix averages of X