# Algorithm Design Slides

• View
14

1

Embed Size (px)

DESCRIPTION

grrgrgrgr

### Text of Algorithm Design Slides

• Analysis of Algorithms

AlgorithmInput Output

An algorithm is a step-by-step procedure forsolving a problem in a finite amount of time.

Analysis of Algorithms 2

Running Time (1.1) w Most algorithms transform

input objects into outputobjects.w The running time of an

algorithm typically growswith the input size.w Average case time is often

difficult to determine.w We focus on the worst case

running time.n Easier to analyzen Crucial to applications such as

games, finance and robotics

0

20

40

60

80

100

120

Ru

nn

ing

Tim

e

1000 2000 3000 4000

Input Size

best case

average caseworst case

Analysis of Algorithms 3

Experimental Studies ( 1.6)

w Write a programimplementing thealgorithmw Run the program with

inputs of varying size andcompositionw Use a method like

System.currentTimeMillis() toget an accurate measureof the actual running timew Plot the results 0

1000

2000

3000

4000

5000

6000

7000

8000

9000

0 50 100

Input Size

Tim

e (

ms)

Analysis of Algorithms 4

Limitations of Experiments

w It is necessary to implement thealgorithm, which may be difficultw Results may not be indicative of the

running time on other inputs not includedin the experiment.w In order to compare two algorithms, the

same hardware and softwareenvironments must be used

Analysis of Algorithms 5

Theoretical Analysis

w Uses a high-level description of thealgorithm instead of an implementationw Characterizes running time as a

function of the input size, n.w Takes into account all possible inputsw Allows us to evaluate the speed of an

algorithm independent of thehardware/software environment

Analysis of Algorithms 6

Pseudocode (1.1)w High-level description

of an algorithmw More structured than

English prosew Less detailed than a

programw Preferred notation for

describing algorithmsw Hides program design

issues

Algorithm arrayMax(A, n)Input array A of n integersOutput maximum element of A

currentMax Afor i 1 to n - 1 do

if A[i] > currentMax thencurrentMax A[i]

return currentMax

Example: find maxelement of an array

• Analysis of Algorithms 7

Pseudocode Details

w Control flown if then [else ]n while do n repeat until n for do n Indentation replaces braces

w Method declarationAlgorithm method (arg [, arg])

Input Output

w Method callvar.method (arg [, arg])

w Return valuereturn expression

w ExpressionsAssignment

(like = in Java)= Equality testing

(like == in Java)n2 Superscripts and other

mathematicalformatting allowed

Analysis of Algorithms 8

The Random Access Machine(RAM) Model

w A CPU

w An potentially unboundedbank of memory cells,each of which can hold anarbitrary number orcharacter

012

w Memory cells are numbered and accessingany cell in memory takes unit time.

Analysis of Algorithms 9

Primitive Operationsw Basic computations

performed by an algorithmw Identifiable in pseudocodew Largely independent from the

programming languagew Exact definition not important

(we will see why later)w Assumed to take a constant

amount of time in the RAMmodel

w Examples:n Evaluating an

expressionn Assigning a value

to a variablen Indexing into an

arrayn Calling a methodn Returning from a

method

Analysis of Algorithms 10

Counting PrimitiveOperations (1.1)w By inspecting the pseudocode, we can determine the

maximum number of primitive operations executed byan algorithm, as a function of the input size

Algorithm arrayMax(A, n) # operationscurrentMax A 2for i 1 to n - 1 do 2 + n

if A[i] > currentMax then 2(n - 1)currentMax A[i] 2(n - 1)

{ increment counter i } 2(n - 1)return currentMax 1

Total 7n - 1

Analysis of Algorithms 11

Estimating Running Timew Algorithm arrayMax executes 7n - 1 primitive

operations in the worst case. Define:a = Time taken by the fastest primitive operationb = Time taken by the slowest primitive operation

w Let T(n) be worst-case time of arrayMax. Thena (7n - 1) T(n) b(7n - 1)

w Hence, the running time T(n) is bounded by twolinear functions

Analysis of Algorithms 12

Growth Rate of Running Time

w Changing the hardware/ softwareenvironmentn Affects T(n) by a constant factor, butn Does not alter the growth rate of T(n)

w The linear growth rate of the runningtime T(n) is an intrinsic property ofalgorithm arrayMax

• Analysis of Algorithms 13

Growth Rates

w Growth rates offunctions:n Linear nn Quadratic n2

n Cubic n3

w In a log-log chart,the slope of the linecorresponds to thegrowth rate of thefunction

1E+01E+21E+41E+61E+8

1E+101E+121E+141E+161E+181E+201E+221E+241E+261E+281E+30

1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n

T(n

)

Cubic

Linear

Analysis of Algorithms 14

Constant Factors

w The growth rate isnot affected byn constant factors orn lower-order terms

w Examplesn 102n + 105 is a linear

functionn 105n2 + 108n is a

1E+101E+121E+141E+161E+181E+201E+221E+241E+26

1E+0 1E+2 1E+4 1E+6 1E+8 1E+10n

T(n

)

LinearLinear

Analysis of Algorithms 15

Big-Oh Notation (1.2)w Given functions f(n) and

g(n), we say that f(n) isO(g(n)) if there arepositive constantsc and n0 such that

f(n) cg(n) for n n0w Example: 2n + 10 is O(n)

n 2n + 10 cnn (c - 2) n 10n n 10/(c - 2)n Pick c = 3 and n0 = 10

1

10

100

1,000

10,000

1 10 100 1,000n

3n

2n+10

n

Analysis of Algorithms 16

Big-Oh Example

w Example: the functionn2 is not O(n)n n2 cnn n cn The above inequality

cannot be satisfiedsince c must be aconstant

1

10

100

1,000

10,000

100,000

1,000,000

1 10 100 1,000n

n^2

100n

10n

n

Analysis of Algorithms 17

More Big-Oh Examplesn 7n-2

7n-2 is O(n)need c > 0 and n 0 1 such that 7n-2 cn for n n0this is true for c = 7 and n0 = 1

n 3n3 + 20n2 + 53n3 + 20n2 + 5 is O(n3)need c > 0 and n 0 1 such that 3n3 + 20n 2 + 5 cn3 for n n0this is true for c = 4 and n0 = 21

n 3 log n + log log n3 log n + log log n is O(log n)need c > 0 and n 0 1 such that 3 log n + log log n clog n for n n0this is true for c = 4 and n0 = 2

Analysis of Algorithms 18

Big-Oh and Growth Ratew The big-Oh notation gives an upper bound on the

growth rate of a functionw The statement f(n) is O(g(n)) means that the growth

rate of f(n) is no more than the growth rate of g(n)w We can use the big-Oh notation to rank functions

according to their growth rate

YesYesSame growthYesNof(n) grows moreNoYesg(n) grows more

g(n) is O(f(n))f(n) is O(g(n))

• Analysis of Algorithms 19

Big-Oh Rules

w If is f(n) a polynomial of degree d, then f(n) isO(nd), i.e.,

n Drop lower-order termsn Drop constant factors

w Use the smallest possible class of functionsn Say 2n is O(n) instead of 2n is O(n2)

w Use the simplest expression of the classn Say 3n + 5 is O(n) instead of 3n + 5 is O(3n)

Analysis of Algorithms 20

Asymptotic Algorithm Analysisw The asymptotic analysis of an algorithm determines

the running time in big-Oh notationw To perform the asymptotic analysis

n We find the worst-case number of primitive operationsexecuted as a function of the input size

n We express this function with big-Oh notation

w Example:n We determine that algorithm arrayMax executes at most

7n - 1 primitive operationsn We say that algorithm arrayMax runs in O(n) time

w Since constant factors and lower-order terms areeventually dropped anyhow, we can disregard themwhen counting primitive operations

Analysis of Algorithms 21

Computing Prefix Averagesw We further illustrate

asymptotic analysis withtwo algorithms for prefixaveragesw The i-th prefix average of

an array X is average of thefirst (i + 1) elements of X:

A[i] = (X + X + + X[i])/( i+1)

w Computing the array A ofprefix averages of anotherarray X has applications tofinancial analysis

0

5

10

15

20

25

30

35

1 2 3 4 5 6 7

XA

Analysis of Algorithms 22

Prefix Averages (Quadratic)w The following algorithm computes prefix averages in

quadratic time by applying the definition

Algorithm prefixAverages1(X, n)Input array X of n integersOutput array A of prefix averages of X #operations A new array of n integers nfor i 0 to n - 1 do n

s X nfor j 1 to i do 1 + 2 + + (n - 1)

s s + X[j] 1 + 2 + + (n - 1)A[i] s / (i + 1) n

return A 1

Analysis of Algorithms 23

Arithmetic Progression

w The running time ofprefixAverages1 isO(1 + 2 + + n)w The sum of the first n

integers is n(n + 1) / 2n There is a simple visual

proof of this fact

w Thus, algorithmprefixAverages1 runs inO(n2) time 0

1

2

3

4

5

6

7

1 2 3 4 5 6

Analysis of Algorithms 24

Prefix Averages (Linear)w The following algorithm computes prefix averages in

linear time by keeping a running sum

Algorithm prefixAverages2(X, n)Input array X of n integersOutput array A of prefix averages of X

Recommended ##### Sample Complexity for Data Driven Algorithm Design ckingsf/AutoAlg2019/slides... Sample Complexity of
Documents ##### Introduction Design and Analysis of sscott/teach/Classes/cse423S12/slides/...آ  Edmonds-Karp Algorithm
Documents ##### 12 A: Algorithm Design Techniques II - NUS Computing cs1102s/slides/slides_12_A... Optimal Binary Search
Documents ##### Algorithm Design and amotong/teaching/algorithm design...آ  (Design) Algorithm design Implementation
Documents ##### Quantum algorithm for P3: polar decomposition, Petz ... yquek/Slides/Petz_slides.pdfآ  Quantum algorithm
Documents     