Upload
others
View
2
Download
0
Embed Size (px)
Citation preview
1
CS103Unit8bSlides
Algorithms
MarkRedekopp
2
ALGORITHMS
3
HowDoYouFindaWordinaDictionary
• Describean“efficient”method• Assumptions/Guidelines
– Lettarget_word =wordtolookup– Npagesinthedictionary– Eachpagehasthestart andlast wordonthatpagelistedatthetopofthepage
– Assumetheuserunderstandshowtoperformalphabetical(“lexicographic”)comparison(e.g.“abc”issmallerthan“acb”or“abcd”)
apple arc
45
4
Algorithms• Algorithmsareattheheartofcomputer
systems,bothinHWandSW– TheyarefundamentaltoComputerScience
andComputerEngineering• Informaldefinition
– Analgorithmisaprecisewaytoaccomplishataskorsolveaproblem
• Softwareprogramsarecollectionsofalgorithmstoperformdesiredtasks
• Hardwarecomponentsalsoimplementalgorithmsfromsimpletocomplex Hardware Software
5
HumansandComputers
• Humansunderstandalgorithmsdifferentlythancomputers
• Humanseasilytolerateambiguityandabstractconceptsusingcontexttohelp.– “Addapinchofsalt.”Howmuchisapinch?– “MichaelJordancouldsoarlikeaneagle.”– “It’sabearmarket”
• Computersonlyexecutewell-definedinstructions(noambiguity)andoperateondigitalinformationwhichisdefiniteanddiscrete(everythingisexactandnot“closeto”)
6
FormalDefinition
• Foracomputer,“algorithm”isdefinedas...– …anorderedsetofunambiguous,executablestepsthatdefinesaterminatingprocess
• Explanation:– OrderedSteps:thestepsofanalgorithmhaveaparticularorder,notjustanyorder
– Unambiguous:eachstepiscompletelyclearastowhatistobedone
– Executable:Eachstepcanactuallybeperformed– TerminatingProcess:Algorithmwillstop,eventually.(sometimesthisrequirementisrelaxed)
7
AlgorithmRepresentation
• Analgorithmisnotaprogramorprogramminglanguage
• Justasastorymayberepresentedasabook,movie,orspokenbyastory-teller,analgorithmmayberepresentedinmanyways– Flowchart– Pseudocode (English-likesyntaxusingprimitivesthatmostprogramminglanguageswouldhave)
– Aspecificprogramimplementationinagivenprogramminglanguage
8
AlgorithmExample1• List/printallfactorsofanaturalnumber,n
– Howwouldyoucheckifanumberisafactorofn?– Whatistherangeofpossiblefactors?i←1while(i<=n)doif (remainderofn/iiszero)thenListiasafactorofn
i←i+1
• Animprovementi←1while(i<=sqrt(n))doif (remainderofn/iiszero)thenListiandn/iasafactorofn
i←i+1
TF
9
AlgorithmTimeComplexity
• Weoftenjudgealgorithmsbyhowlongtheytaketorunforagiveninputsize
• Algorithmsoftenhavedifferentrun-timesbasedontheinputsize[e.g.#ofelementsinalisttosearchorsort]– Differentinputpatternscanleadtobestandworstcasetimes
– Average-casetimescanbehelpful,butweusuallyuseworstcasetimesforcomparisonpurposes
10
Big-ONotation• Givenaninputtoanalgorithmofsizen,wecanderiveanexpressionin
termsofnforitsworstcaseruntime(i.e.thenumberofstepsitmustperformtocomplete)
• Fromtheexpressionwelookforthedominanttermandsaythatisthebig-O(worst-caseorupper-bound)run-time– Ifanalgorithmwithinputsizeofnrunsinn2 +10n+1000steps,wesaythatit
runsinO(n2)becauseifnislargen2 willdominatetheotherterms
i ← 1while(i <= n) do
if (remainder of n/i is zero) thenList i as a factor of n
i ← i+1
11*n2*n
5n+1= O(n)
1*n1*n
11
Big-ONotation• Givenaninputtoanalgorithmofsizen,wecanderiveanexpressionin
termsofnforitsworstcaseruntime(i.e.thenumberofstepsitmustperformtocomplete)
• Fromtheexpressionwelookforthedominanttermandsaythatisthebig-O(worst-caseorupper-bound)run-time– Ifanalgorithmwithinputsizeofnrunsinn2 +10n+1000steps,wesaythatit
runsinO(n2)becauseifnislargen2 willdominatetheotherterms• Mainsourcesofrun-time:Loops
– Evenworse:Loopswithinloops(i.e.executeallofloop2w/inasingleiterationofloop1,andrepeatforalliterationsofloop1,etc.)
i ← 1while(i <= n) do
if (remainder of n/i is zero) thenList i as a factor of n
i ← i+1
11*n2*n
5n+1= O(n)
1*n1*n
12
AlgorithmExample1• List/printallfactorsofanaturalnumber,n
– Whatisafactor?– Whatistherangeofpossiblefactors?i←1while(i<=n)doif (remainderofn/iiszero)thenListiasafactorofni←i+1
• Animprovementi←1while(i<=sqrt(n))doif (remainderofn/iiszero)thenListiandn/iasafactorofni←i+1
O(n)
13
AlgorithmExample2a• Searchinganorderedlist(array)fora
specificvalue,k,andreturnitsindexor-1ifitisnotinthelist
• SequentialSearch– Startatfirstitem,checkifitisequaltok,repeat
forsecond,third,fourthitem,etc.
i ←0while (i <length(myList))doif (myList[i]equaltok)thenreturn ielsei ←i+1
return -1
2 3 4 6 9 10 13 15 19myListindex 1 2 3 4 5 6 7 80
14
AlgorithmExample2b
• Sequentialsearchdoesnottakeadvantageoftheorderednatureofthelist– Wouldworkthesame(equallywell)onan
orderedorunorderedlist
• BinarySearch– Takeadvantageoforderedlistbycomparingk
withmiddleelementandbasedontheresult,ruleoutallnumbersgreaterorsmaller,repeatwithmiddleelementofremaininglist,etc.
2 3 4 6 9 10 13 15 19Listindex
6 < 9
k = 6
Start in middle
2 3 4 6 9 10 13 15 19Listindex
6 > 4
6 9 10 13 15 19Listindex
6 = 6
2 3 4
1 2 3 4 5 6 7 80
1 2 3 4 5 6 7 80
1 2 3 4 5 6 7 80
15
AlgorithmExample2b• BinarySearch
– Comparekwithmiddleelementoflistandifnotequal,ruleout½ofthelistandrepeatontheotherhalf
– Implementation:• Definerangeofsearchableelements=[start,end)• (i.e.startisinclusive,endisexclusive)
start←0;end←length(List);while (startindexnotequaltoendindex)doi ←(start+end)/2;if (k==List[i])then returni;elseif (k>List[i])then start←i+1;else end←i;return-1;
2 3 4 6 9 11 13 15 19Listindex
2 3 4 6 9 11 13 15 19Listindex
i
k = 11
endstart
i endstart
2 3 4 6 9 11 13 15 19Listindex
endstart i
1 2 3 4 5 6 7 80
1 2 3 4 5 6 7 80
1 2 3 4 5 6 7 80
2 3 4 6 9 11 15 19Listindex
endstarti
1 2 3 4 5 6 7 8013
16
Sorting• Ifwehaveanunorderedlist,sequential
searchbecomesouronlychoice• Ifwewillperformalotofsearchesitmay
bebeneficialtosortthelist,thenusebinarysearch
• Manysortingalgorithmsofdifferingcomplexity(i.e.fasterorslower)
• BubbleSort(simplethoughnotterriblyefficient)– Oneachpassthroughthruthelist,pickupthe
maximumelementandplaceitattheendofthelist.Thenrepeatusingalistofsizen-1(i.e.w/othenewlyplacedmaximumvalue)
7 3 8 6 5 1Listindex
Original1 2 3 4 50
3 7 6 5 1 8Listindex
After Pass 11 2 3 4 50
3 6 5 1 7 8Listindex
After Pass 21 2 3 4 50
3 5 1 6 7 8Listindex
After Pass 31 2 3 4 50
3 1 5 6 7 8Listindex
After Pass 41 2 3 4 50
1 3 5 6 7 8Listindex
After Pass 51 2 3 4 50
17
BubbleSortAlgorithm
7 3 8 6 5 1j i
Pass 1
3 7 8 6 5 1j i
3 7 8 6 5 1j i
3 7 6 8 5 1j i
3 7 6 5 8 1j
3 7 6 5 1 8
swap
no swap
swap
swap
swap
j i
Pass 2
3 7 6 5 1 8j i
3 6 7 5 1 8j i
3 6 5 7 1 8
3 6 5 1 7 8
j
no swap
swap
swap
swap
3 7 6 5 1 8i
Pass n-23 1 5 6 7 8
1 3 5 6 7 8 swap
…
void bsort(int* mylist, int n){int i ;for(i=n-1; i > 0; i--){
for(j=0; j < i; j++){if(mylist[j] > mylist[j+1]) {
swap(j, j+1)} } }
}
i
i
j
18
ComplexityofSearchAlgorithms
• SequentialSearch:Listoflengthn– Worstcase:Searchthroughentirelist– Timecomplexity=an+k
• aissomeconstantfornumberofoperationsweperformintheloopasweiterate
• kissomeconstantrepresentingstartup/finishwork(outsidetheloop)
– SequentialSearch=O(n)
• BinarySearch:Listoflengthn– Worstcase:Continuallydividelistintwo
untilwereachsublistofsize1– Time=a*log2n+k=O(log2n)
• Asngetslarge,binarysearchisfarmoreefficientthansequentialsearch
Dividing by 2 k-times yields:
n / 2k = 1k = log2n
Multiplying by 2 k-times yields: 2*2*2…*2 = 2k
0 5 10 15 20 25 30 35 40 45 500
5
10
15
20
25
30
35
40
45
50
N
Run-tim
e
19
ComplexityofSortAlgorithms
• BubbleSort– 2NestedLoops– Executeouterloopn-1times– Foreachouterloopiteration,
innerlooprunsitimes.– Timecomplexityisproportional
to:n-1+n-2+n-3+…+1=(n2 +n)/2=O(n2)
• OthersortalgorithmscanruninO(n*log2n)
0 2 4 6 8 10 12 14 16 18 200
50
100
150
200
250
300
350
400
N
Run-
time
NN2
N*log2(N)
20
ImportanceofTimeComplexity
N O(1) O(log2n) O(n) O(n*log2n) O(n2) O(2n)
2 1 1 2 2 4 4
20 1 4.3 20 86.4 400 1,048,576
200 1 7.6 200 1,528.8 40,000 1.60694E+60
2000 1 11.0 2000 21,931.6 4,000,000 #NUM!
• Itmakesthedifferencebetweeneffectiveandimpossible• Manyimportantproblemscurrentlycanonlybesolvedwithexponentialrun-time
algorithms(e.g.O(2n)time)…wecalltheseNP=Non-deterministicpolynomialtimealgorithms)[Noknownpolynomial-timealgorithmexists]
• UsuallyalgorithmsareonlypracticaliftheyruninP=polynomialtime(e.g.O(n)orO(n2)etc.)
• OneofthemostpressingopenproblemsinCS:“IsNP=P?”– DoPalgorithmsexistfortheproblemsthatwecurrentlyonlyhaveanNPsolutionfor?