AdvancedAlgorithmics(6EAP)courses.cs.ut.eeMTAT.03.238Introduction
JaakVilo2016Fall
1JaakVilo
Programmingononeslide• TRUE/FALSE,boolean operations• Data:3,7,2.47,.333e-02,‘a’,“textdata”, address…• Operations:+- */><=…• Variables:i,j,sum,A[5],B[7,8],a[i].surname,…
• if-then-else,for/while,…• function()
• Input/output• Standardlibraries:java.lang.Math.sqrt()
Goals
• Tolearnthemainconceptsandtechniquesofthealgorithmanddatastructuredesignandanalysis– thepracticalskillsandbasictheoreticalbasis
• Tobeabletochoose,(design,)analyzeandcomparealgorithmsanddatastructures
• Tolearntolearn,usewisely,solve,read,write,andpresent
Algorithms=basicsofCSeducation
• InyourfutureprofessionallifepeopleEXPECTyoutoknowelementarydatastructures,algorithms,andbeabletothinkinhighercategories.
• Weneedtodevelopthelanguagetotalkabouthigh-levelgoals,andyouneedtobeabletoimplementthem– ifsoneeded
• Atworkno-oneisgoingtoteachyoumathematicsorfundamentalsanymore
Algorithms
• Al-Khwārizmī
• AbūʿAbdallāhMuḥammadibnMūsāal-Khwārizmī[1](Arabic:) (أبو عبد الله محمد بن موسى الخوارزمي c.780,Khwārizm[2][3][4] – c.850)wasaPersian[5][2][6]mathematician,astronomer andgeographer,ascholarintheHouseofWisdom inBaghdad.
Example
• Windhasblownawaythe+,*,(,)signs• What’sthemaximalpossiblevalue?• Minimal?
2 1 7 1 4 3
• 217143
• (2+1)*7*(1+4)*3=21*15=315• 2*1+7+1*4+3=16
• Q:Howtomaximizethevalueofanyexpression?
2451981219872441123=?
Schoolmathematics:11*13
13x11
O( n2 )
is the same as:
Analysisofalgorithms
• Theoreticalstudyofcomputer-programperformanceandresourceusage.
• What elseis importantthanperformance?•Modularity•Correctness•Maintainability•Functionality•Robustness
•User-friendliness•Programmer time•Simplicity•Extensibility•Reliability
Whystudyalgorithmsandperformance?• Algorithmshelpustounderstandscalability.• Performanceoftendrawsthelinebetweenwhatisfeasibleandwhatisimpossible.
• Algorithmicmathematicsprovidesalanguagefortalkingaboutprogrambehavior.
• Performanceisthecurrency ofcomputing.• Thelessonsofprogramperformancegeneralize toothercomputingresources.
• Speedisfun!
SteveJürvetson onSkype:
• "Irememberwondering:howcantheybesogood?"hetoldme,speakingabouttheEstoniancoreofSkype."Howcansuchasmallgroupcandosomuchsoquickly,comparedtotypicaldevelopmenteffortsin,forexample,Microsoft?IhadtheimpressionthatmaybecomingoutofatimeofSovietoccupation,whencomputerswereunderpowered,youhadtoknowhowtoreallyprogram,effectively,parsimoniously,beingveryelegantinsculptingtheprogrammingcodetobetight,effective,andfast.[That's]notlikeinMicrosoft,whichhasaverylazyprogrammingenvironment,whereprogramsarecreatedthathavememoryleaksandallsortsofproblems,thatcrashallthetimeandnoonereallycares—becauseit'sMicrosoft!"
• http://arstechnica.com/business/2013/09/skypes-secrets/
Contents• AlgorithmsandDatastructures– basics• Abstractdatatypes (+STL,LEDA,…)• Lists,trees,heaps,graphs(TSP;shortestpaths,diameter, …)
• Analysisofalgorithmcomplexity– O( nlogn),o(nlogn),Θ(nlogn),Ω(nlogn)– recurrences T(n)=2T(n/2)+n– Turingmachines,NP,NP-complete,P-SPACE,…– Verytough (NP)problems (“Monkeyproblem”;unsolvableproblems?)
Contents(2)
• Basicdesigns: Divideandconquer,dynamicprogramming,randomised algorithms,…
• Deterministicornon-deterministic• Exactorapproximate• Heuristics(approximations):
– GreedyAlgorithms(packing,setcover,weightedsetcover,…)localsearch,tabusearch
– Simulated annealing(e.g.TSP),– Genetic algorithms(?),…– MonteCarlo,LasVegas
Contents(3)
• Sorting,search,dictionary,…• Textalgorithms• Numericalalgorithms• Computationalgeometry
• Onlinealgorithmsvsoffline• Parallelalgorithms
Skills/goals
• Bit-levelunderstandingoflow-levelimplementation
• Creativeuseofexistingimplementations• Formalanalysis,complexitytheory• Typicalheuristicsandalgorithmdesignelements
vs
• 67+56=?
SUM(1079882846662946257362655369837393771079882846662946257362655369837393111771079882846662946257362655361983111173937710798828466629462573626553698373937710798182666211946257362655369837319377107988284666294625736265536173937710798828466629462573165536983739377)=?
Answer
• 13,000
• Ischecking fasterthanfinding thesolution?
SATisfiability
• Canaboolean formulawithvariablesinstantiatedtobecome“TRUE”– (AorB)and(notAorB)
• Checkif(some)answersatisfiestheformula• veryquickascomparedtofindingananswer
Subsetsum
• Doessome subsetofasetsumupto0?– Verifier/witness/certificate– Easytocheckifsomeonepointsout…– 2,-4,3,8,-1,9,-6
• IfverificationinPtime,thenprobleminNP
• Complementary:provethatnoneexists!– co-NP– Everysubsetisnon-0
Maindatamodels
…Linear
Tree
Graph
Relational data model
Allfittedonlinear RAM,disk,tape
…Linear
Tree
Graph
Relational
Linear storage: address of a “slot”
Programminglanguages:
• Pseudocode– primarilythiscoursewillusepseudocode
• Pointers,arrays,memoryhandling– Low- andhighlevelabstractions
• C/C++• Java,Python,perl,go,…
– Explicitdatastructuresandalgorithms
• Whicheverchoice:youmustbeabletoexplainit• Clarityandsimplicityisakey
“TheTextbook”• IntroductiontoAlgorithms,SecondEditionThomasH.Cormen,CharlesE.Leiserson,RonaldL.Rivest andCliffordStein
• http://mitpress.mit.edu/algorithms/• http://projects.csail.mit.edu/clrs/• http://books.google.com/books?id=NLngYyWFl_YC
• FoundationsofComputerScience:CEdition
• AlfredV.Aho,JeffreyD.Ullman
• W.H.Freeman(October15,1994)
• Algorithmics:TheSpiritofComputing(3rdEdition)
• DavidHarel,YishaiFeldman• AddisonWesley;3edition(June11,2004)
• http://books.google.com/books?id=txxLovFWkCUC• http://www.wisdom.weizmann.ac.il/~dharel/algorithmic
s.html
• TheArtofComputerProgramming(TAOCP)• DonaldE.Knuth.• http://www-cs-faculty.stanford.edu/~knuth/taocp.html
• TheAlgorithmDesignManual– StevenS.Skiena
• Algorithms[ILLUSTRATED]– SanjoyDasgupta,ChristosPapadimitriou,UmeshVazirani
Mathematics
• ConcreteMathematics:AFoundationforComputerScience(2ndEdition)(Hardcover)
• RonaldL.Graham,DonaldE.Knuth, OrenPatashnik
• Hardcover: 672pages• Addison-WesleyProfessional;• 2edition(March10,1994)• ISBN-10: 0201558025ISBN-13: 978-0201558029
• BundleofAlgorithmsinJava,ThirdEdition,Parts1-5:Fundamentals,DataStructures,Sorting,Searching,andGraphAlgorithms(3rdEdition)(Pts.1-5)
• RobertSedgewick
• SafariBookshelf!(UT)
Morebooks
• Parallelalgorithms– AnIntroductiontoParallelAlgorithms JosephJaja
• Heuristics– HowtoSolveit:ModernHeuristics. ByZbigniewMichalewicz,DavidB.Fogel– Edition:2,illustrated PublishedbySpringer,2004– http://www.springer.com/computer/foundations/book/978-3-540-22494-5
• Randomizedalgorithms• Complexitytheory
– Garey,M.R.;Johnson,D.S. (1979).ComputersandIntractability:AGuidetotheTheoryofNP-Completeness.NewYork:W.H.Freeman.ISBN0-7167-1045-5.Thisbookisaclassic,developingthetheory,thencataloguingmanyNP-Completeproblems.
Wikipedia:
• http://en.wikipedia.org/wiki/Algorithm• http://en.wikipedia.org/wiki/List_of_algorithm_general_topics• http://en.wikipedia.org/wiki/List_of_algorithms• http://en.wikipedia.org/wiki/List_of_data_structures• http://en.wikipedia.org/wiki/Computational_complexity_theory• http://en.wikipedia.org/wiki/List_of_NP-complete_problems• http://en.wikipedia.org/wiki/Randomized_algorithm• http://en.wikipedia.org/wiki/Parallel_algorithm• …