Chapter 1Chapter 1

Sections 1.1 – 1.4Sections 1.1 – 1.4pages 1-40pages 1-40

HomeworkHomework

Read Section 1.4 (recap of data Read Section 1.4 (recap of data structures)structures)

pages 26-37pages 26-37 Answer the following questions:Answer the following questions:

• page 38, questions 1a, 1b, 2a, 2b, & 10page 38, questions 1a, 1b, 2a, 2b, & 10• Also, what exactly is a dictionary (not Also, what exactly is a dictionary (not

Webster's)Webster's)• Due Wed 1/28 (in class)Due Wed 1/28 (in class)

AgendaAgenda

WTF?WTF? Euclid’s algorithmEuclid’s algorithm What is an algorithmWhat is an algorithm What is algorithm analysis all aboutWhat is algorithm analysis all about Problem typesProblem types

Algorithm design & analysis processAlgorithm design & analysis processUnderstand the Problem

Pick an algorithm design technique

Pick data structures, etc.

Design an algorithm

Analyze the algorithm

Prove correctness

Code the algorithmWTF?

Euclid’s AlgorithmEuclid’s Algorithm

while n != 0 dowhile n != 0 dor r m mod n m mod nm m n nn n r r

return mreturn m Isn’t it a cool algorithm?Isn’t it a cool algorithm? Its very efficient, no?Its very efficient, no? How many divisions must it do, as a How many divisions must it do, as a

function of m or n?function of m or n?

Euclid’s AlgorithmEuclid’s Algorithm

Best case: m mod n equal zeroBest case: m mod n equal zero Worst case?Worst case?

• Prime numbers maybe?Prime numbers maybe?• How do you get it to do a lot of How do you get it to do a lot of

divisions?divisions?

Try ThisTry This

Two prime numbers 29 and 7Two prime numbers 29 and 7 Then try 13 and 8.Then try 13 and 8. How can you cause the worst case?How can you cause the worst case?

Euclid’s AlgorithmEuclid’s Algorithm

Answer: Two consecutive Fibonacci Answer: Two consecutive Fibonacci numbers.numbers.

In the worst case it will require about In the worst case it will require about L divisions L divisions • where L is the total number of digits in where L is the total number of digits in

both n and m.both n and m. Proving this is not fun.Proving this is not fun. Lets not do it, OK?Lets not do it, OK?

Euclid’s AlgorithmEuclid’s Algorithm

If I have two 10 digit numbers, If I have two 10 digit numbers, Euclid’s algorithm will take (at most) Euclid’s algorithm will take (at most) about 20 divisions.about 20 divisions.

Whereas, consecutive integer Whereas, consecutive integer checking could take 9,999,999,999 X checking could take 9,999,999,999 X 2 divisions.2 divisions.

How does Euclid’s algorithm gain How does Euclid’s algorithm gain such an advantage?such an advantage?

Euclid’s AlgorithmEuclid’s Algorithm

r r m mod n m mod n If r is not equal to zero then all the If r is not equal to zero then all the

numbers between m and r are NOT numbers between m and r are NOT gcd’s of m and n.gcd’s of m and n.

Its all about understanding the Its all about understanding the properties of the problem.properties of the problem.• What is the definition of a gcd?What is the definition of a gcd?• What is the definition of mod?What is the definition of mod?• The two are related.The two are related.

An AlgorithmAn Algorithm

A sequence of unambiguous A sequence of unambiguous instructions for solving a problem,instructions for solving a problem,

i.e., for obtaining a required output i.e., for obtaining a required output for any legitimate input for any legitimate input

in a finite amount of time.in a finite amount of time.

History LessonHistory Lesson

Muhammad ibn Musa al-KhwarizmiMuhammad ibn Musa al-Khwarizmi• Famous mathematician from Bagdad Famous mathematician from Bagdad

800 A.D800 A.D He wrote He wrote Al-jabr wa'l muqabalaAl-jabr wa'l muqabala (from (from

which our modern word "algebra" which our modern word "algebra" comes) comes)

The English word "algorithm" derives The English word "algorithm" derives from the Latin form of al-Khwarizmi's from the Latin form of al-Khwarizmi's namename

Stupid, irrelevant, personal Stupid, irrelevant, personal anecdoteanecdote

As an undergraduate, I had Indian As an undergraduate, I had Indian Professor who was an expert on Professor who was an expert on algorithms and pronounced the wordalgorithms and pronounced the word

A – grow – rid – em’sA – grow – rid – em’s Lets practice saying the word so we Lets practice saying the word so we

don’t sound like idiots.don’t sound like idiots.

Notion of algorithmNotion of algorithm

“computer”

problem

algorithm

input output

Example of computational problem: Example of computational problem: sortingsorting

Statement of problem:Statement of problem:• Input:Input: A sequence of A sequence of nn numbers <a numbers <a11, , aa22, …, a, …, ann>>

• Output:Output: A reordering of the input sequence <a A reordering of the input sequence <a ´́11, , aa´́

22, …, , …, aa´́

nn> so that a> so that a´́ii ≤ ≤ aa´́

jj whenever whenever ii < < jj

Instance: The sequence <5, 3, 2, 8, 3>Instance: The sequence <5, 3, 2, 8, 3>

Algorithms:Algorithms:• Selection sortSelection sort• Insertion sortInsertion sort• Merge sortMerge sort• (many others)(many others)

ExampleExample

Input: array Input: array a[1],…,a[n]a[1],…,a[n]

Output: array Output: array aa sorted in non-decreasing sorted in non-decreasing orderorder

Algorithm:Algorithm:

for i=1 to nfor i=1 to n

swap a[i] with smallest of a[i],…a[n]swap a[i] with smallest of a[i],…a[n]

Algorithm design strategiesAlgorithm design strategies

Brute forceBrute force

Divide and conquerDivide and conquer

Decrease and Decrease and conquerconquer

Transform and Transform and conquerconquer

Greedy approachGreedy approach

Dynamic Dynamic programmingprogramming

Backtracking and Backtracking and Branch and boundBranch and bound

Space and time Space and time tradeoffstradeoffs

Analysis of AlgorithmsAnalysis of Algorithms

How good is the algorithm?How good is the algorithm?• CorrectnessCorrectness• Time efficiencyTime efficiency• Space efficiencySpace efficiency

Does there exist a better algorithm?Does there exist a better algorithm?• Lower boundsLower bounds• OptimalityOptimality

Some Well-known Computational Some Well-known Computational ProblemsProblems

SortingSorting SearchingSearching Shortest paths in a graphShortest paths in a graph Minimum spanning treeMinimum spanning tree Primality testingPrimality testing Traveling salesman problemTraveling salesman problem Knapsack problemKnapsack problem ChessChess Towers of HanoiTowers of Hanoi Program terminationProgram termination

Why do we care?Why do we care?

SortingSorting SearchingSearching Shortest paths in a Shortest paths in a

graphgraph Minimum spanning treeMinimum spanning tree Primality testingPrimality testing Traveling salesman Traveling salesman

problemproblem Knapsack problemKnapsack problem ChessChess Towers of HanoiTowers of Hanoi Program terminationProgram termination

Variations of the solutions to Variations of the solutions to these problems are used to…these problems are used to… Schedule flightsSchedule flights Route Internet trafficRoute Internet traffic Make Google workMake Google work Prevent the stock-market from Prevent the stock-market from

crashingcrashing Assemble the human genomeAssemble the human genome Encrypt dataEncrypt data Save UPS millions of dollarsSave UPS millions of dollars Track terrorist cell Track terrorist cell

communication on the Internetcommunication on the Internet Make spell checkers workMake spell checkers work help keep you from going help keep you from going

through the walls in games like through the walls in games like Halo and CounterStrikeHalo and CounterStrike

Basic Issues Related to AlgorithmsBasic Issues Related to Algorithms

How to design algorithmsHow to design algorithms How to express algorithmsHow to express algorithms Proving correctnessProving correctness EfficiencyEfficiency

• Theoretical analysisTheoretical analysis• Empirical analysis (experiments)Empirical analysis (experiments)

OptimalityOptimality

What is an algorithm?What is an algorithm? Recipe, process, method, technique, procedure, Recipe, process, method, technique, procedure,

routine,… with following requirements:routine,… with following requirements:1.1. FinitenessFiniteness

terminates after a finite number of stepsterminates after a finite number of steps

2.2. DefinitenessDefiniteness rigorously and unambiguously specifiedrigorously and unambiguously specified

3.3. InputInput valid inputs are clearly specifiedvalid inputs are clearly specified

4.4. OutputOutput can be proved to produce the correct output given a valid can be proved to produce the correct output given a valid

inputinput

5.5. EffectivenessEffectiveness steps are sufficiently simple and basicsteps are sufficiently simple and basic

Why study algorithms?Why study algorithms?

Theoretical importanceTheoretical importance

• the core of computer sciencethe core of computer science

Practical importancePractical importance

• A practitioner’s toolkit of known algorithmsA practitioner’s toolkit of known algorithms

• Framework for designing and analyzing Framework for designing and analyzing algorithms for new problemsalgorithms for new problems

Important Types of ProblemsImportant Types of Problems

Sorting & SearchingSorting & Searching String matching & pattern matchingString matching & pattern matching Graph ProblemsGraph Problems Combinatorial ProblemsCombinatorial Problems Geometric ProblemsGeometric Problems Numerical ProblemsNumerical Problems

Sorting & SearchingSorting & Searching

Intrinsically relatedIntrinsically related• Sorting can make searching easierSorting can make searching easier

sort keysort key – an important attribute – an important attribute used to sort dataused to sort data

stable sortstable sort – preserves the relative – preserves the relative order of any two equal elementsorder of any two equal elements

in place sortingin place sorting – doesn’t require – doesn’t require extra memoryextra memory

search keysearch key – not necessarily unique – not necessarily unique

String matching & pattern matchingString matching & pattern matching

Amazingly well-studiedAmazingly well-studied Surprisingly difficult problem for certain Surprisingly difficult problem for certain

variationsvariations Tons of applications:Tons of applications:

• Approximate matching used in spell checkersApproximate matching used in spell checkers• Pattern matching used to find genes in DNAPattern matching used to find genes in DNA• Or, used to find patterns in the stock marketOr, used to find patterns in the stock market

Graph ProblemsGraph Problems Graphs can be used to model all sorts of Graphs can be used to model all sorts of

real-world environments, scenarios, and real-world environments, scenarios, and systems.systems.• Relationship modeling using graphs is very Relationship modeling using graphs is very

powerfulpowerful Representing the real-world as a graph Representing the real-world as a graph

can allow computers to solve all sorts of can allow computers to solve all sorts of problems.problems.

Graph related algorithms are being used Graph related algorithms are being used to detect terrorist cell communication via to detect terrorist cell communication via the Internet.the Internet.

Hell, the Internet itself is a massive graph.Hell, the Internet itself is a massive graph.

Combinatorial ProblemsCombinatorial Problems

Certain problems, like optimal scheduling Certain problems, like optimal scheduling or routing, can be reduced to finding the or routing, can be reduced to finding the optimal parameters among the set of all optimal parameters among the set of all possible parameterspossible parameters

But, the number of different parameter But, the number of different parameter combinations can become very large, too combinations can become very large, too large.large.

Reducing the combinatorial space of Reducing the combinatorial space of problems is a very important strategy in problems is a very important strategy in algorithms algorithms

Geometric ProblemsGeometric Problems

These problems can be as simple as These problems can be as simple as finding the intersection point of two finding the intersection point of two lines tolines to

Finding the Delaunay Triangulation of Finding the Delaunay Triangulation of the smallest convex polygon the smallest convex polygon containing a given set of points.containing a given set of points.

These types of problems arise from These types of problems arise from modeling all sorts of real-world modeling all sorts of real-world entities.entities.

Numerical ProblemsNumerical Problems

Problems involving mathematical Problems involving mathematical objectsobjects

solving equationssolving equations computing integralscomputing integrals efficiently evaluating functionsefficiently evaluating functions i.e., really boring stuffi.e., really boring stuff

Data StructuresData Structures

The assigned reading is all about common The assigned reading is all about common data structures used in algorithmsdata structures used in algorithms

Data structures are just one of many tools Data structures are just one of many tools that can be used to improve algorithmsthat can be used to improve algorithms

Perhaps they are the most important toolPerhaps they are the most important tool Do the reading and homeworkDo the reading and homework This material should not be new.This material should not be new.

HomeworkHomework

Read Section 1.4 (recap of data Read Section 1.4 (recap of data structures)structures)

pages 26-37pages 26-37 Answer the following questions:Answer the following questions:

• page 38, questions 1a, 1b, 2a, 2b, & 10page 38, questions 1a, 1b, 2a, 2b, & 10• Also, what exactly is a dictionary (not Also, what exactly is a dictionary (not

Webster's)Webster's)• Due Wed 1/28 (in class)Due Wed 1/28 (in class)