Fall 2008NOEA – Computer Science1 Session 11 Revisiting Graph Algorithms Algorithm Patterns Algorithm Efficiency Difficult and Unsolvable Problems

Fall 2008 NOEA – Computer Science 1

Session 11

Revisiting Graph AlgorithmsAlgorithm PatternsAlgorithm Efficiency

Difficult and Unsolvable Problems


Læringsmål

• Kunne redegøre for begrebet algoritme pattern• Kende nogle forskellige algoritme patterns• Kunne redegøre for algoritmekompleksitet og

forskellige kompleksitetsklasser• Kunne forklare eksempler på algoritmer i forskellige

kompleksitetsklasser


Example: Prim’s Algorithm


Example: Dijkstra’s Algorithm


Greedy Algorithms

• C: a collection of candidates• S: a collection of candidates representing the partly

solution, eventually becoming the total solution• Solution: a function that determines if a collection of

candidates is a solution to the problem• Feasible: a function that checks if collection of

candidates may become a solution (not necessary an optimal solution, but any solution)

• Select: a function that returns a measure for how good a candidate is


The Greedy Pattern

proc Greedy( in C: <<collection of candidates>>;out S: <<collection of candidates>>)pre C is a collection of all candidates and a solution existspost S is a collection of candidates constituting an optimal solutionS:= []; -- initially S is emptywhile not Solution(S) and C <> [] dox:= an element in C that maximises Select(x)C:= C - [x] -- x is removed from the collection of candidatesif Feasible(S+[x]) then S:= S+[x]; -- x is added to the solutionendwhile

endproc


Prim’s Algorithm and

the Greedy Pattern

proc prim(in G: Graph, v: vertex; out T: Tree)pre G is connected, v is in Gpost T is a minimum spanning tree for GT:= [v]mark(v)while there are unvisited vertices do

Select the minimum weight edge (w,u) from a vertex w in T to an unvisited vertex u

mark(u)add u and (w,u) to T

endwhileendproc prim What

happened to Feasible?


Kruskal’s Algorithm(finding a minimum spanning tree)

Input: a set of edges, E Output: a tree, T

T:= []; -- T, initially T is emptywhile noOfEdges(T) < n-1 and E <> [] do

x:= an edge, (v,w), from E with minimum weightE:=E - [x] – remove x from the candidatesif (v,w) + T is acyclic then T:=T+[x] -- x is added to the solutionfi

endwhile Exercise:Define: C:

S:Solution():Feasible():Select():

Compare with the pattern.


Dijkstra’s Algorithm and the Greedy Pattern

proc dijkstra(in G: GraphMatrix, v: vertexNo; out w: arrayOfCosts)pre truepost w contains the length of the shortest path from v to all other

vertices in G (if no path exists to u, then w[u]==“infinity”)S:= [1]for v:= 1 to G.size do

w[v]:= G[1,v];

for v:= 2 to N doselect u not in S, so w[u] is a minimum in wS:= S + [u];for each u not in S do

if w[u] > w[v] + G[v,u] then w[u]:= w[v] + G[v,u];endfor

endproc dijkstra

Does this follow the greedy pattern?


Algorithm Patterns

• Sweep Algorithms• Search Algorithms• Merge Algorithms• Divide and Conquer Algorithms• Greedy Algorithms• Backtracking Algorithms• Dynamic Programming – improving

efficiency of recursive algorithms


Divide and Conquer Algorithms

proc DivCon(in P: Problem; out S: Solution)if P is simple then

Simple(P, S)else

Divide(P, P1, P2) DivCon(P1, S1)

DivCon(P2, S2) Combine(S, S1, S2)

endifendproc DivCon


Sorting and Divide and Conquer Algorithms

proc Sort(in S: Sequence; out S’: Sequence)if S.length>1 then

Split(S, S1, S2)Sort(S1, S’1)

Sort(S2, S’2)

Join(S’1, S’2, S’ )endif

endproc Sort

• easy split/hard join• hard split/easy join


MergeSort

proc mergeSort(inout v: vector; in l, u: int)pre truepost The segment v[l..u] is sorted in increasing order

if l<u thenm:= (l+ u) div 2mergeSort(v,l,m)mergeSort(v,m+1,u)merge(v,l,m,u)

-- total merge of v[l..m] and v[m+1..u]; returns vfi

endproc mergeSort


QuickSort

proc quickSort(inout v: vector; in l, u: int)pre truepost the segment v[l..u] is sorted in increasing order

if l<u thenp:= -- Select some index as pivot (the first for instance)-- do the pivot process, so each element in v[l..p-1]-- is less than each element in v[p+1..u]quickSort(v,l,p-1)quickSort(v,p+1,u)

fi

endproc quickSort


Efficiency of Merge- and QuickSort

• mergeSort:– O(n log n) in worst case– O(n) additional memory– In average a larger overhead than quickSort

• quickSort:– In average :

• O(n log n) (worst case O(n2) – if for instance input is sorted)

– additional memory for the recursion stack:• O(log n) (O(n) in worst case)

– Choosing pivot as ”mean of three” (or the median) will in practice often make worst case very unlikely.


Some Sorting Algorithms

• See them work:http://cg.scs.carleton.ca/~morin/misc/sortalg/

http://cg.scs.carleton.ca/~morin/misc/sortalg/




Backtracking algorithms(TicTacToe, 8 Queens, Sudoko and others)

• A recursive strategy:– in each step (j) the algorithm “guesses” a partial

solution (picking a candidate at random)– If there is a candidate to pick, the algorithm calls it

selves recursively for step j+1– If there at some step is no candidate to pick for the

solution, then the algorithm returns (backtracks) to the previous level


8-Queens Problem

• Place 8 queens on a chessboard, so no queen can attack an other. (A queen may attack horizontally, vertically and diagonally):


The Backtrack-Patternproc backtrack(in j) if j<= n then while any more candidates at level j and a solution is not found do choose some candidate if the candidate is a possible solution at this level then add the candidate to the total solution backtrack(j+1) if no more candidates at level j+1 then remove the chosen candidate from level j from the total solution

choose a new candidate fi fi endwhileendproc backtrack


Dynamic Programming

• Improving running time for recursive algorithms using additional space:

• For instance. Finding the nth Fibonacci number:func fib(in n) if n<=1 then return n else return fib(n-1)+fib(n-2) fiendfunc fib

• efficiency (draw a recursion tree)?


Recursion Trees

• The execution of a recursive algorithm can be modelled using a tree, where each node models a recursive call:– The number of nodes indicates the time complexity– The depth of the tree indicates the size of the

recursion stack, and hence the additional space needed due to recursion


Improving Running Time for Fib // We will use an global array:t: array[0..max] of intfunc fib1(in n) if t[n] = empty then if n<=1 then t[n]:=n else t[n]:= fib(n-1) + fib(n-2) fi fi fib:= t[n]endfunc

Tail Recursion!


Tail Recursion

• If the recursive call is the last statement in the method, then it is called tail recursion.

• In the case of tail recursion, recursion may be removed: – Since the algorithm returns just after returning

from the recursive call, there is no need for remembering temporary results at the recursion stack

– In this case the recursive algorithm may be formulated iteratively with no extra costs


A Word on Divide and Conquer and Efficiency

• The efficiency of solution developed applying Divide and Conquer depends on how the problem is divided into sub problems:– Binary Search: The problem is halved at each step – very

efficient: O(log n)– MergeSort: The problem is divided into two sub problems at

each step. The sub problems are of half the size of the original problem: O(nlogn)

– Fibonacci-numbers or the Towers of Hanoi: the problem is divided into two sub problems each of almost the same size as the original problem (n-1): O(2n)


Exercise

1. Write an iterative version of Fib, that uses no extra space. (Remember the version on slide 20 has tail recursion)


Recursive Data Structures

• General Trees• Lists of lists• Composite Pattern

27

General Trees• Structure:

– A single node (the root), or– A root with an arbitrary

number of children (subtrees)• Cannot be implemented as

binary trees since the number of subtrees may vary a lot

• For instance: – Directory structure on a

computer.– Organisation structure– Project Management– Etc.

A

C B D

E F G H I

28

Implementation of General Trees

• Recursive Structures:– Linked implementation:

general lists:• Lists with list-elements

– Object-oriented:• Composite Pattern

A

C B D

E F G H I

29

Linked Implementation

30

The Object-Oriented Implementation: The Composite

PatternAbstractTree

Leaf Tree

0..*

•Examples in C#:OrganisationProjectFile Directories


The Algorithmic Space

UnsolvableProblems

DifficultProblems

Practical SolvableProblems


Practical Solvable Problems

• Problems that can be solved by algorithms with polynomial running time.

• That is, algorithms with complexity

O(nk)

k been some fixed integer constant


Some Sample Complexities

• Kh fig.


Some Difficult Problems - O(2n)

• Traveling Salesman– Given a collection of towns connected with roads or

railroads in a weighted graph:– find the shortest path from one town (the source) that visits

all other towns exactly once and ends in the source (a cycle) • Hamilton-Circuit:

– Given some graph: does there exist a cyclic path passing all vertices?

• Scheduling Problems:– Give a collection of lectures, a collection of classes and a

collection rooms:– Make schedule where no resource is booked more than

once in same period.


• Logic Satisfiability:– Given some arbitrary Boolean expression:

A && B || !C && D …etc– Is it possible to assign truth values to all the variables, so the

expression becomes true?• Map Colouring

– Any map can be coloured with four colours, so no two adjacent countries have the same colour.

– Many maps can be coloured using only three colours.– Only maps with no points where an odd number of countries is

adjacent can be coloured with two colours.– Determine if a map can be coloured with three colours.

• Graph Partition– Split the vertices of a graph in two sets (approximately of the same

size – balanced), so a minimum number of edges crosses the boundary between the two sets.

– Many applications uses this: data clustering in data warehouse and data mining, divide-and-conquer solutions, parallel architectures and algorithms, packet routing in distributed networks, integrated circuit layout and many more.

– O(logn) approximations in polynomial time are known

At most O(logn) worse

than an optimal

solution, n is the number of

vertices.


Unsolvable Problems

• Automatic generation of programs

• Automatic proof of correctness of a program

• The Halting Problem:– Is it possible to write an algorithm that given an arbitrary

program as input determine whether the program terminates (halts) or loops forever?


The Halting Problem

• We are going to prove that the Halting Problem has no algorithmic solution. We will make the proof by showing a contradiction:

• We will assume that there is an algorithmic solution, and then show that this assumption leeds to a contradiction.

• So assume that the algorithm halt(a) returns true, if the program a halts and false if not.


No consider the following program called evil:

void evil(){ if(halt(evil())) while(true);//indefinite loop else break;}

• If halt(evil()) returns true (that is that evil() will terminate), then evil() enters in an infinite loop (loops forever). So evil() does not terminate after all???

• If halt(evil()) returns false (that is that evil() will not terminate), then evil() enters the else part and terminates???

• So in either case halt(evil()) gives the wrong answer, and hence our assumption about the existence of halt is wrong, and we conclude that halt cannot exist?


Læringsmål

• Kunne redegøre for begrebet algoritme pattern• Kende nogle forskellige algoritme patterns• Kunne redegøre for algoritmekompleksitet og

forskellige kompleksitetsklasser• Kunne forklare eksempler på algoritmer i forskellige

kompleksitetsklasser

Documents

Fall 2008NOEA – Computer Science1 Session 11 Revisiting Graph Algorithms Algorithm Patterns Algorithm Efficiency Difficult and Unsolvable Problems