ANALYSIS OF ALGORITHMS

Dr inż. ZDZISŁAW PÓLKOWSKI

Polkowice, 2015

University of PitestiDolnośląska Wyższa Szkoła Przedsiębiorczości i Techniki

w Polkowicach

Ion Paul Mihai

What is Algorithm ? What is Algorithm Design ? How to Design an Algorithm ? Graph Algorithm Flow Chart Importance of Algorithm Design Conclusion

TOPICS MENU

An algorithm (pronounced AL-go-rith-um) is a procedure or formula for solving a problem. The word derives from the name of the mathematician, Mohammed ibn-Musa al-Khwarizmi, who was part of the royal court in Baghdad and who lived from about 780 to 850. Al-Khwarizmi's work is the likely source for the word algebra as well.A computer program can be viewed as an elaborate algorithm. In mathematics and computer science, an algorithm usually means a small procedure that solves a recurrent problem.

http://whatis.techtarget.com/definition/algorithm

What is algorithm ?

What is algorithm-design ?

Algorithm design is a creative activity that is not subject to recipes. The existence of many important problems for which no efficient algorithms are known is an evidence of this fact. In practice, the space of choices to develop algorithms is enormous. For this reason, and in opposition to the development from scratch, the task of solving algorithmic problems can be broached applying more abstract design techniques. These techniques offer common solving strategies for different problems. A design technique is often expressed in pseudocode as a template that can be particularized for concrete problems. Let us name this template algorithm schemas.

http://hillside.net/europlop/HillsideEurope/Papers/EuroPLoP1998/1998_Galve-FrancesEtAl_AlgorithmDesignByPatterns.pdf

How to design algorithm ??There are four FAMOUS techniques of algorithm design:

Greedy Algorithm Divide and Conquer Dynamic Programming Back Tracking

How to design algorithm ?

www.cis.upenn.edu/.../36-algorithm-types.ppthttp://thumbs.dreamstime.com/z/algorithm-27863175.jpg

Greedy Algorithm

A greedy algorithm is a mathematical process that looks for simple, easy-to-implement solutions to complex, multi-step problems by deciding which next step will provide the most obvious benefit.

Such algorithms are called greedy because while the optimal solution to each smaller instance will provide an immediate output, the algorithm doesn’t consider the larger problem as a whole. Once a decision has been made, it is never reconsidered.

Greedy algorithms work by recursively constructing a set of objects from the smallest possible constituent parts. Recursion is an approach to problem solving in which the solution to a particular problem depends on solutions to smaller instances of the same problem. The advantage to using a greedy algorithm is that solutions to smaller instances of the problem can be straightforward and easy to understand. The disadvantage is that it is entirely possible that the most optimal short-term solutions may lead to the worst possible long-term outcome.

http://whatis.techtarget.com/definition/greedy-algorithm

Image of greedy algorithm

http://images.devshed.com/da/stories/Greedy_Algorithm/greedy_algorithm_html_m5c526e56.png

Divide and Conquer

Divide and Conquer Algorithm

Divide-and-conquer is a top-down technique for designing algorithms that consists of dividing the problem into smaller subproblems hoping that the solutions of the subproblems are easier to find and then composing the partial solutions into the solution of the original problem.Little more formally, divide-and-conquer paradigm consists of following major phases:-Breaking the problem into several sub-problems that are similar to the original problem but smaller in size,-Solve the sub-problem recursively (successively and independently),-Combine these solutions to subproblems to create a solution to the original problem.Binary Search (simplest application of divide-and-conquer)Binary Search is an extremely well-known instance of divide-and-conquer paradigm. Given an ordered array of n elements, the basic idea of binary search is that for a given element we "probe" the middle element of the array. We continue in either the lower or upper segment of the array, depending on the outcome of the probe until we reached the required (given) element.

http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/divide.htm

Image of Divide and Conquer

http://www.enggpedia.com/answers/?qa=blob&qa_blobid=12378523199884154543

Dynamic Programming

Example: Fibonacci numbers computed by iteration.

Warshall ’s algorithm implemented by iterations.

Dynamic Programming Algorithm

Dynamic programming is a fancy name for using divide-and-conquer technique with a table. As compared to divide-and-conquer, dynamic programming is more powerful and subtle design technique. Let me repeat , it is not a specific algorithm, but it is a meta-technique (like divide-and-conquer). This technique was developed back in the days when "programming" meant "tabular method" (like linear programming). It does not really refer to computer programming. Here in our advanced algorithm course, we'll also think of "programming" as a "tableau method" and certainly not writing code. Dynamic programming is a stage-wise search method suitable for optimization problems whose solutions may be viewed as the result of a sequence of decisions.

http://www.personal.kent.edu/~rmuhamma/Algorithms/MyAlgorithms/Dynamic/dynamicIntro.htm

Images of Dynamic Programming

http://upload.wikimedia.org/wikipedia/commons/thumb/0/06/Fibonacci_dynamic_programming.svg/108px-Fibonacci_dynamic_programming.svg.pnghttp://3.bp.blogspot.com/_XfmDdL0570Q/TD68ZwYDNbI/AAAAAAAAADI/b0U9ZBOBg2A/s1600/arbOpp.png

Example: Eight queens puzzle. Traveling salesman problem.

Back Tracking Algorithm

Backtracking is a general algorithmic technique that considers searching every possible combination in order to solve an optimization problem. Backtracking is also known as depth-first search or branch and bound. By inserting more knowledge of the problem, the search tree can be pruned to avoid considering cases that don't look promising. While backtracking is useful for hard problems to which we do not know more efficient solutions, it is a poor solution for the everyday problems that other techniques are much better at solving.However, dynamic programming and greedy algorithms can be thought of as optimizations to backtracking, so the general technique behind backtracking is useful for understanding these more advanced concepts. Learning and understanding backtracking techniques first provides a good stepping stone to these more advanced techniques because you won't have to learn several new concepts all at once.

http://en.wikibooks.org/wiki/Algorithms/Backtracking

Image of back tracking

http://www.worldit.info/wp-content/uploads/2010/03/Depthfirst1.png http://upload.wikimedia.org/wikipedia/commons/thumb/1/1e/Backtracking-no-backjumping.svg/250px-

Backtracking-no-backjumping.svg.png

Graph Algorithm

Graph Algorithm

What is a graph?It's an abstract notion, used to represent the idea of some kind of connection between pairs of objects. A graph consists of:-A collection of "vertices", which I'll usually draw as small circles on the blackboard, -A collection of "edges", each connecting some two vertices. I'll usually draw these as curves on the blackboard connecting the given pair of vertices.For this definition it doesn't matter what the vertices or edges represent -- that will be different depending on what application the graph comes from. It also doesn't matter how I draw the graph, the only part that matters is which pairs of vertices are connected with each other.There are two different types of graph that we'll commonly see.

https://www.ics.uci.edu/~eppstein/161/960201.htmlhttp://www.jambonewspot.com/wp-content/uploads/2015/05/graph.gif

-In the other type of graph, known as a directed graph, an edge goes from one of the vertices, towards the other. In this case I'll draw an arrowhead at the vertex towards which the edge is going. If we drew a graph like the handshake graph described above, but instead connected two people if one had written a letter to the other, the result would be directed: if I've written a letter to you, you may not have written a letter back to me.In one type, known as an undirected graph it doesn't matter which end of the graph is which -- the relation between the two is symmetric. In that case I'll just draw the edges as an undecorated curve.-For instance, suppose we draw a graph with one vertex for every person in the U.S., and one edge connecting any two people who have shaken hands with each other. If I've shaken hands with you, you've shaken hands with me, so each edge is symmetric. It seems to be true that graphs like this have very small {\em diameter}: you can connect any two people with a very short chain of handshakes.-In the other type of graph, known as a directed graph, an edge goes from one of the vertices, towards the other. In this case I'll draw an arrowhead at the vertex towards which the edge is going.

Image of Graph Algorithm

http://www.psdgraphics.com/file/red-business-graph.jpg

Flow chart• A flowchart is a type of diagram that

represents an algorithm, workflow or process, showing the steps as boxes of various kinds, and their order by connecting them with arrows.

• Flowcharts are used in analyzing, designing, documenting or managing a process or program in various fields.

Flow chart

http://nctide.org/Fall2014/Tim%20Huffman%20-%20Process%20Mapping%20Presentation.pdf

Symbols of Flow Chart

http://creately.com/blog/wp-content/uploads/2011/11/Flowchart-Symbols1.pnghttp://cdn.vectorstock.com/i/composite/15,86/flowchart-symbols-vector-251586.jpg

• It is used to store and access large quantities of data efficiently.

• It is used to solve complex computational problems and to design of good programs

• It is important to justify an algorithm correctness mathematically

• It provides clear , simple and unambiguous description

http://www.slideshare.net/mahtabnahid/our-presentation-on-algorithm-designhttp://openclassroom.stanford.edu/MainFolder/courses/IntroToAlgorithms/cs161logo.png

Importance of algorithm design

Conclusion

Usually a given problem can be solved using various approaches however it is not wise to settle for the first that comes to mind. More often than not, some approaches result in much more efficient solutions than others. Consider again the Fibonacci numbers computed recursively using the decrease-and-conquer approach, and computed by iterations using dynamic programming. In the first case the complexity is O(2n), while in the second case the complexity is O(n). On the other hand, consider sorting based on decrease-and-conquer (insertion sort) and brute force sorting. For almost sorted files insertion sort will give almost linear complexity, while brute force sorting algorithms have quadratic complexity.

The basic question here is: How to choose the approach? First, by understanding the problem, and second, by knowing various problems and how they are solved using different approaches.

http://faculty.simpson.edu/lydia.sinapova/www/cmsc250/LN250_Weiss/L28-Design.htm#conclusion