Upload
vukhanh
View
219
Download
1
Embed Size (px)
Citation preview
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Chapter 5:
Algorithms
Computer Science: An Overview
Tenth Edition
by
J. Glenn Brookshear
Presentation files modified by Farn Wang
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-2
Chapter 5: Algorithms
• 5.1 The Concept of an Algorithm
• 5.2 Algorithm Representation
• 5.3 Algorithm Discovery
• 5.4 Iterative Structures
• 5.5 Recursive Structures
• 5.6 Efficiency and Correctness
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-3
Definition of Algorithm
An algorithm is an
• ordered set of
• unambiguous,
• executable steps that defines a
• terminating process.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-4
Algorithm Representation
• Requires well-defined primitives
• A collection of primitives constitutes a
programming language.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-5
Folding a bird from a square piece of
paper
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-6
Origami primitives
折り紙
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-7
Problem Solving Steps
Designed the method in Hungary
teaching the son of a baron.
1. Understand the problem.
2. Devise a plan for solving the
problem.
3. Carry out the plan.
4. Evaluate the solution for
accuracy and its potential as a
tool for solving other problems.
George Pólya (1887-1985)
Stanford, ETH-Zurich
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-8
Getting a Foot in the Door
Basic approaches:
• Try working the problem backwards
• Solve an easier related problem
– Relax some of the problem constraints
– Solve pieces of the problem first (bottom up
methodology)
• Stepwise refinement: Divide the problem into
smaller problems (top-down methodology)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-9
A trial case study in solving problems
- Ages of Children Problem
• Person A is charged with the task of determining
the ages of B’s three children.
– B tells A that the product of the children’s ages is 36.
– A replies that another clue is required.
– B tells A the sum of the children’s ages.
– A replies that another clue is needed.
– B tells A that the oldest child plays the piano.
– A tells B the ages of the three children.
• How old are the three children?
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-10
Understanding the domain instances
for a solution
The only pair that needs clue 3 to distinguish.
• You can spend all the time fretting without a clue.
• Or you can start to play with the numbers.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-11
Representation of algorithms
- Pseudocode Primitives
• Assignment
name expression
• Conditional selection
if condition then action
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-12
Representation of algorithms
- Pseudocode Primitives (continued)
• Repeated execution
while condition do activity
• Procedure
procedure name (generic names)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-13
Representation of algorithms
- The procedure Greetings in
pseudocode
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-14
Iterative Structures
• Pretest loop:
while (condition) do
(loop body)
• Posttest loop:
repeat (loop body)
until(condition)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-15
The sequential search algorithm in
pseudocode
Assumption: sorted list.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-16
Components of repetitive control
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-17
Representation of algorithms
- flowchart
The while loop structure:
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-18
Representation of algorithms
- flowchart
The repeat-until loop structure
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-19
How to design an algorithm ?
- Intuition behind a sorting algorithm
Given a data item b and a sorted list d1,…,dk ,
we can easily insert b into d1,…,dk .
• sort d1
• sort d1, d2
• sort d1, d2, d3
• …
• sort d1, d2, …., dn
Insert d1 to the right position in .
Insert d2 to the right position in d1 .
Insert d3 to the right position in d1,d2 .
Insert dn to the right position in d1,…,dn-1 .
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-20
Test run of the algorithm in mind: Sorting the list Fred, Alex, Diana, Byron, and Carol alphabetically
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-21
Pseudo-code
for insertion sort - an example of loop algorithms
This is the pseudo-
code in the textbook.
I think it is a bad
example.
Why ?
Too close to the real code.
Too many details.
Difficult to understand.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
A better pseudo-code
for insertion sort
Procedure sort(List) {
Loop for N = 2 to |List| {
Insert List[N] to the right position in List[1..N-1]
to make List[1..N] sorted;
}
5-22
High-level
mathematical
notations.
High-level and
intuitive operations
in natural languages.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-23
Recursion
• The execution of a procedure leads to
another execution of the procedure.
• Multiple activations of the procedure are
formed, all but one of which are waiting for
other activations to complete.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Recursion
F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
F(n) {
if (n <= 0) return(0);
else if (n == 1) return 1;
else return F(n-1)+F(n-2);
}
5-24
How many times F() is called ?
f(n) = 1+f(n-1)+f(n-2); f(0)=1, f(1)=1;
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Recursion
- implemented with iteration F(n) = F(n-1) + F(n-2)
F(0) = 0
F(1) = 1
F(n) {
F(0) = 0; F(1) = 1; k = 2;
while (k <= n) { F(k)=F(k-1)+F(k-2); k=k+1;}
return (F(n));
}
5-25
How many steps ? f’(n) = 1 + n;
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
2013/04/17 stopped here.
5-26
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Recursion
- with improved performance
F(n) = F(n-1) + F(n-2) ; F(0) = 1; F(1) = 1
F(n) {
if (n < 0) exit(0);
else if (n == 0 || n == 1) return n;
else if (D[n] == -1)
D[n] = F(n-1)+F(n-2);
return(D[n]);
}
Main (n) {
k=0; while (k<=n) {D[k]=-1; k=k+1;}
return F(n);
}
5-27
Initialize the D array.
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-28
Recursion for search ?
- Trying out an example for the entry John
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-29
Binary search technique
- A first draft
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-30
Binary search algorithm
- pseudocode with recursion concept
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Binary search in recursion
bsearch(L[j,k], t) {
if (j>k) return search failure.
m = (j+k)/2;
if (t==L[m]) return m;
else (t<L[m]) return bsearch(L[j,m-1],t);
else return bsearch(L[m+1,k], t);
}
5-31
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Binary search in recursion
bsearch(L[j,k], t) {
while (j<=k) {
m = (j+k)/2;
if (t==L[m]) return m;
else (t<L[m]) k = m-1;
else j=m+1;
}
return search failure.
} 5-32
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-33
Conceptualizing the recursion (I/III)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-34
Conceptualizing the recursion (II/III)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-35
Conceptualizing the recursion (III/III)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-36
Algorithm Efficiency
• Measured as number of instructions
executed
• Big O notation: Used to represent
efficiency classes. Examples:
– Insertion sort for n elements is in O(n2)
– binary search for n elements is in O(log n)
– F(n) iterative: O(n)
– F(n) purely recurisive: O(f(n)),
• f(n)=1+f(n-1)+f(n-2).
• Best, worst, and average case analysis
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-37
Applying the insertion sort in a worst-
case situation
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-38
Graph of the worst-case analysis of the
insertion sort algorithm
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-39
Graph of the worst-case analysis of the
binary search algorithm
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-40
Algorithm Efficiency
Big O concepts for worst-case analysis
• Insertion sort for n elements is in O(n2)
– 0.00003n2 + 0.00001 n + 0.00009 sec
– with efficient compiler & CPU
• binary search for n elements is in O(log n)
– 123478999 log n sec
– with lousy compiler & CPU
• As n becomes bigger and bigger,
0.00003n2 + 0.00001 n + 0.00009
>> 123478999 log n sec
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-41
The lower-bound for worst cases ?
- An example
Chain Separating Problem
• A traveler has a gold chain of seven links.
• He must stay at an isolated hotel for seven nights.
• The rent each night consists of one link from the chain.
• What is the fewest number of links that must be cut so that the traveler can pay the hotel one link of the chain each morning without paying for lodging in advance?
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-42
Separating the chain using only
three cuts
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-43
Solving the problem with only one
cut
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Lower-bound of worst-case
efficiency
How do we know if an algorithm is good
enough for a task ?
• What is the assumptions of basic operations ?
• Have we tied up ourselves ?
Lemma: One cut is the minimum.
Proof:
• At least one chain is given in the first
morning.
• So one cut is the least.
5-44
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Lower-bound of worst-case
efficiency
other techniques
• reduction:
– translate any instance of a problem A with
known lower-bound to the given problem B.
Then we can show B’s lower-bound is no
lower than A’s.
• information theory:
– Is log(n) the best lower-bound for the worst
case of searching algorithms ?
– Is n log(n) the best for sorting algorithms ?
5-45
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-46
Software Verification
• Proof of correctness
– Assertions
• Preconditions
• Loop invariants
• Testing
– send testcases to SW
– check the SW behavior or output
• Verification
– analysis of models of SW
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley 5-47
The assertions associated with a
typical while structure
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Assertions
- an example on recursive code. F(n) = F(n-1) + F(n-2) , F(0) =0, F(1) = 1
F(n) {
F(0) = 0; F(1) = 1; k = 2;
while (k <= n) {
F(k)=F(k-1)+F(k-2); k=k+1;
}
return (F(n));
}
5-48
precondition: true
precondition: k=2
Loop invariants:
k<=n,
i<k, F(i)=F(i-1)+F(i-2)
can be proved with
mathematical
induction.
precondition: F(n)=F(n-1)+F(n-2)
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Various algorithms (I)
• Numerical
– factorization, prime number testing, …
• Constraint solving
– equations, inequalities, differential systems
• Sequence:
– Sorting, searching, merging, pattern matching,
data mining
5-49
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Various algorithms (II)
• Graph algorithms
– searching, traversal,
– critical paths, flow analysis, ….
– structure identification,
– spanning trees
– ……
5-50
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Various algorithms (III)
• Graph algorithms
– searching, traversal,
– critical paths, flow analysis, ….
– structure identification,
– spanning trees
– ……
5-51
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Various algorithms (III)
• Logics
– solutions, Boolean operations, greatest
fixpoint, least fixpoint
• Distributed algorithms
– leader elections,
– snapshots,
– deadlock detection,
– consensus
– ….
5-52
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Algorithms for nonterminating
systems
• Operating systems
– deadlock
– mutual exclusion
– security
• Medical systems
5-53
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
When there is no algorithm
• genetic algorithm
• swamp intelligence
• ant intelligence
• machine learning
• heuristics
• solution as proofs
• …..
5-54
Copyright © 2008 Pearson Education, Inc. Publishing as Pearson Addison-Wesley
Complexities
Average case ? Worst case ?
Algorithms:
• time complexities
• space complexities
Problems:
• How efficient can we make the program
run ?
• Is there an algorithm ?
5-55