Embed Size (px)
Citation preview
SVU CS502
Design and Analysis of Algorithms Lecture-‐1: Introduction
Dr. Chung-‐Wen Albert Tsao [email protected]
http://408codingschool.com/CS502_Algorithm
1/12/161
SVU CS502
(1) CS502 DAA, Wed •Fnu Heena, [email protected] • Phaneendra Ginjupalli, [email protected]
(2) CS502 DAA, Friday •Madhu Narayana, [email protected] • Sindhuja Ghattamaneni, [email protected]
Graders:
SVU CS5021/12/163
About Myself• Me: Chung-‐Wen (Albert) Tsao, originally from Taiwan • PhD Computer Science from UCLA. • 20 years industry R&D • Mostly in EDA industry • VLSI Algorithms
• Teaching • CS500 OSD • CS502 Algorithm design • C++/Python Programming in San Jose City College
SVU CS502
• Textbook • Introduction to Algorithms, Third Edition by Thomas H. Cormen, Charles E. Leiserson, Ronald L. Rivest, Clifford Stein. Publisher: MIT Press (July 31, 2009). ISBN-‐10: 0262033844. ISBN-‐13: 978-‐0262033848
• Course website • http://www.408codingschool.com/CS502_Algorithm • Lecture notes, homework assignments, and solutions will be posted • Some of slides are based on material from
• Prof. Eugene Chang, • Prof. K.K. Low • Prof. Jianhua Ruan, The University of Texas at San Antonio
41/12/16
SVU CS502
Teaching Plan (subject to change)
1/12/165
Week Topic Chapters 1 Introduction Ch. 1, 2 2 Asymptotic Analysis Ch. 3, 3 Divide-‐and-‐Conquer Ch. 4 4 Heapsort Ch. 6, 5 Quicksort, Sorting in Linear Time Ch. 7,9 6 Data Structures, Hash Tables Ch. 10, 11 7 Binary Search Tree Ch. 12, 8 Mid-‐Term Exam 9 Red-‐Black Trees Ch. 13 10 B-‐Trees Ch. 18, 11 Dynamic Programming Ch. 15 12 Greedy Algorithms Ch. 16 13 Graph Algorithms Ch. 22,23 14 Graph Algorithms Ch. 24,25 15 Final Exam
SVU CS502
Grade Policies• Homework/Quiz-20% • Lab - 10% • Midterm exam – 30% • Final exam – 30% • Both exams are comprehensive, i.e. based on all material covered to date. • Most questions will be from HW and quiz in class
• Class participation – 10% • Attendance • Active during the class: answering questions, suggesting solutions, etc.
1/12/166
• How to succeed in this class? • Or how to fail this class?
7
SILICON VALLEY UNIVERSITY CONFIDENTIAL
How to Fail this class?
• Cheat • No motivation. • Do not practice the homework questions. • A good portion of HW will be in Quiz/Midterm/Final.
• Be late or skip class, midterm or final • Do not earn extra credit. • Show no respect
➔ If you do not want to fail, I can help you succeed.
January 12, 20168
SVU CS502
What is an algorithm?
• A programmable procedure, i.e., a sequence of computational steps acceptable by computers, to solve a problem. • Given input (problem instances) and produce output (solutions). • Math vs. Computer Science • In Math, we look for closed-‐form formula or equations • When no equations found, algorithms and computer are the tools to help
• Example: • Algorithms are like the strategies or tactics to win the chess games, or logic puzzles (when NO obvious closed-‐form formula available.)
1/12/169
SVU CS502
Types of Algorithm Problems and Evaluation Criteria
1/12/1610
• Constraint Satisfaction problems • Given constrains • Find the solutions that satisfy given constraints • Example: Most logic puzzles
• Optimization problems • Given constrains and an objective function • Find solutions that satisfy constraints & maximize the objective function
• Example: Traveling Salesman Problem (TSP)
• Algorithm evaluation criteria : Correctness, Efficiency, Quality (for optimization problems)
SVU CS502
Algorithm Design Techniques
• Brute Force • Enumerate all possibilities
• Backtracking • Prune infeasible
• Branch and Bound • Prune infeasible & non-promising
• Greedy • Repeatedly do what is best now
• Dynamic Programming • Break problem into overlapping
subproblems • Transform & Conquer
• Convert problem to another one • Divide & Conquer
• Divide problem recursively into subproblems and combine sub-solutions
• Iterative (Decrease & Conquer) • Heuristic and approximation algorithms
1/12/1611
•Solve problems in the most obvious ways • Enumerate all possibilities, and simply pick up solutions from them • Pros – Often simple to implement • Cons – May do more work than necessary (inefficiency) – Sometimes, NOT that obvious
12
Brute-‐force Algorithms
SVU CS502
Sorting: Brute-‐Force
• Input: A random array of integers called arr[] • Output: arr[i] <= arr[j] iff i < j • Sorting: Brute-‐Force Approach
• Generate n! permutations • Select one of them that gives the correct ordering. • Example: Given a=3,b=2,c=1, Generate all permutations
• Generate permutation abc,acb,bac,bca,cab,cba • Only cab gives the correct ordering.
• We seek algorithms that are both correct and efficient • Can we do better?
1/12/1613
SVU CS502
Selection Sort (Decrease and Conquer)
1/12/1614
5
4
3
2
1
0
#Comparisons
Done!
• Find smallest item, move to 1st location (n comparisons).
• Find next smallest item, move to 2nd location (n-‐1 comparisons).
• … • Repeat util the final location is reached.
• Time Complexity ??? • # comparison and swapping
SVU CS502
# Comparisons (Maximum)
5 2 4 6 1 3
2 5 4 6 1 3
2 4 5 6 1 3
2 4 5 6 1 3
1 2 4 5 6 3
1 2 3 4 5 6Done!1/12/16
1
2
3
4
5
Insertion Sort (Decrease and Conquer)
SVU CS502
Running time comparison
1/12/1616
Insertion Sort Selection Sort
Best case: O(N), Worst case=O(N2)
Best case: O(N2) Worst case: O(N2)
∂∂ ∂
∂
∂
∂
∂
Example: House Robber
• You are a professional robber planning to rob houses along a street. • Each house has a certain amount of money stashed • You will trigger alarm if you break into any two adjacent houses. • Example:
• Money in 7 houses: M=[5, 5, 6, 4, 0, 1, 5] • Houses robbed : [1, 0, 1, 0, 0, 0, 1] • Money robbed : $16
• Solution: • Brute Force: generate all possible (2^n) scenarios, and pick the best. • Can we do better?
17
Example: The Frog Puzzle http://britton.disted.camosun.bc.ca/frog_puzzle.htm
18
Frogs move (forward only) by hopping over a frog or sliding onto the empty spot (lily pad). ~
Send all frog Z’s to the right hand side of the pond, and all A’s to the left.
Z Z Z A A A
How many moves to get the frogs A’s and Z’s interchanged?
19
Brute-Force Solution: (Enumerate all possibilities)
• Define “state” as the positions of frogs. • Starting from the initial state (ZZ-AA) • Enumerate all Unique states from each state at current step. • Repeat the above step until the target state (AA-ZZ) is reached.
0: ZZ-AA <—— Initial state 1: -ZZAA Z-ZAA ZZA-A ZZAA- <—- New states reached in 1 step
2: ZAZ-A Z-AZA <—- New states reached in 2 steps
3: ZA-ZA ZAZA- -ZAZA <—- New states reached in 3 steps
4: -AZZA ZAAZ- ZA-AZ AZ-ZA <—- New states reached in 4 steps 5: A-ZZA ZAA-Z -AZAZ AZAZ- <—- New states reached in 5 steps
6: A-ZAZ AZA-Z <—- New states reached in 6 steps
7: AAZ-Z A-AZZ <—- New states reached in 7 steps
8: AA-ZZ <—- The target states reached in 8 steps
Example: 2 frogs at each side
20
How many moves to get the frogs interchanged?
0: ZZZ-AAA <—— Initial state 1: Z-ZZAAA ZZ-ZAAA ZZZA-AA ZZZAA-A <—- new states reached in 1 step 2: -ZZZAAA ZZAZ-AA ZZ-AZAA ZZZAAA- <—- new states reached in 2 steps 3: ZZA-ZAA ZZAZA-A ZZAZAA- -ZZAZAA Z-ZAZAA 4: Z-AZZAA ZZAAZ-A ZZA-AZA ZAZ-ZAA 5: -ZAZZAA ZA-ZZAA ZZAA-ZA ZZAAZA- Z-AZAZA ZAZAZ-A 6: AZ-ZZAA -AZZZAA ZZAAAZ- ZZAA-AZ -ZAZAZA ZA-ZAZA ZAZA-ZA ZAZAZA- 7: A-ZZZAA ZZAAA-Z AZ-ZAZA -AZZAZA ZAAZ-ZA ZA-AZZA ZAZAAZ- ZAZA-AZ 8: A-ZZAZA AZAZ-ZA ZAA-ZZA ZAAZAZ- -AZAZZA ZAZAA-Z ZA-AZAZ 9: AZA-ZZA AZAZAZ- ZAAZA-Z A-ZAZZA -AZAZAZ ZAA-ZAZ 10: A-AZZZA AZAZA-Z ZAA-AZZ AAZ-ZZA A-ZAZAZ ZAAAZ-Z 11: AA-ZZZA AZA-AZZ ZAAA-ZZ AAZ-ZAZ <—- new states reached in 11 steps 12: A-AZAZZ AZAA-ZZ AA-ZZAZ AAZAZ-Z <—- new states reached in 12 steps 13: AA-ZAZZ AAZA-ZZ <—- new states reached in 13 steps 14: AAAZ-ZZ AA-AZZZ <—- new states reached in 14 steps 15: AAA-ZZZ <—- Target state reached in 15 steps
Example: 3 frogs at each side
21
Conjecture: # Moves in Frog Puzzles
Conjecture: The least number of moves is expected as shown below
number of frogs a side N: 1 2 3 4 5 6 7 8 … N number of moves F(N): 3 8 15 24 35 ? ? ? … ?
F(N) = (N+1)^2-‐ 1 Mathematical Proof?
Example: The Three-‐Jug Problem
SVU CS5021/12/1622
• Given an empty 3-‐, 5-‐, and a full 8-‐liter water jug. • Use the three jugs to measure out exactly 4 liters of water with the “least number of steps”.
• Solution: How to enumerate all the possible schemes? (Not that obvious)
• Define State (a b c): the amounts of water in each of 3-‐,5-‐ and 8-‐liter jugs
• From the initial state, enumerate all possible states at each step
• Caveat: avoid repeated states
3 5 8
SVU CS5021/12/16
State (a b c): the amounts of water in each of jugs of 3-‐,5-‐ and 8-‐liter From the initial state, enumerate all possible states step by step (Avoid repeated states)
Solution: Enumerate All Possibilities
---------------------------------------------------------------- step 1: 0 0 8 <--- initial state 8 <--- newly measured amount of water ---------------------------------------------------------------- step 2: 3 0 5 0 5 3 3,5 ---------------------------------------------------------------- step 3: 3 5 0 0 3 5 3 2 3 2 ---------------------------------------------------------------- step 4: 3 3 2 0 2 6 6 ---------------------------------------------------------------- step 5: 2 0 6 ---------------------------------------------------------------- step 6: 2 5 1 1 ---------------------------------------------------------------- step 7: 3 4 1 4 ---------------------------------------------------------------- step 8: 0 4 4 ---------------------------------------------------------------- step 0: 3 1 4 3 4 1 ---------------------------------------------------------------- step 10:0 1 7 7 ----------------------------------------------------------------
Exercises:
• Consider the problem for jugs of different sizes. • Suppose the empty jugs are 2-‐liter and 6-‐liter, and the full jug is 8-‐liter? • What amounts can you measure out in this scenario?
• What about if the jugs are 4, 6, and 10 gallons respectively? • What about 5, 6, and 11?
24
SVU CS502
Correctness
• What makes a sorting algorithm correct? • In the output sequence, the elements are ordered non-‐decreasingly
• For any algorithm, we must prove that it always returns the desired output for all legal instances of the problem.
• It is easier to find a counterexample to show that an algorithm does NOT work.
1/12/1625
SVU CS502
Use loop invariants to prove the correctness of loops
• A loop invariant (LI) is a formal statement about the variables in your program which holds true throughout the loop (iteration).
• Example: The Green List is sorted all the time during the selection sort. • Proof by induction
• Initialization: LI is true prior to the 1st iteration • Maintenance: if LI is true before the jth iteration, it remains true after the jth iteration • Termination: When the loop terminates, LI ensure the correctness of the algorithm
1/12/1626
SVU CS502
5 2 4 6 1 3
2 5 4 6 1 3
2 4 5 6 1 3
2 4 5 6 1 3
1 2 4 5 6 3
1 2 3 4 5 6
k= 1 2 3 4 5 6
i= 1
i= 3
i= 0
i= 2
Assume LI is true prior to iteration k
Maintenance: LI will be true after iteration k+1
Initialization: Subarray A[1] is sorted.
LI: The green list is always sorted
SVU CS502
Best Time to Buy and Sell Stock (LeetCode 121)
Given an array A where A[i] is the stock price on day i. If only one transaction permitted (i.e., buy one and sell one share of the stock), ➔ Design an algorithm to find the maximum profit.
1/12/1628
SVU CS502
Best Time to Buy and Sell Stock (LeetCode 121)
• Solution (Brute-‐force) // code in Java
Int A[] = { -2,1,-3,4,-1,2,1,-5,4} ; Int n = A.length; // array length Int ans = 0 ; //for each transaction ending with sale on day ifor ( i=1; I < n; ++I ) {
//test lowest price before day ifor (j=0; j < i-1; ++j ) { ans = max ( ans, A[j]-A[i]) }
} Return 0;
• T(n)= (n-‐1)+(n-‐2)+ … + 1 = O(n2)
1/12/1629
SVU CS502
For each transaction ending at day x, can we get the lowest purchase price prior to day x in constant time? (Ideas: pre-‐compute this information first. ) Define array B, such that B[i]= min {A[k], k ≤ I } = the lowest price prior to day I B[i] = min { A[i],B[i-‐1] } Then the maximum profit of transaction ending on day i can be computed as C[i]= A[i]-‐B[i]=max{A[i]-‐A[k], k<=i} In constant time.
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16A 100 113 110 85 105 102 86 63 81 101 94 106 101 79 94 90 97
B 100 100 100 85 85 85 85 63 63 63 63 63 63 63 63 63 63C 0 13 10 0 20 17 1 0 18 38 31 43 38 16 31 27 34
1/12/1630
SVU CS5021/12/1631
Quiz: Best Time to Buy and Sell Stock Given an array A where A[i] is the stock price on day i. Only one transaction permitted (ie, buy one and sell one share of the stock). The following table is used to find the maximum profit in O(n) time. Which of the following statements is false:
0 1 2 3 4 5 6 7 8 8 10 11 12 13 14 15 16A 100 113 110 85 105 102 86 63 81 101 94 106 101 79 94 90 97
B 100 100 100 85 85 85 85 63 63 63 63 63 63 63 63 63 63C 0 13 10 0 20 17 1 0 18 38 31 43 38 16 31 27 34
(a) C[i] is the maximum possible profit of the transaction with sale on day i (b) B[i] is the lowest stock price from day 1 to day i (c) A[i]-‐B[i] is the maximum possible profit of the transaction with sale on day I (d) Array B does not have to be in sorted order.
Q#1. Shown in the following are operation sequences that corresponds to the selection sort or the insertion sort. M1: Input : 5 4 3 step 1: 4 5 3 step 2: 3 4 5 M2: Input : 5 4 3 step 1: 3 4 5 step 2: 3 4 5 M3: Input : 2 3 1 step 1: 2 3 1 step 2: 1 2 3 M4: Input : 2 3 1 step 1: 1 3 2 step 2: 1 2 3
Which statements is NOT correct? a) M1 is insertion sort, M2 is the selection sort b) M2 is insertion sort, M1 is the selection sort c) M3 is insertion sort, M4 is the selection sort b) M4 is insertion sort, M3 is the selection sort Ans: _____________________ (1pt)
Quiz #1, CS502, Design and Analysis of Algorithm, Summer 2016, Session#1
Group number: _____________________ Student ID:________________________ Name: _____________________________ Student ID:________________________ Name: _____________________________ Student ID:________________________ Name: _____________________________
Q#2. Best Time to Buy and Sell Stock (LeetCode 121) Given an array A where A[i] is the stock price on day i, as shown below: int A[] = { 9,5,7,11, 3 , 7, 12, 5}; int n = A.length; // array size, or total number of days for trading Only one transaction permitted (ie, buy one and sell one share of the stock). The following table is used to ]ind the maximum pro]it in O(n) time.
De]ine array B, such that B[i]= min {A[k], k ≤ i} = the lowest price on or prior to day i. Then the maximum pro]it of transaction ending on day i can be computed in constant time as C[i]= A[i]-‐B[i]=max{A[i]-‐A[k], k<=i}. Then the overall maximum pro]it by one transaction can be computed as max {C[i], 0<i<n } in linear time O(n).
array A : 9 5 7 11 3 7 12 5 array B : 9 5 X 5 3 3 3 3 array C : 0 0 Y Z 0 4 9 2
#9. What are the values of X, Y, and Z a) 5 2 6 b) 2 5 6 c) 6 5 2 d) 5 6 2 Ans: _____________________ (1pt)
33
Quiz #1, CS502, Design and Analysis of Algorithm, Summer 2016, Session# 2
Group number: _____________________ Student ID:________________________ Name: _____________________________ Student ID:________________________ Name: _____________________________ Student ID:________________________ Name: _____________________________
