Warm Up
How many arithmetic operations are required to multiply a 𝑛 ×𝑚 Matrix with a 𝑚 × 𝑝 Matrix?
(don’t overthink this)
1
𝑛
𝑚
𝑚
𝑝
×
CS4102 Algorithms Fall 2020
Warm Up
• 𝑚 multiplications and additions per element
• 𝑛 ⋅ 𝑝 elements to compute
• Total cost: 𝑚 ⋅ 𝑛 ⋅ 𝑝 2
𝑛
𝑚
𝑚
𝑝
𝑛
𝑝
× =
How many arithmetic operations are required to multiply a 𝑛 ×𝑚 Matrix with a 𝑚 × 𝑝 Matrix?
(don’t overthink this)
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the (optimal) solutions to smaller
ones
• Idea: 1. Identify recursive structure of the problem
• What is the “last thing” done?
3 𝑛 − 1 𝑛 − 2
𝑇𝑖𝑙𝑒 𝑛 = 𝑇𝑖𝑙𝑒 𝑛 − 1 + 𝑇𝑖𝑙𝑒(𝑛 − 2)
𝑇𝑖𝑙𝑒 0 = 𝑇𝑖𝑙𝑒 1 = 1
How to compute 𝑇𝑖𝑙𝑒(𝑛)?
4
Tile(n): if n < 2: return 1 return Tile(n-1)+Tile(n-2)
Problem?
Recursion Tree
5
Tile(5)
Tile(4) Tile(3)
Tile(3) Tile(2) Tile(2) Tile(1)
Tile(0) Tile(1) Tile(0) Tile(1) Tile(1) Tile(2)
Tile(0) Tile(1)
Many redundant calls!
Better way: Use Memory!
Run time: Ω(2𝑛)
Computing 𝑇𝑖𝑙𝑒(𝑛) with Memory - “Top Down”
6
Initialize Memory M Tile(n): if n < 2: return 1 if M[n] is filled: return M[n] M[n] = Tile(n-1)+Tile(n-2) return M[n]
1
1
2
3
5
8
13
M
0
1
2
3
4
5
6
Dynamic Programming
• Requires Optimal Substructure
– Solution to larger problem contains the (optimal) solutions to smaller ones
• Idea:
1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
7
Top-Down Dynamic Programming Template mem = {} def myDPalgo(problem): if mem[problem] not blank: return mem[problem] if baseCase(problem): solution = solve(problem) mem[problem] = solution return solution for subproblem of problem: subsolutions.append(myDPalgo(subproblem)) solution = OptimalSubstructure(subsolutions) mem[problem] = solution return solution
8
memoization
memoization
Dynamic Programming
• Requires Optimal Substructure
– Solution to larger problem contains the (optimal) solutions to smaller ones
• Idea:
1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems
• “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
9
Log Cutting
10
Given a log of length 𝑛 A list (of length 𝑛) of prices 𝑃 (𝑃[𝑖] is the price of a cut of size 𝑖) Find the best way to cut the log
1 5 8 9 10 17 17 20 24 30
10 9 8 7 6 5 4 3 2 1 Length:
Price:
Select a list of lengths ℓ1, … , ℓ𝑘 such that: ℓ𝑖 = 𝑛 to maximize 𝑃[ℓ𝑖] Brute Force: 𝑂(2
𝑛)
“Greedy” won’t work
11
Greedy: Lengths: 5, 1 Profit: 51
Better: Lengths: 2, 4 Profit: 54
1 18 24 36 50
5 4 3 2 1 Length:
Price: 50
6
• Greedy algorithms (next unit) build a solution by picking the best option “right now”
– Select the most profitable cut first
Greedy won’t work
• Greedy algorithms (next unit) build a solution by picking the best option “right now”
– Select the “most bang for your buck”
• (best price / length ratio)
12
1 18 24 36 50
5 4 3 2 1 Length:
Price: Greedy: Lengths: 5, 1 Profit: 51
Better: Lengths: 2, 4 Profit: 54
50
6
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the solutions to smaller ones
• Idea: 1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems • “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
13
1. Identify Recursive Structure
14
𝐶𝑢𝑡(𝑛) = value of best way to cut a log of length 𝑛
ℓ𝑘 𝐶𝑢𝑡(𝑛 − ℓ𝑘)
𝐶𝑢𝑡 𝑛 = max
𝐶𝑢𝑡 𝑛 − 1 + 𝑃 1 𝐶𝑢𝑡 𝑛 − 2 + 𝑃 2 … 𝐶𝑢𝑡 0 + 𝑃[𝑛]
Last Cut best way to cut a log of length 𝒏 − ℓ𝒌
𝑃 𝑖 = value of a cut of length 𝑖
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the solutions to smaller ones
• Idea: 1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems • “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
15
Top-Down Log Cutting mem = {} def best_cut(length): if mem[length] not blank: return mem[length] if length == 0: solution = 0 mem[length] = solution return solution for shorter ≤ length: subsolutions[shorter] = best_cut(shorter) solution = max{subsolutions[shorter] + price[shorter]} mem[length] = solution return solution
16
memoization
Base Case
Substructure
memoization
3. Bottom-up: solve subproblems in order
17
10 9 8 7 6 5 4 3 2 1 Length:
𝐶𝑢𝑡(𝑖): 0
0
Solve Smallest subproblem first
𝐶𝑢𝑡 0 = 0
0
3. Bottom-up: solve subproblems in order
18
10 9 8 7 6 5 4 3 2 1 Length:
𝐶𝑢𝑡(𝑖): 0
0
Solve Smallest subproblem first
1
𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 1 = 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 0 + 𝑃[1]
3. Bottom-up: solve subproblems in order
19
10 9 8 7 6 5 4 3 2 1 Length:
𝐶𝑢𝑡(𝑖): 0
0
Solve Smallest subproblem first
2
𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 2 = max 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 1 + 𝑃 1 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 0 + 𝑃[2]
3. Bottom-up: solve subproblems in order
20
10 9 8 7 6 5 4 3 2 1 Length:
𝐶𝑢𝑡(𝑖): 0
0
Solve Smallest subproblem first
3
𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 3 = max 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 2 + 𝑃 1 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 1 + 𝑃 2 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 0 + 𝑃[3]
3. Bottom-up: solve subproblems in order
21
10 9 8 7 6 5 4 3 2 1 Length:
𝐶𝑢𝑡(𝑖): 0
0
Solve Smallest subproblem first
𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 4 = max
𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 3 + 𝑃[1] 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 2 + 𝑃 2 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 1 + 𝑃 3 𝑏𝑒𝑠𝑡_𝑐𝑢𝑡 0 + 𝑃[4]
4
Bottom-Up Log Cutting Pseudocode
22
mem = {} def best_cut(length): mem[0] = 0 for sub=1 to length: // smallest subproblem first best = 0 for last-cut = 1 to sub: //last cut best = max(best, mem[i-j] + price[j]) mem[i] = best return C[n]
Run Time: 𝑂(𝑛2)
How to find the cuts?
• This procedure told us the profit, but not the cuts themselves
• Idea: remember the choice that you made, then backtrack
23
Top-Down Log Cutting mem = {} choices = {} def best_cut(length): if mem[length] not blank: return mem[length] if length == 0: solution = 0 mem[length] = solution return solution for last_cut ≤ length: subsolutions[length- last_cut] = best_cut(length- last_cut) solution = max{subsolutions[length- last_cut] + price[last_cut]} mem[length] = solution choices[length] = best last_cut return solution
24
Gives the size of the last cut
Reconstruct the Cuts
25
1 1 2 4 3 4 1 2 4 3
10 9 8 7 6 5 4 3 2 1 Length:
Choices: 0
0
• Backtrack through the choices
7 6 2 1
Example to demo Choices[] only.
For a log of length 10, The best choice for the Last cut’s length is 3
Backtracking Pseudocode
i = n
while i > 0:
print choices[i]
i = i – choices[i]
26
Matrix Chaining
27
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× × 𝑟4
𝑐4
• Given a sequence of Matrices (𝑀1, … ,𝑀𝑛), what is the most efficient way to multiply them?
𝑀2 𝑀4
Order Matters!
• 𝑀1 ×𝑀2 ×𝑀3
– uses 𝑐1 ⋅ 𝑟1 ⋅ 𝑐2 + c2 ⋅ 𝑟1 ⋅ 𝑐3 operations
28
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× 𝑀2
𝑟1
𝑐2
𝑐1 = 𝑟2 𝑐2 = 𝑟3
Order Matters!
• 𝑀1 × (𝑀2 ×𝑀3)
– uses c1 ⋅ r1 ⋅ 𝑐3 + (c2 ⋅ 𝑟2 ⋅ 𝑐3) operations
29
𝑟2
𝑐3
𝑐1 = 𝑟2 𝑐2 = 𝑟3
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× 𝑀2
Order Matters!
• 𝑀1 ×𝑀2 ×𝑀3
– uses 𝑐1 ⋅ 𝑟1 ⋅ 𝑐2 + c2 ⋅ 𝑟1 ⋅ 𝑐3 operations
– 10 ⋅ 7 ⋅ 20 + 20 ⋅ 7 ⋅ 8 = 2520
• 𝑀1 × (𝑀2 ×𝑀3)
– uses 𝑐1 ⋅ 𝑟1 ⋅ 𝑐3 + (c2 ⋅ 𝑟2 ⋅ 𝑐3) operations
– 10 ⋅ 7 ⋅ 8 + 20 ⋅ 10 ⋅ 8 = 2160
30
𝑐1 = 𝑟2 𝑐2 = 𝑟3
𝑐1 = 10 𝑐2 = 20 𝑐3 = 8 𝑟1 = 7 𝑟2 = 10 𝑟3 = 20
𝑀1 = 7 × 10 𝑀2 = 10 × 20 𝑀3 = 20 × 8
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the solutions to smaller ones
• Idea: 1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems • “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
31
1. Identify the Recursive Structure of the Problem
32
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× × 𝑟4
𝑐4
𝑀2 𝑀4
𝐵𝑒𝑠𝑡 1, 𝑛 = cheapest way to multiply together 𝑀1 through 𝑀𝑛
1. Identify the Recursive Structure of the Problem
33
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× × 𝑟4
𝑐4
𝑀2 𝑀4
𝐵𝑒𝑠𝑡 1, 𝑛 = cheapest way to multiply together 𝑀1 through 𝑀𝑛
𝐵𝑒𝑠𝑡 1,4 = min
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 2,4 + 𝑟1𝑟2𝑐4
𝑐4
𝑟2
1. Identify the Recursive Structure of the Problem
34
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× × 𝑟4
𝑐4
𝑀2 𝑀4
𝐵𝑒𝑠𝑡 1, 𝑛 = cheapest way to multiply together 𝑀1 through 𝑀𝑛
𝐵𝑒𝑠𝑡 1,4 = min
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 2,4 + 𝑟1𝑟2𝑐4
𝑐4
𝑟3
𝑐2
𝑟1
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 4 + 𝑟1𝑟3𝑐4
1. Identify the Recursive Structure of the Problem
35
𝑀1 𝑟1
𝑐1
𝑟2 ×
𝑐2
𝑀3 𝑟3
𝑐3
× × 𝑟4
𝑐4
𝑀2 𝑀4
𝐵𝑒𝑠𝑡 1, 𝑛 = cheapest way to multiply together 𝑀1 through 𝑀𝑛
𝐵𝑒𝑠𝑡 1,4 = min
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 2,4 + 𝑟1𝑟2𝑐4
𝑐3
𝑟1
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 4 + 𝑟1𝑟3𝑐4
𝐵𝑒𝑠𝑡 1,3 + 𝑟1𝑟4𝑐4
1. Identify the Recursive Structure of the Problem
• In general:
36
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = cheapest way to multiply together 𝑀𝑖 through 𝑀𝑗
𝐵𝑒𝑠𝑡 1, 𝑛 = min
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 2,4 + 𝑟1𝑟2𝑐4
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 𝑛 + 𝑟1𝑟3𝑐𝑛
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 𝑛 + 𝑟1𝑟4𝑐𝑛
𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 𝑛 + 𝑟1𝑟5𝑐𝑛
…
𝐵𝑒𝑠𝑡 1, 𝑛 − 1 + 𝑟1𝑟𝑛𝑐𝑛
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the solutions to smaller ones
• Idea: 1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems • “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
37
2. Save Subsolutions in Memory
• In general:
38
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = cheapest way to multiply together 𝑀𝑖 through 𝑀𝑗
𝐵𝑒𝑠𝑡 1, 𝑛 = min
𝐵𝑒𝑠𝑡 2, 𝑛 + 𝑟1𝑟2𝑐𝑛
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 𝑛 + 𝑟1𝑟3𝑐𝑛
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 𝑛 + 𝑟1𝑟4𝑐𝑛
𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 𝑛 + 𝑟1𝑟5𝑐𝑛 …
𝐵𝑒𝑠𝑡 1, 𝑛 − 1 + 𝑟1𝑟𝑛𝑐𝑛
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
Save to M[i,j]
Read from M[i,j] if present
Dynamic Programming
• Requires Optimal Substructure – Solution to larger problem contains the solutions to smaller ones
• Idea: 1. Identify the recursive structure of the problem
• What is the “last thing” done?
2. Save the solution to each subproblem in memory
3. Select a good order for solving subproblems • “Top Down”: Solve each recursively
• “Bottom Up”: Iteratively solve smallest to largest
39
3. Select a good order for solving subproblems
• In general:
40
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = cheapest way to multiply together 𝑀𝑖 through 𝑀𝑗
𝐵𝑒𝑠𝑡 1, 𝑛 = min
𝐵𝑒𝑠𝑡 2, 𝑛 + 𝑟1𝑟2𝑐𝑛
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 𝑛 + 𝑟1𝑟3𝑐𝑛
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 𝑛 + 𝑟1𝑟4𝑐𝑛
𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 𝑛 + 𝑟1𝑟5𝑐𝑛 …
𝐵𝑒𝑠𝑡 1, 𝑛 − 1 + 𝑟1𝑟𝑛𝑐𝑛
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
Save to M[i][j]
Read from M if present
Top-Down Matrix Chain mem = {} def best(i, j): if mem[i][j] not blank: return mem[i][j] if i == j: solution = 0 mem[i][j] = solution return solution for i ≤ last_mult < j:
subsolutions[last_mult] = best(i, last_mult) + best(last_mult, j) + 𝑟𝑖𝑐𝑙𝑎𝑠𝑡𝑐𝑗 solution = min{subsolutions[last_mult]} mem[i][j] = solution return solution
41
3. Select a good order for solving subproblems
42
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 𝑗 =
0
0
0
0
0
0
1 2 3 4 5 6
1
2
3
4
5
6
43
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
𝐵𝑒𝑠𝑡 1,2 = min 𝐵𝑒𝑠𝑡 1,1 + 𝐵𝑒𝑠𝑡 2, 2 + 𝑟1𝑟2𝑐2
0 15750
0
0
0
0
0
1 2 3 4 5 6
1
2
3
4
5
6
3. Select a good order for solving subproblems
𝑗 =
44
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
𝐵𝑒𝑠𝑡 2,3 = min 𝐵𝑒𝑠𝑡 2,2 + 𝐵𝑒𝑠𝑡 3, 3 + 𝑟2𝑟3𝑐3
0 15750
0 2625
0
0
0
0
1 2 3 4 5 6
1
2
3
4
5
6
3. Select a good order for solving subproblems
𝑗 =
45
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 0 15750
0 2625
0 750
0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
3. Select a good order for solving subproblems
𝑗 =
46
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 0 15750
0 2625
0 750
0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
𝐵𝑒𝑠𝑡 1,3 = min 𝐵𝑒𝑠𝑡 1,1 + 𝐵𝑒𝑠𝑡 2, 3 + 𝑟1𝑟2𝑐3
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 3 + 𝑟1𝑟3𝑐3
𝑟1𝑟2𝑐3 = 30 ⋅ 35 ⋅ 5 = 5250 𝑟1𝑟3𝑐3 = 30 ⋅ 15 ⋅ 5 = 2250
0
0
2625
15750
3. Select a good order for solving subproblems
7875
𝑗 =
47
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 0 15750 7875
0 2625
0 750
0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
3. Select a good order for solving subproblems
To find 𝐵𝑒𝑠𝑡(𝑖, 𝑗): Need all preceding terms of row 𝑖 and column 𝑗
Conclusion: solve in order of diagonal
𝑗 =
Matrix Chaining
48
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 0 15750 7875 9375 11875
0 2625 4375 7125 10500
0 750 2500 5375
3500 0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
𝐵𝑒𝑠𝑡 1,6 = min
𝐵𝑒𝑠𝑡 1,1 + 𝐵𝑒𝑠𝑡 2, 6 + 𝑟1𝑟2𝑐6
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 6 + 𝑟1𝑟3𝑐6
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 6 + 𝑟1𝑟4𝑐6 𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 6 + 𝑟1𝑟5𝑐6 𝐵𝑒𝑠𝑡 1,5 + 𝐵𝑒𝑠𝑡 6, 6 + 𝑟1𝑟6𝑐6
15125
𝑗 =
Run Time
1. Initialize 𝐵𝑒𝑠𝑡[𝑖, 𝑖] to be all 0s
2. Starting at the main diagonal, working to the upper-right, fill in each cell using:
1. 𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0
2. 𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
49
Θ(𝑛2) cells in the Array
Θ(𝑛) options for each cell
Θ(𝑛3) overall run time
Each “call” to Best() is a O(1) memory lookup
Backtrack to find the best order
50 50
“remember” which choice of 𝑘 was the minimum at each cell
0 15750 7875 9375 11875
0 2625 4375 7125 10500
0 750 2500 5375
3500 0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
𝐵𝑒𝑠𝑡 1,6 = min
𝐵𝑒𝑠𝑡 1,1 + 𝐵𝑒𝑠𝑡 2, 6 + 𝑟1𝑟2𝑐6
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 6 + 𝑟1𝑟3𝑐6
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 6 + 𝑟1𝑟4𝑐6 𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 6 + 𝑟1𝑟5𝑐6 𝐵𝑒𝑠𝑡 1,5 + 𝐵𝑒𝑠𝑡 6, 6 + 𝑟1𝑟6𝑐6
15125
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 3 1
5
𝑗 =
Matrix Chaining
51
30
35
× 𝑀1 35
15
× 𝑀2 15
5
× 𝑀3 5
10
× 𝑀4
10
20
× 𝑀5
20
25
𝑀6
𝐵𝑒𝑠𝑡 𝑖, 𝑗 = min𝑗−1
𝑘=𝑖𝐵𝑒𝑠𝑡 𝑖, 𝑘 + 𝐵𝑒𝑠𝑡 𝑘 + 1, 𝑗 + 𝑟𝑖𝑟𝑘+1𝑐𝑗
𝐵𝑒𝑠𝑡 𝑖, 𝑖 = 0 0 15750 7875 9375 11875
0 2625 4375 7125 10500
0 750 2500 5375
3500 0 1000
5000 0
0
1 2 3 4 5 6
1
2
3
4
5
6
𝐵𝑒𝑠𝑡 1,6 = min
𝐵𝑒𝑠𝑡 1,1 + 𝐵𝑒𝑠𝑡 2, 6 + 𝑟1𝑟2𝑐6
𝐵𝑒𝑠𝑡 1,2 + 𝐵𝑒𝑠𝑡 3, 6 + 𝑟1𝑟3𝑐6
𝐵𝑒𝑠𝑡 1,3 + 𝐵𝑒𝑠𝑡 4, 6 + 𝑟1𝑟4𝑐6 𝐵𝑒𝑠𝑡 1,4 + 𝐵𝑒𝑠𝑡 5, 6 + 𝑟1𝑟5𝑐6 𝐵𝑒𝑠𝑡 1,5 + 𝐵𝑒𝑠𝑡 6, 6 + 𝑟1𝑟6𝑐6
15125
𝑗 =
3 1
5
Storing and Recovering Optimal Solution
• Maintain table choice[i,j] in addition to mem table – choice[i,j] = k means the best “split” was right after Mk – Work backwards from value for whole problem, choice[1,n]
– Note: choice[i,i+1] = i because there are just 2 matrices
• From our example: – choice[1,6] = 3. So [M1 M2 M3] [M4 M5 M6]
– We then need choice[1,3] = 1. So [(M1) (M2 M3)]
– Also need choice[4,6] = 5. So [(M4 M5) M6]
– Overall: [(M1) (M2 M3)] [(M4 M5) M6]
52