CPSC 335

Greedy methods and Backtracking

LECTURE 16

Jon Rokne Computer Science University of Calgary

Canada

Modified from Marina’s lectures

• Optimal structure: optimal solution to problem consists of optimal solutions to subproblems.

• Overlapping subproblems: few subproblems in total, many recurring instances of each.

• Solve in “near linear” time through recursion, using a table to show state of problem at any given moment of time.

• Reduces computation time from exponential to linear in some cases.

Dynamic programming

Greedy algorithm always makes the choice that is the locally optimal (best) at the moment. It can reduce complex problem to linearly solvable time, and can be much easier to code than dynamic programming solutions

3

Greedy methods

Matrix Chain Multiplication - Review n  Recall how to multiply matrices. n  Given two matrices

n  The product is n  Time is

A

B

C

X =

)( ed×A )( fe×B

d

e

e

f

∑−

=

•=1

0

],[],[],[e

kjkkiji BAC

d

f

)(defO

n  What about multiplying multiple matrices?

n  We know matrix multiplication is associative. Can we use this to help minimize calculations?

n  Sure, consider

n  If we multiply we do 4000 operations

n  But instead if we try we do 1575 operations

Matrix Chaining contd.

54321 AAAAAA ⋅⋅⋅⋅=

)35( ×A )1003( ×B )5100( ×C

( ) CBA ⋅⋅

)( CBA ⋅⋅

How do we parenthesize

n  Brute force - try all ways to parenthesize

n  Find out which has smallest number of operations by doing multiplication, then pick the best one.

n  But how many ways can we parenthesize these? n  Equivalent to the number of ways to make a binary

tree with n nodes. This is see Catalan numbers.

nAAAAA ⋅⋅⋅⋅= …321

)2( nΩ

Be greedy

n  We just learned that a greedy algorithm can sometimes work, let’s try.

n  Why not try starting with the product with the most operations.

n  Consider

n  If we do our greedy method we would compute

n  This is 2000 operations.

n  Try instead, which takes 1000 operations.

)105()510()105()510( ×××× DCBA

( ) ( )DCBA ⋅⋅⋅

( )( )DCBA ⋅⋅

Another Greedy way n  Select the product with the fewest operations.

n  Consider

n  Here the new greedy method would compute

n  This is 109989+9900+108900=228789 operations

n  Try

n  Here we only need 9999+89991+89100=189090

)99100()1009()911()11101( ×××× DCBA

( )( )DCBA ⋅⋅⋅

( ) ( )DCBA ⋅⋅⋅

n  Shortest-path problem and multiple instance knapsack problems can be solved using greedy approach.

n  Activity selection is another example.

9

Greedy methods

Problem: Stampede midway problem. » Buy a wristband that lets you onto any ride. » Lots of rides, each starting and ending at different times. » Your goal: ride as many rides as possible. • Alternative goal that we don’t solve here: maximize time spent on rides

10

Greedy methods

Let A be an optimal solution of S and let k be the minimum activity in A (i.e., the one with the earliest finish time). Then A - {k} is an optimal solution to S’ = {i ∈ S: si ≥ fk}. » In words: once activity #1 is selected, the problem reduces to finding an optimal solution for activity selection over activities in S compatible with #1.

11

Greedy algorithm

The algorithm is following: » Sort the activities by finish time » Schedule the first activity » Then schedule the next activity in sorted list which starts after previous activity finishes » Repeat until no more activities

Idea: Always pick the shortest ride available at the time Greedy-choice Property. » A globally optimal solution can be arrived at by making a locally optimal (greedy) choice.

12

Greedy Algorithm

n  Backtracking can reduce a NP compete problem to linear problem by only going through selected branches of the global solution.

13

Backtracking

n  Graph-coloring problem n  8 queens problem n  4 knight problem n  Perfect hash function problems are examples of backtracking method

14

Backtracking

15

Backtracking

n  The goal is to color vertices in a graph G={V,E} so that no 2 adjacent vertices have the same color. Partial 3-coloring problem means only 3 colors are considered.

n  Direct approach builds the tree of ALL possibilities in exponential time.

16

Backtracking

n  Partial 3-coloring (3 colors) is solved by the following method: n  Color first vertex with 1st color, color next vertex with the next color,

check if those two vertices are adjacent, if not - coloring is legal, proceed to next vertex, if yes and color is the same – coloring is illegal, try next color for second vertex. If all colors tried and all colorings are illegal, backtrack, try next color for previous vertex etc.

n  Note: sometimes solution is impossible. n  Exponential O(3^n) complexity is reduced to O(n) on average.

n  Finally, job scheduling problem can be solved by converting the algebraic relationship xi-xj>=c to directed graph with vertices xi, xj, direction from xj to xi, and edge cost (or time required to complete job xj) is c.

n  The problem of finding time (minimum) when the last activity can commence (i.e. all preceding activities has been completed) is then converted to longest path problem on the graph.

17

Job scheduling problem

n  What are greedy methods and how multi-instance knapsack problem can be solved through them

n  How backtracking works for graph coloring

n  How job scheduling problem can be solved through directed graphs

18

Review questions

Thank You