HKOI Training 2009HKOI Training 2009Kelly ChoiKelly Choi27 June 200927 June 2009

Acknowledgements:Acknowledgements:References and slides extracted fromReferences and slides extracted from1.1. [Advanced] Dynamic Programming, 24-04-2004, by Chi-Man Liu[Advanced] Dynamic Programming, 24-04-2004, by Chi-Man Liu2.2. [Intermediate] Dynamic Programming, 21-08-2004, by Ng Tung[Intermediate] Dynamic Programming, 21-08-2004, by Ng Tung

PrerequisitesPrerequisites

Concepts in RecurrenceConcepts in RecurrenceBasic RecursionBasic RecursionFunctionsFunctionsDivide-and-conquerDivide-and-conquer

IntroductionIntroduction

Components of DPComponents of DPA recurrence formula (with state & state A recurrence formula (with state & state

value)value)Memo(r)izationMemo(r)ization

Characteristics of DP ProblemsCharacteristics of DP ProblemsOverlapping SubproblemsOverlapping SubproblemsOptimal SubstructureOptimal SubstructureMemoryless PropertyMemoryless Property

Grid Path CountingGrid Path Counting In a N*M grid, we want to move from the top left cell In a N*M grid, we want to move from the top left cell

to the bottom right cellto the bottom right cell You may only move You may only move downdown or or rightright Some cells may be blockedSome cells may be blocked Example of one path:Example of one path:

Naïve Algorithm: DFSNaïve Algorithm: DFS DFS(x,y){DFS(x,y){

if (x <= N) and (y <= M) {if (x <= N) and (y <= M) {if (x = N) and (y = M)if (x = N) and (y = M)

return 1 {base case}return 1 {base case}else if (x,y) is not blockedelse if (x,y) is not blocked

return DFS(x+1,y) + DFS(x,y+1)return DFS(x+1,y) + DFS(x,y+1)elseelse

return 0 {blocked cell}return 0 {blocked cell}}}

}}

Time ComplexityTime Complexity

Exponential time complexity:Exponential time complexity:Each time the procedure branches into two Each time the procedure branches into two

pathspathsAlternativelyAlternatively

The base case is reached as many times as The base case is reached as many times as the number of paths, so the time complexity is the number of paths, so the time complexity is ΩΩ(number of paths(number of paths))

Have we done anything redundant?Have we done anything redundant?

Slow...Slow...

DFS(1,2)

DFS(1,3) DFS(2,2)

DFS(1,4) DFS(2,3) DFS(3,2)DFS(2,3)

DFS(1,1)

DFS(2,1)

DFS(2,2) DFS(3,1)

DFS(3,2)DFS(2,3)

Overlapping Subproblems Overlapping Subproblems

Note that DFS(Note that DFS(22,2) ,2) iis called twis called twicce, e, DFS(2,3) three times, etc.DFS(2,3) three times, etc.

But the But the wwork perork perfformeormedd by these DFS(2,2) by these DFS(2,2) ccalls are alls are unaffected by what called themunaffected by what called them, , aand thus are nd thus are reduredunndantdant

Memorize the values Memorize the values tthheese calls se calls rreturn, eturn, and avoid and avoid tthe reduhe redunndant workdant work

Memo(r)izationMemo(r)ization

Compute and Compute and storestore the value of DFS(x,y) the value of DFS(x,y) in the first time we called it. in the first time we called it. Afterwards, retrieve the value of DFS(x,y) Afterwards, retrieve the value of DFS(x,y)

from the table directly, without calling from the table directly, without calling DFS(x+1,y) and DFS(x,y+1) againDFS(x+1,y) and DFS(x,y+1) again

This is called recursion with This is called recursion with memo(r)ization.memo(r)ization.

Time complexity: O(NM)Time complexity: O(NM)

Example : Grid Path CountingExample : Grid Path Counting DFS(x,y){DFS(x,y){

If (x <= N) and (y <= M){If (x <= N) and (y <= M){If (Memory[x,y] = -1) {If (Memory[x,y] = -1) {

If (x = N) and (y = M)If (x = N) and (y = M)Memory[x,y]Memory[x,y] ← 1 ← 1

else if (x,y) is not blockedelse if (x,y) is not blockedMemory[x,y]Memory[x,y] ← DFS(x+1,y) + DFS(x,y+1) ← DFS(x+1,y) + DFS(x,y+1)elseelse

Memory[x,y]Memory[x,y] ← 0 ← 0}}return Memory[x,y];return Memory[x,y];

}}}}

Bottom-up approachBottom-up approach

Iterative, Without the use of function callsIterative, Without the use of function calls Consider the arrays Grid[x,y] and Memory[x,y]:Consider the arrays Grid[x,y] and Memory[x,y]:

66 33 33 11 00

33 00 22 11 11

33 22 11 00 11

11 11 11 11 11

Treat DFS(x,y) not as a function, but as an arrayTreat DFS(x,y) not as a function, but as an array Evaluate the values for DFS[x,y] row-by-row, column-by-Evaluate the values for DFS[x,y] row-by-row, column-by-

columncolumn

Example : Grid Path CountingExample : Grid Path Counting DFS[N,M] ← 1DFS[N,M] ← 1

For x ← 1 to N DoFor x ← 1 to N Do If (x,M) is blocked Then DFS[x,M] ← 0 Else DFS[x,M] ← 1If (x,M) is blocked Then DFS[x,M] ← 0 Else DFS[x,M] ← 1For y ← 1 to M DoFor y ← 1 to M Do If (N,y) is blocked Then DFS[N,y] ← 0 Else DFS[N,y] ← 1If (N,y) is blocked Then DFS[N,y] ← 0 Else DFS[N,y] ← 1

For x ← N-1 downto 1 DoFor x ← N-1 downto 1 DoFor y ← M-1 downto 1 DoFor y ← M-1 downto 1 Do

If (x,y) is blocked ThenIf (x,y) is blocked ThenDFS[x,y] ← 0DFS[x,y] ← 0

ElseElseDFS[x,y] ← DFS[x+1,y] + DFS[x,y+1]DFS[x,y] ← DFS[x+1,y] + DFS[x,y+1]

Order of ComputationOrder of Computation Top-down approach: Recursive function calls Top-down approach: Recursive function calls

with memo(r)izationwith memo(r)ization Easy to implementEasy to implement Avoid calculating values for impossible statesAvoid calculating values for impossible states Further optimization possibleFurther optimization possible

Bottom-up approachBottom-up approach Easy to code once the order is identifiedEasy to code once the order is identified Avoid function callsAvoid function calls Usually preferedUsually prefered But we nBut we need to see the dependence of states eed to see the dependence of states onon

other states to decide the orderother states to decide the order

Components of DPComponents of DP

It is very important to be able to describe a It is very important to be able to describe a DP algorithmDP algorithm

Two essential components of DPTwo essential components of DPState(State( 狀態狀態 ) and the state value: ) and the state value:

State – a description of the subproblemState – a description of the subproblemState value – the value we need according to the State value – the value we need according to the

subproblem, usually optimal in some sense, but subproblem, usually optimal in some sense, but can be defined flexibly.can be defined flexibly.

Recurrence – A rule describing the Recurrence – A rule describing the relationship between states, with base casesrelationship between states, with base cases

Grid Path CountingGrid Path Counting

In the above problem, the state is the In the above problem, the state is the position – (x,y)position – (x,y)

The state value is defined as The state value is defined as N(x,y) = Number of paths from that position to the N(x,y) = Number of paths from that position to the

destinationdestination

Recurrence FormulaRecurrence Formula

N(x,y) =N(x,y) =

1, x=N and y = M1, x=N and y = M

0, Grid[x,y] is impassable0, Grid[x,y] is impassable

N(x+1,y) + N(x,y+1), otherwiseN(x+1,y) + N(x,y+1), otherwise

Defining StatesDefining States

Defining a good state is the key to solving a DP Defining a good state is the key to solving a DP problemproblem Affects the recurrence formulaAffects the recurrence formula Affects the dimension of the problem and thus the Affects the dimension of the problem and thus the

time complexitytime complexity

Should include all information relevant to the Should include all information relevant to the computation of the value (except those static to computation of the value (except those static to the test cases). Recall what a “the test cases). Recall what a “functionfunction” means ” means in the mathematical sense.in the mathematical sense.

Defining StatesDefining States

For optimization problems : usually the For optimization problems : usually the optimal value subject to a set of optimal value subject to a set of constraints (i.e. the states). constraints (i.e. the states). Sometimes from varying the constraints from Sometimes from varying the constraints from

the problemthe probleme.g. cost(number of students, resource e.g. cost(number of students, resource

allocated)allocated)

Features of DP ProblemsFeatures of DP Problems

Overlapping Overlapping sub-problemssub-problems ( ( 重疊子問題重疊子問題 ))Optimal SubstructureOptimal Substructure ( ( 最優子結構最優子結構 ))Memoryless Property (Memoryless Property ( 無後效性無後效性 ))

The future depends only on the present, but The future depends only on the present, but not the past. not the past.

i.e. Decision leading to the subproblem does i.e. Decision leading to the subproblem does not affect how we solve the subproblem.not affect how we solve the subproblem.

Memo(r)ization VS DPMemo(r)ization VS DP

Memo(r)ization – speeding up Memo(r)ization – speeding up computations by storing previously computations by storing previously computed results in memorycomputed results in memory

Dynamic Programming – process of Dynamic Programming – process of setting up and evaluating a recurrence setting up and evaluating a recurrence relation efficiently by employing relation efficiently by employing memo(r)izationmemo(r)ization

Triangle (IOI ’94)Triangle (IOI ’94)

Given a triangle with Given a triangle with NN levels levels like the one on the left, find a like the one on the left, find a path with maximum sum from path with maximum sum from the top to the bottomthe top to the bottom

Only the sum is requiredOnly the sum is required

7

6 9

8 1 4

2 4 5 3

Triangle Triangle : A: Analysisnalysis

Exhaustion? Exhaustion? How many paths are there in total?How many paths are there in total?

Greedy?Greedy? It doesn’t work. Why?It doesn’t work. Why?

Graph problem?Graph problem?Possible, but not simple enoughPossible, but not simple enoughFail to make use of the special shape of the Fail to make use of the special shape of the

graphgraph

Triangle: Defining the StateTriangle: Defining the State

We need to find the maximum sum on a We need to find the maximum sum on a path from (1,1) to the bottompath from (1,1) to the bottom

We can attempt to define the state by the We can attempt to define the state by the cell (cell (ii,,jj) and state value F[) and state value F[ii][][jj] to be the ] to be the maximum sum from (maximum sum from (ii,,jj) to be bottom) to be bottom

TriangleTriangle: F: Formulationormulation

Let A[Let A[ii][][jj] denote the number in the ] denote the number in the ii-th row -th row and and jj-th column, (-th column, (ii, , jj))

Let F[Let F[ii][][jj] denote the ] denote the maximum maximum sum of the sum of the numbers on a path numbers on a path from the bottomfrom the bottom to ( to (ii, , jj))

Answer = F[Answer = F[11][1]][1]The problem exhibits optimal substructure The problem exhibits optimal substructure

herehere

TriangleTriangle: F: Formulationormulation

Base caseBase casess: F[: F[NN][][ii] = A[] = A[NN][][ii]] for 1≤ for 1≤ i i ≤≤NNRecurrenceRecurrence: (: (ii << NN, 1 , 1 ≤≤ jj ≤≤ ii) )

F[F[ii][][jj] = ] = max{max{F[F[ii+1+1][][jj]],F[,F[ii+1][+1][jj+1]}+1]}+A[+A[ii][][jj]]

7

6 9

8 1 4

TriangleTriangle: Computation Order: Computation Order

F[F[ii][*] depends on F[][*] depends on F[ii++1][*]1][*]Compute F row by row, from Compute F row by row, from bottombottom to to toptop

TriangleTriangle: Algorithm: Algorithm

Algorithm:Algorithm: ffor or ii 11 to to NN do do

F[F[NN][][ii] ] A[ A[NN][][ii] ;] ;

for for ii NN-1-1 downdownto to 11 do do

for for jj 11 to to ii do do

F[F[ii][][jj] ] max{F[ max{F[ii+1+1][][jj],F[],F[ii+1+1][][jj++1]}1]}

+ A[+ A[ii][][jj]]

answer answer F[1][1]F[1][1]

TriangleTriangle: Complexity: Complexity

Number of array entriesNumber of array entries: : NN((NN++1)/21)/2Time for computing one entry: O(1)Time for computing one entry: O(1)Thus the total time complexity is O(Thus the total time complexity is O(NN22))Memory complexity: O(Memory complexity: O(NN22))

Triangle: BacktrackingTriangle: Backtracking

What if we are asked to output the path?What if we are asked to output the path?We can store the optimal decision, i.e. left We can store the optimal decision, i.e. left

or right to go for each cellor right to go for each cell We then travel from (1,1) down to the We then travel from (1,1) down to the

bottom to obtain the pathbottom to obtain the pathThis is called Backtracking, “back” in the This is called Backtracking, “back” in the

sense that we go in the reversed order of sense that we go in the reversed order of which we obtained the answer. which we obtained the answer.

TriangleTriangle

If you refer to past training notes, there is If you refer to past training notes, there is another definition of the state value F[another definition of the state value F[ii][][jj] – ] – the maximum sum of a path from (1,1) to the maximum sum of a path from (1,1) to ((ii,,jj))

Both algorithm gives the same time Both algorithm gives the same time complexity. But the second definition (from complexity. But the second definition (from the top to bottom) probably gives you the top to bottom) probably gives you some insights for some insights for Triangle IITriangle II on the HKOI on the HKOI Online Judge. Think about it. Online Judge. Think about it.

FishingFishing

There are There are NN fish ponds and you are going fish ponds and you are going to spend to spend MM minutes on fishing minutes on fishing

Given the Given the time-rewardtime-reward relationship of each relationship of each pond, determine the time you should pond, determine the time you should spend at each pond in order to get the spend at each pond in order to get the biggest rewardbiggest reward

time/pondtime/pond 11 22 33

1 minute1 minute 0 fish0 fish 2 fish2 fish 1 fish1 fish

2 minutes2 minutes 3 fish3 fish 2 fish2 fish 2 fish2 fish

3 minutes3 minutes 3 fish3 fish 4 fish4 fish 4 fish4 fish

Fishing (example)Fishing (example)

For example, if N=3, M=3 and the For example, if N=3, M=3 and the relationships are given in the previous relationships are given in the previous slide, then the optimal schedule isslide, then the optimal schedule isPond 1: 2 minutesPond 1: 2 minutesPond 2: 1 minutePond 2: 1 minutePond 3: 0 minutePond 3: 0 minuteReward: 5 fishReward: 5 fish

Fishing (analysis)Fishing (analysis)

You can think of yourself visiting ponds 1, 2, 3, You can think of yourself visiting ponds 1, 2, 3, …, …, NN in order in orderWhy?Why?

Suppose in an optimal schedule you spend Suppose in an optimal schedule you spend KK minutes on fishing at pond 1minutes on fishing at pond 1

So you have So you have MM--KK minutes to spend at the minutes to spend at the remaining remaining NN-1 ponds-1 pondsThe problem is reducedThe problem is reduced

But how can I know what is But how can I know what is KK??You don’t know, so try all possible values!You don’t know, so try all possible values!

Fishing (formulation)Fishing (formulation)

Let F[Let F[ii][][jj] be the maximum reward you can ] be the maximum reward you can get by spending get by spending jj minutes at the first minutes at the first ii pondsponds

Base cases: 0 ≤ Base cases: 0 ≤ ii, , jj ≤ ≤ NN

F[F[ii][0] = 0][0] = 0

F[0][F[0][jj] = 0] = 0Progress: 1 ≤ Progress: 1 ≤ ii ≤ ≤ NN, 1 ≤ , 1 ≤ jj ≤ ≤ MM

F[F[ii][][jj] = max{F[] = max{F[ii-1][-1][kk]+R[]+R[ii][][jj--kk]}]}0 ≤ k ≤ j

Polygon TriangulationPolygon Triangulation

Given an Given an NN--sided convex sided convex polygon polygon AA, find a , find a triangulation triangulation scheme with scheme with minimum total minimum total cut lengthcut length

1

2

34

5

6

Polygon (analysis)Polygon (analysis)

Every edge of A belongs to exactly one Every edge of A belongs to exactly one triangle resulting from the triangulationtriangle resulting from the triangulation

1

2

4

5

6

4

1

34

2

We get two We get two (or one) (or one) smaller smaller polygons after polygons after deleting a deleting a triangletriangle

Polygon (analysis)Polygon (analysis)

The order of cutting does not matterThe order of cutting does not matterOptimal substructureOptimal substructure

If the cutting scheme for A is optimal, then the If the cutting scheme for A is optimal, then the cutting schemes for B and C must also be cutting schemes for B and C must also be optimaloptimal

A

B

C

Best Flying RouteBest Flying Route11

Cities from 1 to Cities from 1 to NN, from West to East, from West to EastOne-way flights connecting some pairs of One-way flights connecting some pairs of

citiescitiesTo fly from city 1 to city To fly from city 1 to city NN, always flying to , always flying to

the East, then back to city 1, always flying the East, then back to city 1, always flying to the Westto the West

Each city, except city 1, can be visited at Each city, except city 1, can be visited at most oncemost once

Best Flying RouteBest Flying Route

e.g. When N = 10, e.g. When N = 10, You can fly in the sequence {1, 2, 5, 8, 10, 7, You can fly in the sequence {1, 2, 5, 8, 10, 7,

3, 1} (if there are flights connecting each 3, 1} (if there are flights connecting each adjacent pair of them), but not in the adjacent pair of them), but not in the sequence {1, 4, 2, 5, 10, 1}, nor {1, 3, 5, 10, 5, sequence {1, 4, 2, 5, 10, 1}, nor {1, 3, 5, 10, 5, 1}, even if there are flights connectin each 1}, even if there are flights connectin each adjacent pair of them.adjacent pair of them.

Assume at least one such route existsAssume at least one such route existsMaximize the number of cities you can Maximize the number of cities you can

visit within the above constraintsvisit within the above constraints

Best Flying RouteBest Flying Route

Does the problem demonstrate optimal Does the problem demonstrate optimal substructure? Are the states memoryless?substructure? Are the states memoryless?Unfortunately, after flying from 1 to Unfortunately, after flying from 1 to NN, how we , how we

can fly back to 1 depends on what cities we can fly back to 1 depends on what cities we visited in flying to visited in flying to NN

How to formulate the problem? How are How to formulate the problem? How are subproblems related?subproblems related?

Time Complexity and Memory ComplexityTime Complexity and Memory ComplexityOutputting the routeOutputting the route

ReviewReview

Memoryless PropertyMemoryless PropertyOptimal substructure and Overlapping Optimal substructure and Overlapping

SubproblemsSubproblemsState representation and recurrence State representation and recurrence

formulaformulaBottom-up approach VS top-down Bottom-up approach VS top-down

approachapproachTradeoff between memory and runtimeTradeoff between memory and runtime

Common ModelsCommon Models

DP on rectangular arraysDP on rectangular arraysDP on treesDP on treesDP on optimized states (ugly states)DP on optimized states (ugly states)

This involves some state representationThis involves some state representation

Further OptimizationFurther Optimization

Memory optimization – rolling arrayMemory optimization – rolling arrayRuntime optimization – reducing the cost Runtime optimization – reducing the cost

in recurrencein recurrence

Practice ProblemsPractice Problems

1058 The Triangle II1058 The Triangle II3023 Amida De Go II3023 Amida De Go II30423042 Need for SpeedNeed for Speed6000 Palindrome6000 Palindrome6990 Little Shop of Flowers6990 Little Shop of Flowers