Z00 REND1011 11 SE MOD2 PP2 - Pearson Educationwps.pearsoned.co.uk/wps/media/objects/12987/13299709/rsh... · 2011-07-28 · Z00_REND1011_11_SE_MOD2 PP2.QXD 2/22/11 12:19 PM Page

M2-1

1. Understand the overall approach of dynamicprogramming.

2. Use dynamic programming to solve the shortest-route problem.

3. Develop dynamic programming stages.

After completing this module, students will be able to:

2

LEARNING OBJECTIVES

Dynamic Programming

MODULE

Summary • Glossary • Key Equations • Solved Problem • Self-Test • Discussion Questions and Problems • Case

Study: United Trucking • Internet Case Study • Bibliography

4. Describe important dynamic programming terminology.

5. Describe the use of dynamic programming in solvingknapsack problems.

M2.1 Introduction

M2.2 Shortest-Route Problem Solved Using DynamicProgramming

M2.3 Dynamic Programming Terminology

M2.4 Dynamic Programming Notation

M2.5 Knapsack Problem

MODULE OUTLINE

Z00_REND1011_11_SE_MOD2 PP2.QXD 2/22/11 12:18 PM Page 1

M2-2 MODULE 2 • DYNAMIC PROGRAMMING

M2.1 Introduction

Dynamic programming is a quantitative analysis technique that has been applied to large, com-plex problems that have sequences of decisions to be made. Dynamic programming dividesproblems into a number of decision stages; the outcome of a decision at one stage affects the de-cision at each of the next stages. The technique is useful in a large number of multiperiod busi-ness problems, such as smoothing production employment, allocating capital funds, allocatingsalespeople to marketing areas, and evaluating investment opportunities.

Dynamic programming differs from linear programming in two ways. First, there is no al-gorithm (as in the simplex method) that can be programmed to solve all problems. Dynamic pro-gramming is, instead, a technique that allows us to break up difficult problems into a sequenceof easier subproblems, which are then evaluated by stages. Second, linear programming is amethod that gives single-stage (one time period) solutions. Dynamic programming has thepower to determine the optimal solution over a one-year time horizon by breaking the probleminto 12 smaller one-month horizon problems and to solve each of these optimally. Hence, it usesa multistage approach.

Solving problems with dynamic programming involves four steps:

Four Steps of Dynamic Programming

1. Divide the original problem into subproblems called stages.2. Solve the last stage of the problem for all possible conditions or states.3. Working backward from the last stage, solve each intermediate stage. This is done by

determining optimal policies from that stage to the end of the problem (last stage).4. Obtain the optimal solution for the original problem by solving all stages sequentially.

In this module we show how to solve two types of dynamic programming problems: net-work and nonnetwork. The shortest-route problem is a network problem that can be solved bydynamic programming. The knapsack problem is an example of a nonnetwork problem that canbe solved using dynamic programming.

The second step is to solve thelast stage—stage 1.

M2.2 Shortest-Route Problem Solved Using Dynamic Programming

George Yates is about to make a trip from Rice, Georgia (1), to Dixieville, Georgia (7). Georgewould like to find the shortest route. Unfortunately, there are a number of small towns betweenRice and Dixieville. His road map is shown in Figure M2.1.

The circles on the map, called nodes, represent cities such as Rice, Dixieville, Brown, andso on. The arrows, called arcs, represent highways between the cities. The distance in miles isindicated along each arc. This problem can, of course, be solved by inspection. But seeing howdynamic programming can be used on this simple problem will teach you how to solve largerand more complex problems.

Step 1: The first step is to divide the problem into subproblems or stages. Figure M2.2 revealsthe stages of this problem. In dynamic programming, we usually start with the last part of theproblem, stage 1, and work backward to the beginning of the problem or network, which is stage3 in this problem. Table M2.1 summarizes the arcs and arc distances for each stage.

Step 2: We next solve stage 1, the last part of the network. Usually, this is trivial. We find theshortest path to the end of the network, node 7 in this problem. At stage 1, the shortest paths,

The first step is to divide theproblem into subproblems orstages.

Dynamic programming breaks a difficult problem into subproblems.


M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2-3

1 3

2

4

6

5

7

Stage 3

A Branch

Stage 2 Stage 1

A Node

452

10

126

4 10

2

14

FIGURE M2.2Three Stages for theGeorge Yates Problem

10 Miles

10 Miles

12 Miles

4 M

iles

Brown

2 M

iles

14 Miles

5 Miles

4 Miles

2 Miles

Lake City

Hope

Athens

Georgetown

Dixieville

4

1 3

2

5

6

7

6 Miles

Rice

FIGURE M2.1Highway Map betweenRice and Dixieville

STAGE ARC ARC DISTANCE

1 5–7 14

6–7 2

2 4–5 10

3–5 12

3–6 6

2–5 4

2–6 10

3 1–4 4

1–3 5

1–2 2

TABLE M2.1Distance Along Each Arc



452

10

126

4 10

2

14

Minimum Distanceto Node 7 fromNode 6


1 3

2

4

6

5

7

2

14

FIGURE M2.3Solution for the One-Stage Problem

452

10Minimum Distanceto Node 7 fromNode 3

12

6

4 10

2

14



1 3

2

4

6

5

7

8

12

14

2

24

FIGURE M2.4Solution for the Two-Stage Problem

Step 3: Moving backward, we now solve for stages 2 and 3. At stage 2 we will use Figure M2.4.Step 3 involves moving backwardsolving intermediate stages.

from node 5 to node 6, to node 7 are the only paths. Also note in Figure M2.3 that the minimumdistances are enclosed in boxes by the entering nodes to stage 1, node 5 and node 6. The objec-tive is to find the shortest distance to node 7. The following table summarizes this procedure forstage 1. As mentioned previously, the shortest distance is the only distance at stage 1.

STAGE 1

BEGINNING SHORTEST DISTANCE ARCS ALONGNODE TO NODE 7 THIS PATH

5 14 5–7

6 2 6–7


M2.2 SHORTEST-ROUTE PROBLEM SOLVED USING DYNAMIC PROGRAMMING M2-5

1 3

2

4

6

5

7

452

10

8

126

4 10

2

14

24


12

14

2

13

FIGURE M2.5Solution for the Three-Stage Problem

If we are at node 4, the shortest and only route to node 7 is arcs 4–5 and 5–7. At node 3, theshortest route is arcs 3–6 and 6–7 with a total minimum distance of 8 miles. If we are at node 2,the shortest route is arcs 2–6 and 6–7 with a minimum total distance of 12 miles. This informa-tion is summarized in the stage 2 table:

STAGE 2


4 24 4–5

5–7

3 8 3–6

6–7

2 12 2–6

6–7

The solution to stage 3 can be completed using the following table and the network inFigure M2.5:

STAGE 3


1 13 1–3

3–6

6–7



M2.3 Dynamic Programming Terminology

Regardless of the type or size of a dynamic programming problem, there are some importantterms and concepts that are inherent in every problem. Some of the more important of themfollow:

1. Stage: a period or a logical subproblem.2. State variables: possible beginning situations or conditions of a stage. These have also

been called the input variables.3. Decision variables: alternatives or possible decisions that exist at each stage.4. Decision criterion: a statement concerning the objective of the problem.5. Optimal policy: a set of decision rules, developed as a result of the decision criteria, that

gives optimal decisions for any entering condition at any stage.6. Transformation: normally, an algebraic statement that reveals the relationship between

stages.

In the shortest-route problem, the following transformation can be given:

This relationship shows how we are able to go from one stage to the next in solving for theoptimal solution to the shortest-route problem. In more complex problems, we can use symbolsto show the relationship between stages.

State variables, decision variables, the decision criterion, and the optimal policy can be de-termined for any stage of a dynamic programming problem. This is done here for stage 2 of theGeorge Yates shortest-route problem.

Distance from thebeginning of a givenstage to the last node

=

Distance from the beginningof the previous stage

to the last node+

Distance from thegiven stage to the

previous stage

The fourth and final step is tofind the optimal solution after allstages have been solved.

Step 4: To obtain the optimal solution at any stage, all we consider are the arcs to the next stageand the optimal solution at the next stage. For stage 3, we only have to consider the three arcs tostage 2 (1–2, 1–3, and 1–4) and the optimal policies at stage 2, given in a previous table. This ishow we arrived at the preceding solution. When the procedure is understood, we can perform allthe calculations on one network. You may want to study the relationship between the networksand the tables because more complex problems are usually solved by using tables only.

Dynamic Programming in Nursery Production Decisions

Managing a nursery that produces ornamental plants is diffi-cult. In most cases, ornamental plants increase in value with in-creased growth. This value-added growth makes it difficult todetermine when to harvest the plants and place them on themarket. When plants are marketed earlier, revenues are gener-ated earlier and the costs associated with plant growth are mini-mized. On the other hand, delaying the harvesting of theornamental plants usually results in higher prices. But are the ad-ditional months of growth and costs worth the delay?

In this case, dynamic programming was used to determinethe optimal growth stages for ornamental plants. Each stage wasassociated with a possible growth level. The state variablesincluded acres of production of ornamental plants and carryover

plants from previous growing seasons. The objective of thedynamic programming problem was to maximize the after-taxcash flow. The taxes included self-employment, federal income,earned income credit, and state income taxes.

The solution was to produce one- and three-gallon containersof ornamental plants. The one-gallon containers are sold in the falland carried over for spring sales. Any one-gallon containers notsold in the spring are combined into three-gallon container prod-ucts for sale during the next season. Using dynamic programminghelps to determine when to harvest to increase after-tax cash flow.

Source: Jeffrey Stokes, et al. “Optimal Marketing of Nursery Crops fromContainer-Based Production Systems,” American Journal of AgriculturalEconomics (February 1997): 235–246.

IN ACTION


M2.3 DYNAMIC PROGRAMMING TERMINOLOGY M2-7

1. State variables for stage 2 are the entering nodes, which are(a) node 2(b) node 3(c) node 4

2. Decision variables for stage 2 are the following arcs or routes:(a) 4–5(b) 3–5(c) 3–6(d) 2–5(e) 2–6

3. The decision criterion is the minimization of the total distances traveled.4. The optimal policy for any beginning condition is shown in Figure M2.6 and the following

table:

1 3

2

4

6

5

7

10

8

126

4 10

24

12

14

2

Statevariablesare theenteringnodes.

The optimal policy is the arc,for any entering node, thatwill minimize total distanceto the destination at thisstage.

Decisionvariablesare allthe arcs.

Stage 2

FIGURE M2.6Stage 2 from theShortest-Route Problem

GIVEN THIS ENTERING THIS ARC WILL MINIMIZECONDITION TOTAL DISTANCE TO NODE 7

2 2–6

3 3–6

4 4–5



An input, decision, output, andreturn are specified for eachstage.

The input to one stage is theoutput from another stage.

M2.4 Dynamic Programming Notation

In addition to dynamic programming terminology, we can also use mathematical notation to de-scribe any dynamic programming problem. This helps us to set up and solve the problem. Con-sider stage 2 in the George Yates dynamic programming problem first discussed in SectionM2.2. This stage can be represented by the diagram shown in Figure M2.7 (as could any givenstage of a given dynamic programming problem).

As you can see, for every stage, we have an input, decision, output, and return. Look againat stage 2 for the George Yates problem in Figure M2.6. The input to this stage is which con-sists of nodes 2, 3, and 4. The decision at stage 2, or choosing which arc will lead to stage 1, isrepresented by The possible arcs or decisions are 4–5, 3–5, 3–6, 2–5, and 2–6. The output tostage 2 becomes the input to stage 1. The output from stage 2 is The possible outputs fromstage 2 are the exiting nodes, nodes 5 and 6. Finally, each stage has a return. For stage 2, the re-turn is represented by In our shortest-route problem, the return is the distance along the arcsin stage 2. These distances are 10 miles for arc 4–5, 12 miles for arc 3–5, 6 miles for arc 3–6, 4miles for arc 2–5, and 10 miles for arc 2–6. The same notation applies for the other stages andcan be used at any stage. In general, we use the following notation for these important concepts:

(M2-1)

(M2-2)

(M2-3)

You should also note that the input to one stage is also the output from another stage. Forexample, the input to stage 2, is also the output from stage 3 (see Figure M2.7). This leads usto the following equation:

(M2-4)sn-1 = Output from stage n

s2,

rn = Return at stage n

dn = Decision at stage n

sn = Input to stage n

r2.

s1.d2.

s2,

Figure M2.6 may also be helpful in understanding some of the terminology used in the dis-cussion of dynamic programming.

Stage 2

Decisiond2

Outputs1

Returnr2

Inputs2

FIGURE M2.7Input, Decision, Output,and Return for Stage 2 inGeorge Yates’s Problem


M2.5 KNAPSACK PROBLEM M2-9

A transformation function allows us to go from one stage to another.

The total return function allows us to keep track of profits and costs.

The objective of this problem isto maximize profits.

M2.5 Knapsack Problem

The “knapsack problem” involves the maximization or the minimization of a value, such as prof-its or costs. As in a linear programming problem, there are restrictions in a knapsack problem.Imagine a knapsack or pouch that can hold only a certain weight or volume. We can place dif-ferent types of items in the knapsack. Our objective is to place items in the knapsack to maxi-mize total value without breaking the knapsack because of too much weight or a similarrestriction.

Types of Knapsack ProblemsMany kinds of problems can be classified as knapsack problems. Choosing items to place in thecargo compartment of an airplane and selecting which payloads to put on the next NASA SpaceShuttle are examples. The restriction can be volume, weight, or both. Some scheduling prob-lems are also knapsack problems. For example, we may want to determine which jobs to com-plete in the next two weeks. The two-week period is the knapsack, and we want to load it withjobs in such a way as to maximize profits or minimize costs. The restriction is the number ofdays or hours during the two-week period.

Roller’s Air Transport Service ProblemRob Roller owns and operates Roller’s Air Transport Service, which ships cargo by plane tomost large cities in the United States and Canada. The remaining capacity for one of theflights from Seattle to Vancouver is 10 tons. There are four different items that Rob can shipbetween Seattle and Vancouver. Each item has a weight in tons, a net profit in thousands ofdollars, and a total number of that item that is available for shipping. This information is pre-sented in Table M2.2.

PROBLEM SETUP Roller’s Air Transport problem is an ideal problem to solve using dynamicprogramming. Stage 4 will be item 1, stage 3 will be item 2, stage 2 will be item 3, and stage

The final concept is transformation. The transformation function allows us to go from onestage to another. The transformation function for stage 2, converts the input to stage 2, andthe decision made at stage 2, to the output from stage 2, Because the transformation func-tion depends on the input and decision at any stage, it can be represented as In gen-eral, the transformation function can be represented as follows:

(M2-5)

The following general formula allows us to go from one stage to another using the transforma-tion function:

(M2-6)

Although this equation may seem complex, it is really just a mathematical statement of the factthat the output from a stage is a function of the input to the stage and any decisions made at thatstage. In the George Yates shortest-route problem, the transformation function consists of anumber of tables. These tables show how we could progress from one stage to another in orderto solve the problem. For more complex problems, we need to use dynamic programming nota-tion instead of tables.

Another useful quantity is the total return at any stage. The total return allows us to keeptrack of the total profit or costs at each stage as we solve the dynamic programming problem. Itcan be given as follows:

(M2-7)fn = Total return at stage n

sn-1 = tn1sn, dn2

tn = Transformation function at stage n

t2 (s2, d2).s1.d2,

s2,t2,



TABLE M2.2Items to Be Shipped

Defining the ProblemThe Southern Company, with service areas in Georgia, Alabama, Mississippi, and Florida, is a majorprovider of electric service, with about 240 generating units. In recent years, fuel costs have increasedfaster than other costs. Annual fuel costs are about $2.5 billion, representing about one-third of total ex-penses for the Southern Company. The problem for the Southern Company is to reduce total fuel costs.

Developing a ModelTo deal with this fuel cost problem, the company developed a state-of-the-art dynamic programmingmodel. The dynamic programming model is embedded in the Wescouger optimization program, which isa computer program used to control electric generating units and reduce fuel costs through better utiliza-tion of existing equipment.

Acquiring Input DataData were collected on past and projected electric usage. In addition, daily load/generation data were an-alyzed. Load/generation charts were used to investigate the fuel requirements for coal, nuclear, hydroelec-tric, and gas/oil.

Developing a SolutionThe solution of the dynamic programming model provides both short-term planning guidelines and long-term fuel usage for the various generating units. Optimal maintenance schedules for generating units areobtained using Wescouger.

Testing the SolutionTo test the accuracy of the Wescouger optimization program, Southern used a real-time economic dis-patch program. The results were a very close match. In addition, the company put the solution through anacid test, in which seasoned operators compared the results against their intuitive judgment. Again, theresults were consistent.

Analyzing the ResultsThe results were analyzed in terms of their impact on the use of various fuels, the usage of various gener-ating units, and maintenance schedules for generating units. Analyzing the results also revealed otherneeds. This resulted in a full-color screen editing routine, auxiliary programs to automate data input, andsoftware to generate postanalysis reports.

Implementing the ResultsThe Southern Company implemented the dynamic programming solution. Over a seven-year period, theresults saved the company over $140 million.

Source: S. Erwin, et al. “Using an Optimization Software to Lower Overall Electric Production Costs for Southern Company,”Interfaces 21, 1 (January–February 1991): 27–41.

MODELING IN THE REAL WORLD Reducing Electric Production CostsUsing Dynamic Programming

Definingthe Problem

Developinga Model

AcquiringInput Data

Developinga Solution

Testing theSolution

Analyzingthe Results

Implementingthe Results

ITEM WEIGHT PROFIT/UNIT NUMBER AVAILABLE

1 1 $3 6

2 4 9 1

3 3 8 2

4 2 5 2



s1Stage 2(Item 3)

Stage 3(Item 2)

d4

s3

r4

s4 Stage 4(Item 1)

s0Stage 1(Item 4)

d3

r3

d2

r2

d1

r1

Decisions

Returns

s2

FIGURE M2.8Roller’s Air TransportService Problem

1 will be item 4. This is shown in Table M2.3. During the solution, we will be using stagenumbers.

Roller’s Air Transport problem can be represented graphically (see Figure M2.8). As youcan see, each item is represented by a stage. Look at stage 4, which is item 1. The total weightthat can be used is represented by This amount is 10 because we haven’t assigned any unitsto be shipped at this time. The decision at this stage is (the number of units of item 1 we willship). If is 1, for example, we will be shipping 1 unit of item 1. Also note that is the returnor profit at stage 4 (item 1). If we ship 1 unit of item 1, the profit will be $3 (see Table M2.2).

As mentioned previously, the decision variable represents the number of units of eachitem (stage) that can be shipped. Looking back at the original problem, we see that the problemis constrained by the number of units. This is summarized in the following table:

dn,

r4d4

d4

s4.

TABLE M2.3Relationship BetweenItems and Stages

STAGE MAXIMUM VALUE OF DECISION

4 6

3 1

2 2

1 2

THE TRANSFORMATION FUNCTIONS Next, we need to look at the transformation function. Thegeneral transformation function for knapsack problems follows:

Note that and are coefficients for the problem and that represents the decision atstage n. This is the number of units to ship at stage sn.

dncnan, bn,

sn-1 = (an * sn) + (bn * dn) + cn

ITEM STAGE

1 4

2 3

3 2

4 1



The following chart shows the transformation coefficients for Rob Roller’s transport problem:

COEFFICIENTS OFTRANSITION FUNCTION

STAGE an bn cn

4 1 1 0

3 1 4 0

2 1 3 0

1 1 2 0-

-

-

-

First, note that is 10, the total weight that can be shipped. Because represents the first item,all 10 tons can be utilized. The transformation equations for the four stages are as follows:

(a)

(b)

(c)

(d)

Consider stage 3. Equation b reveals that the number of tons still available after stage 3, isequal to the number of tons available before stage 3, minus the number of tons shipped atstage In this equation, means that each item at stage 3 weighs 4 tons.

THE RETURN FUNCTIONS Next, we will look at the return function for each stage. This is thegeneral form for the return function:

Note that and are the coefficients for the return function. Using this general form ofthe return function, we can put the return function values in the following table:

cnan, bn,

rn = 1an * sn2 + 1bn * dn2 + cn

4d33, 4d3.s3,

s2,

s0 = s1 - 2d1 stage 1

s1 = s2 - 3d2 stage 2

s2 = s3 - 4d3 stage 3

s3 = s4 - 1d4 stage 4

s4s4

COEFFICIENTS OFDECISIONS RETURN FUNCTION

STAGE LOWER UPPER an bn cn

4 0 dn 6 0 3 0

3 0 dn 1 0 9 0

2 0 dn 2 0 8 0

1 0 dn 2 0 5 0……

……

……

……



The lower value for each decision is zero, and the upper value is the total number available.The coefficient is the profit per item shipped. The actual return functions are:

STAGE-BY-STAGE SOLUTION As you would expect, the return at any stage is equal to theprofit per unit at that stage multiplied by the number of units shipped at that stage, With thisinformation, we can solve Roller’s Air Transport problem, starting with stage 1 (item 4). Thefollowing table shows the solution for the first stage. You may wish to refer to Figure M2.8 forthis discussion.

dn.rn,

r1 = 5d1

r2 = 8d2

r3 = 9d3

r4 = 3d4

bn

Because we don’t know how many tons will be available at stage 1, we must consider allpossibilities. Thus, the number of tons available at stage 1, can vary from 1 to 10. This is seenin the first column of numbers for stage 1. The number of units that we ship at stage 1, canvary from 0 to 2. We can’t go over 2 because the number available is only 2. For any decisionwe compute the return at stage 1, by multiplying the number of items shipped by 5, the profitper item. The profit at this stage will be 0, 5, or 10, depending on whether 0, 1, or 2 items areshipped. Note that the total return at this stage, is the same as because this is the only stagewe are considering so far. Also note that the total return before stage 1, is 0 because this isthe beginning of the solution and we are shipping nothing at this point.

f0,r1f1,

r1,

d1,s1,

We consider all possibilities.

is the total return at the firststage. The total return before thefirst stage is f0.

f1

STAGE 1

S1 d1 r1 s0 f0 f1

0 0 0 0 0 0

1 0 0 0 0 0

2 0 0 0 0 0

1 5 0 0 5

3 0 0 0 0 0

1 5 0 0 5

4 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

5 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

6 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

STAGE 1

S1 d1 r1 s0 f0 f1

7 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

8 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

9 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10

10 0 0 0 0 0

1 5 0 0 5

2 10 0 0 10



The solution for stage 1 shows the best decision, the return for this stage, and the total re-turn, given all possible number of tons available (0 to 10 tons). Using the results of stage 1, wecan now proceed to stage 2. The solution for stage 2 is as follows:

STAGE 2

S2 d2 r2 s1 f1 f2

0 0 0 0 0 0

1 0 0 1 0 0

2 0 0 2 2 5

3 0 0 3 5 5

1 8 0 0 8

4 0 0 4 10 10

1 8 1 0 8

5 0 0 5 10 10

1 8 2 5 13

6 0 0 6 10 10

1 8 3 5 13

2 16 0 0 16

STAGE 2

S2 d2 r2 s1 f1 f2

7 0 0 7 10 10

1 8 4 10 18

2 16 1 0 16

8 0 0 8 10 10

1 8 5 10 18

2 16 2 5 21

9 0 0 9 10 10

1 8 6 10 18

2 16 3 5 21

10 0 0 10 10 10

1 8 7 10 18

2 16 4 10 26

The solution for stage 2 is found in exactly the same way as for stage 1. At stage 2 we stillhave to consider all possible number of tons available (from 0 to 10). See the column (the firstcolumn). At stage 2 (item 3) we still only have 2 units that can be shipped. Thus, (second col-umn) can range from 0 to a maximum of 2. The return for each and combination at stage 2,

is shown in the third column. These numbers are the profit per item at this stage, 8, times thenumber of items shipped. Because items shipped at stage 2 can be 0, 1, or 2, the profit at thisstage can be 0, 8, or 16. The return for stage 2 can also be computed from the return function:

Now look at the fourth column, which lists the number of items available after stage 2.This is also the number of items available for stage 1. To get this number, we have to subtractthe number of tons we are shipping at stage 2 (which is the tonnage per unit times the number ofunits) from the number of tons available before stage 2 Look at the row in which is 6 and

is 1. We have 6 tons available before stage 2, and we are shipping 1 item, which weighs 3tons. Thus we will have 3 tons still available after this stage. The values can also be deter-mined using the transformation function, which is

The last two columns of stage 2 contain the total return. The return after stage 1 and beforestage 2 is These are the same values that appeared under the column for stage 1. The re-turn after stage 2 is It is equal to the return from stage 2 plus the total return before stage 2.

Stage 3 is solved in the same way as stages 1 and 2. The following table presents thesolution for stage 3; look at each row and make sure that you understand the meaning of eachvalue.

f2.f1f1.

s1 = s2 - 3d2.s1

d2

s2(s2).

s1,r2 = 8d2.

r2,d2s2

d2

s2



Now we solve the last stage of the problem, stage 4. The following table shows the solutionprocedure:

The first thing to note is that we only have to consider one value for because we knowthe number of tons available for stage 4; must be 10 because we have all 10 tons available.There are six possible decisions at this stage, or six possible values for because the numberof available units is 6. The other columns are computed in the same way. Note that the return af-ter stage 4, is the total return for the problem. We see that the highest profit is 28. We alsosee that there are three possible decisions that will give this level of profit, shipping 4, 5, or 6items. Thus we have alternate optimal solutions. One possible solution is as follows:

f4,

d4,s4

s4,

STAGE 3

s3 d3 r3 s2 f2 f3

0 0 0 0 0 0

1 0 0 1 0 0

2 0 0 2 2 5

3 0 0 3 8 8

4 0 0 4 10 10

1 9 0 0 9

5 0 0 5 13 13

1 9 1 0 9

6 0 0 6 16 16

1 9 2 5 14

STAGE 3

s3 d3 r3 s2 f2 f3

7 0 0 7 18 18

1 9 3 8 17

8 0 0 8 21 21

1 9 4 10 19

9 0 0 9 21 21

1 9 5 13 22

10 0 0 10 26 26

1 9 6 16 25

STAGE 4

s4 d4 r4 s3 f3 f4

10 0 0 10 26 26

1 3 9 22 25

2 6 8 21 27

3 9 7 18 27

4 12 6 16 28

5 15 5 13 28

6 18 4 10 28

FINAL SOLUTION

STAGE OPTIMAL DECISION OPTIMAL RETURN

4 6 18

3 0 0

2 0 0

1 2 10

Total 8 28



We start with shipping 6 units at stage 4. Note that is 4 from the stage 4 calculations,given that is 6. We use this value of 4 and go to the stage 3 calculations. We find the rows inwhich is 4 and pick the row with the highest total return, In this row is 0 items with atotal return of 10. As a result, the number of units available, is still 4. We next go to thecalculations for stage 2 and then stage 1 in the same way. This gives us the optimal solution al-ready described. see if you can find one of the alternate optimal solutions.

s2,(f3)d3f3.s3

d4

s3

Key Equations

(M2-1)The input to stage n. This is also the output from stage

(M2-2)The decision at stage n.

(M2-3)The return function, usually profit or loss, at stage n.

(M2-4)The output from stage n. This is also the input to stagen - 1.

sn-1 = Output from stage n

rn = Return at stage n

dn = Decision at stage n

n + 1.

sn = Input to stage n (M2-5)A transformation function that allows us to go fromone stage to another.

(M2-6)The general relationship that shows how the outputfrom any stage is a function of the input to the stageand the decisions made at that stage.

(M2-7)This equation gives the total return (profit or costs) atany stage. It is obtained by summing the return at eachstage, rn.

fn = Total return at stage n

sn-1 = tn1sn, dn2

tn = Transformation function at stage n

Summary

Dynamic programming is a flexible, powerful technique. Alarge number of problems can be solved using this technique,including the shortest-route and knapsack problems. Theshortest-route problem finds the path through a network thatminimizes total distance traveled, and the knapsack problemmaximizes the value or return. An example of a knapsack prob-lem is to determine what to ship in a cargo plane to maximizetotal profits given the weight and size constraints of the cargoplane. The dynamic programming technique requires foursteps: (1) Divide the original problem into stages, (2) solve thelast stage first, (3) work backward solving each subsequentstage, and (4) obtain the optimal solution after all stages havebeen solved.

Dynamic programming requires that we specify stages,state variables, decision variables, decision criteria, an optimalpolicy, and a transformation function for each specific prob-lem. Stages are logical subproblems. State variables are possi-ble input values or beginning conditions. The decisionvariables are the alternatives that we have at each stage, suchas which route to take in a shortest-route problem. The deci-sion criterion is the objective of the problem, such as findingthe shortest route or maximizing total return in a knapsackproblem. The optimal policy helps us obtain an optimal solu-tion at any stage, and the transformation function allows us togo from one stage to the next.

Glossary

Decision Criterion A statement concerning the objective ofa dynamic programming problem.

Decision Variable The alternatives or possible decisions thatexist at each stage of a dynamic programming problem.

Dynamic Programming A quantitative technique thatworks backward from the end of the problem to the begin-ning of the problem in determining the best decision for anumber of interrelated decisions.

Optimal Policy A set of decision rules, developed as a resultof the decision criteria, that gives optimal decisions at anystage of a dynamic programming problem.

Stage A logical subproblem in a dynamic programmingproblem.

State Variable A term used in dynamic programming todescribe the possible beginning situations or conditions of a stage.

Transformation An algebraic statement that shows therelationship between stages in a dynamic programmingproblem.


SOLVED PROBLEM M2-17

Solved Problem

Solved Problem M2-1Lindsey Cortizan would like to travel from the university to her hometown over the holiday season. Aroad map from the university (node 1) to her home (node 7) is shown in Figure M2.9. What is the bestroute that will minimize total distance traveled?

1

2

3

4

5

6

7

20

18

22

39

28

36

18

13

10

FIGURE M2.9Road Map forLindsey Cortizan

1

4

3

2

6

5

7

20

18

22

39

28

36

18

13

10

Stage 3 Stage 2 Stage 1

50 41

28

52

10

13

FIGURE M2.10Solution for theLindsey CortizanProblem

SolutionThe solution to this problem is identical to the one presented earlier in the chapter. First, the problem isbroken into three stages. See the network in Figure M2.10. Working backward, we start by solving stage3. The closest and only distance from node 6 to 7 is 13, and the closest and only distance from node 5 tonode 7 is 10. We proceed to stage 2. The minimum distances are 52, 41, and 28 from node 4, node 3,and node 2 to node 7. Finally, we complete stage 3. The optimal solution is 50. The shortest route is1–2–5–7 and is shown in the following network. This problem can also be solved using tables, as shownfollowing the network solution.

Problem type: Minimization network

Number of stages: 3

Transition function type:

Recursion function type: fn = rn + fn-1

sn-1 = sn - dn



STAGE NUMBER OF DECISIONS

3 3

2 4

1 2

STAGE STARTING NODE ENDING NODE RETURN VALUE

3 1 2 22

1 3 18

1 4 20

2 2 5 18

2 6 36

3 6 28

4 6 39

1 5 7 10

6 7 13

STAGE 1

s1 d1 r1 s0 f0 f1

6 6–7 13 7 0 13

5 5–7 10 7 0 10

STAGE 2

s2 d2 r2 s1 f1 f2

4 4–6 39 6 13 52

3 3–6 28 6 13 41

2 2–6 36 6 13 49

2–5 18 5 10 28

STAGE 3

s3 d3 r3 s2 f2 f3

1 1–4 20 4 52 72

1–3 18 3 41 59

1–2 22 2 28 50

FINAL SOLUTION

STAGE OPTIMAL DECISION OPTIMAL RETURN

3 1–2 22

2 2–5 18

1 5–7 10

Total 50


SELF-TEST M2-19

Self-Test

� Before taking the self-test, refer back to the learning objectives at the beginning of the module, the notes in the margins, andthe glossary at the end of the module.

� Use the key at the back of the book to correct your answers.� Restudy pages that correspond to any questions that you answered incorrectly or material you feel uncertain about.

c. a decision variable.d. the optimal policy.e. a stage.

9. The statement that the distance from the beginning stageis equal to the distance from the preceding stage to thelast node plus the distance from the given stage to the pre-ceding stage is calleda. the transformation.b. state variables.c. decision variables.d. the optimal policy.e. stages.

10. In dynamic programming, isa. the input to the stage n.b. the decision at stage n.c. the return at stage n.d. the output of stage n.e. none of the above.

11. The distance from the beginning stage is equal to the dis-tance from the preceding stage to the last node plus thedistance for the given stage to the preceding stage. Thisrelationship is used to solve which type of problem?a. knapsackb. JITc. shortest routed. minimal spanning treee. maximal flow

12. In using dynamic programming to solve a shortest-routeproblem, the distance from one point to the next would becalled aa. state.b. stage.c. return.d. decision.

13. In using dynamic programming to solve a shortest-routeproblem, the decision variables at one stage of the prob-lem would bea. the distances from one node to the next.b. the possible arcs or routes that can be selected.c. the number of possible destination nodes.d. the entering nodes.

14. In using dynamic programming to solve a shortest-routeproblem, the entering nodes would be calleda. the stages.b. the state variables.c. the returns.d. the decision variables.

sn

1. Dynamic programming divides problems intoa. nodes.b. arcs.c. decision stages.d. branches.e. variables.

2. Possible beginning situations or conditions of a dynamicprogramming problem are calleda. stages.b. state variables.c. decision variables.d. optimal policy.e. transformation.

3. The statement concerning the objective of a dynamic pro-gramming problem is calleda. stages.b. state variables.c. decision variables.d. optimal policy.e. decision criterion.

4. The first step of a dynamic programming problem is toa. define the nodes.b. define the arcs.c. divide the original problem into stages.d. determine the optimal policy.e. none of the above.

5. In working a problem with dynamic programming, weusuallya. start at the first part of the problem and work forward

to the next parts.b. start at the end of the problem and work backward.c. start at the most expensive part of the problem.d. start at the least expensive part of the problem.

6. An algebraic statement that reveals the relationship be-tween stages is calleda. the transformation.b. state variables.c. decision variables.d. the optimal policy.e. the decision criterion.

7. In this chapter, dynamic programming is used to solvewhat type of problem?a. quantity discountb. just-in-time inventoryc. shortest routed. minimal spanning treee. maximal flow

8. In dynamic programming terminology, a period or logicalsubproblem is calleda. the transformation.b. a state variable.



Discussion Questions and Problems

Discussion QuestionsM2-1 What is a stage in dynamic programming?

M2-2 What is the difference between a state variable and adecision variable?

M2-3 Describe the meaning and use of a decision crite-rion.

M2-4 Do all dynamic programming problems require anoptimal policy?

M2-5 Why is transformation important for dynamic pro-gramming problems?

ProblemsM2-6 Refer to Figure M2.1. What is the shortest route be-

tween Rice and Dixieville if the road between Hopeand Georgetown is improved and the distance is re-duce to 4 miles?

M2-7 Due to road construction between Georgetown andDixieville, a detour must be taken through countryroads (Figure M2.1). Unfortunately, this detour hasincreased the distance from Georgetown to Dix-ieville to 14 miles. What should George do? Shouldhe take a different route?

M2-8 The Rice brothers have a gold mine between Riceand Brown. In their zeal to find gold, they haveblown up the road between Rice and Brown. Theroad will not be in service for five months. Whatshould George do? Refer to Figure M2.1.

M2-9 The Rice brothers (Figure M2.1) wish to determinethe potential savings from using the shortest routefrom Rice to Dixieville rather than just randomly se-lecting any route. Use dynamic programming to findthe longest route from Rice to Dixieville. How muchfarther is this than the shortest route?

M2-10 Solve the shortest-route problem for the networkshown in Figure M2.11.

M2-11 Solve the shortest-route problem for the networkshown in Figure M2.12.

M2-12 Mail Express, an overnight mail service, deliversmail to customers throughout the United States,Canada, and Mexico. Fortunately, Mail Express hasadditional capacity on one of its cargo planes. Tomaximize profits, Mail Express takes shipmentsfrom local manufacturing plants to warehouses forother companies. Currently, there is room for an-other 6 tons. The following table shows the itemsthat can be shipped, their weights, the expectedprofit for each, and the number of available parts.How many units of each item do you suggest thatMail Express ship?

1 5

2

3

7

4

8

6

9

5

4

11

10

4

12

6

10

4

6

24

6

FIGURE M2.11(for Problem M2-10)

ITEMS TO BE SHIPPED

WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $3 6

2 2 9 1

3 3 8 2

4 1 2 2

M2-13 Leslie Bessler must travel from her hometown toDenver to see her friend Austin. Given the road mapof Figure M2.13, what route will minimize the dis-tance that Leslie travels?

M2-14 An air cargo company has the shipping requirementsshown in the accompanying table. Two planes areavailable with a total capacity of 11 tons. How manyof each item should be shipped to maximize profits?


DISCUSSION QUESTIONS AND PROBLEMS M2-21

1

3

2

6

5

4

7

8

9

10

11

3

4

5

3

2

2

7

5

2

2

4 3

3

3

6


M2-15 Because of a new manufacturing and packaging pro-cedure, the weight of item 2 in Problem M2-14 canbe cut in half. Does this change the number or typesof items that can be shipped by the air transportcompany?

M2-16 What is the shortest route through the network inFigure M2.14?

M2-17 Refer to Problem M2-16. The road between node 6and node 11 is no longer in service due to construc-tion. What is the shortest route, given this situation?

M2-18 In Problem M2-16, what is the shortest distancefrom node 6 to the ending node? How does thischange if the road between node 6 and node 11 is nolonger in service (see Problem M2-17)?

M2-19 Paige Schwartz is planning to go camping in BigBend National Park. She will carry a backpack withfood and other items, and she wants to carry no morethan 18 pounds of food. The four possible food itemsshe is considering each have certain nutritional ingre-dients. These items and their weights are as follows:

How many of each item should paige carry if shewishes to maximize the total nutritional units?

1

4

3

2

5

6

9

8

7

10

11

12

2

1

3 3

4

4

3 3

2

3

2

3

4

2

3

3


ITEMS TO BE SHIPPED

WEIGHT NUMBERITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $3 6

2 2 9 1

3 3 8 2

4 2 5 5

5 5 8 6

6 1 2 2

ITEM WEIGHT (POUNDS) NUTRITIONAL UNITS

A 3 600

B 4 1,000

C 5 1,500

D 2 300



1 3

2

47

8

6

59

10

11

12

13

16

15

14

17

18

19 20

4

2

3 3

3

4

3

4

3

3

4

3

3

2

4

3

4

4

4

3

2

5

2

2

2


Case Study

United Trucking

Like many other trucking operations, United Trucking gotstarted with one truck and one owner—Judson Maclay. Judsonis an individualist and has always liked to do things his way.He was a fast driver, and many people called the 800 numberon the back of his truck when he worked for Hartmann Truck-ing. After two years with Hartmann and numerous calls abouthis bad driving, Judson decided to go out on his own. Hestarted United Trucking.

In the early days of United Trucking, Judson was the onlydriver. On the back of his truck was the message: How do youlike my driving? Call 1-800-AMI-FAST. He was convincedthat some people actually tried to call the number. Soon, a num-ber of truck operators had the same or similar messages on theback of their trucks. After three years of operation, Judson had15 other trucks and drivers working for him. He traded his driv-ing skills for office and management skills. Although 1-800-AMI-FAST was no longer visible, Judson decided to neverplace an 800 number on the back of any of his trucks. If some-one really wanted to complain, they could look up UnitedTrucking in the phone book.

Judson liked to innovate with his trucking company. Heknew that he could make more money by keeping his trucksfull. Thus he decided to institute a discount trucking service. Hegave substantial discounts to customers that would accept deliv-ery to the West Coast within two weeks. Customers got a greatprice, and he made more money and kept his trucks full. Overtime, Judson developed steady customers that would usually

have loads to go at the discounted price. On one shipment, hehad an available capacity of 10 tons in several trucks going tothe West Coast. Ten items can be shipped at discount. Theweight, profit, and number of items available are shown in thefollowing table:

WEIGHTITEM (TONS) PROFIT/UNIT AVAILABLE

1 1 $10 2

2 1 10 1

3 2 5 3

4 1 7 20

5 3 25 2

6 1 11 1

7 4 30 2

8 3 50 1

9 1 10 2

10 1 5 4

Discussion Questions1. What do you recommend that Judson do?2. If the total available capacity were 20 tons, how would

this change Judson’s decision?

See our Internet home page at www.pearsonhighered.com/render for this additional case study:Briarcliff Electronics.

Internet Case Study


BIBLIOGRAPHY M2-23

BibliographyBellman, R. E. Dynamic Programming. Princeton, NJ: Princeton University

Press, 1957.

Bourland, Karla. “Parallel-Machine Scheduling with Fractional Operator Re-quirements,” IEE Transactions (September 1994): 56.

Elmaghraby, Salah. “Resource Allocation via Dynamic Programming,”European Journal of Operations Research (January 1993): 199–215.

El-Rayes, Khaled, et al. “Optimized Scheduling for Highway Construction,”Transactions of AACE International (February 1997): 311–315.

Hillard, Michael R., et al. “Scheduling the Operation Desert Storm Airlift: AnAdvanced Automated Scheduling Support System,” Interfaces 21, 1(January–February 1992): 131–146.

Howard, R. A. Dynamic Programming. Cambridge, MA: The MIT Press,1960.

Ibarake, Toshihide, and Yuichi Nakamura. “A Dynamic Programming Methodfor Single Machine Scheduling,” European Journal of OperationsResearch (July 1994): 72–82.

Idem, Fidelis, and A. M. Reisman. “An Approach to Planning for PhysicianRequirements in Developing Countries Using Dynamic Programming,”Operations Research (July 1990): 607–618.

Stokes, Jeffrey, et al. “Optimal Marketing of Nursery Crops from Container-Based Production Systems,” American Journal of AgriculturalEconomics (February 1997): 235–246.



Documents

Z00 REND1011 11 SE MOD2 PP2 - Pearson Educationwps.pearsoned.co.uk/wps/media/objects/12987/13299709/rsh... · 2011-07-28 · Z00_REND1011_11_SE_MOD2 PP2.QXD 2/22/11 12:19 PM Page