Part 1 Overview, introduction,

examples

• What is Operations Research?• What is Optimization• What is Sequential Decision

Making?• What is Dynamic Programming?• Examples, please!

Perspective

Universe

AppliedMaths

OR

OptimizationSequential Decision

Making

620-113

What is Operations Research?

• A branch of applied mathematics dealing with the Science of Decision Making

• Also known as Management Science• One of the streams at the Maths & Stats

Department• More information at www.ifors.org• Local chapter

www.asor.ms.unimelb.edu.au/melbourne

What is optimization?• Extensive literature (books,

journals, encyclopedia)• Education and training programs• Jobs• Commercial software (eg. Excel)• Consulting services• Conferences and workshops• Meeting place: e-optimization.com

Convention• Universal Convention for the description

of optimisation problems

opt Objective (dvs) dvs

Subject to: Constraints (dvs)

• Opt: either min or max.• Objective: an expression

(function) quantifying how good/bad our decisions are.

• dvs: decision variables• Constraints:

Restrictions on the decision variables.

Help Desk

opt Objective (dvs) dvs

Subject to: Constraints (dvs)

TipUsually it is best to build

the model in the following order:– dvs (Decision variables)– Objective– Opt– Constraints But, ….. do not be dogmatic!

Naïve ExampleEnglish Version

• Given:The value of the perimeter of a rectangle

• Task: Determine the width and height of the rectangle that will make its area as large as possible

Maths Version

maxw,h

A=w×h

s.t.

2(w+h) =p

Which one do you like better?

Help Desk

maxw,h

A=w×h

s.t.

2(w+h) =p

Formally• Opt = max • dvs: w,h• Objective: w x h• Constraint:

2(w + h) = p

Help Desk

h

w

Perimeter = 2(w+h)Area = w x h

maxw,h

A=w×h

s.t.

2(w+h) =p

Beware of the distinction between

• Problem FormulationDescribing what the

problem is all about

• Problem SolvingConstructing a

solution to a given problem

• Most students under estimate the difficulties encountered at the formulation stage

Naïve Example

English Version• Given:The value of the

perimeter of a rectangle

• Task: Determine the width and height of the rectangle that will make its area as large as possible

Maths Version

maxw,h

A=w×h

s.t.

2(w+h) =p

This is the (maths) model

Observations

maxw,h

A=w×h

s.t.

2(w+h) =p

There are infinitely many feasible solutions: Example

2(w + h) = p = 6 w = 3 - h/2We can let h take

value in the range (0,3).

Observations

• Not difficult to show that the optimal solution is

maxw,h

A=w×h

s.t.

2(w+h) =pw=h=p4

A=p2

16

I am asquare Prove this!

Strong

Operations Research

Flavour

Warning

Parental Guidance is required

Role of Models in Problem SolvingRealisation

InformalDescription

FormalModel Solution

WorkingSystem

Modelling

Analysis

Implementation

Monitoring

mathematical

^

Don’t panic!

In Real-life

Realisation

InformalDescription

FormalModel Solution

WorkingSystem

Modelling

Analysis

Implementation

Monitoring

We shall focus on

Realisation

InformalDescription

FormalModel Solution

WorkingSystem

Modelling

Analysis

Implementation

MonitoringGiven

Beware of the distinction between

• Problem FormulationDescribing what the

problem is all about

• Problem SolvingConstructing a

solution to a given problem

• Most students under estimate the difficulties encountered at the formulation stage

Example (Knapsack Problem)

English Version• Given: A container

and a collection of items having known values and volumes.

• Task: Determine what is the sub-collection of items of maximum total value that will fit into the container.

Maths Version

?????

5 minutes

English Version• Given: A container

and a collection of items having known values and volumes.

• Task: Determine what is the sub-collection of items of maximum value that will fit into the container.

Maths Version

Help!

What is a Sequential Decision Problem ?

• A problem (typically an optimization problem) representing a sequential decision making situation.

• Example: Knapsack problem.• Note the distinction between the problem

itself and the formulation given to it.

What is a Dynamic Programming?

Short version• A paradigm for the

formulation, analysis and solution for sequential decision problems.

Answer

Long version• Wait and see …

• Difficult to teach• Difficult to learn• Usually done “by example”

Important Facts about DP

DP Approach to Problem Solving

• Step 1: Transform your problem into a family of related problems

• Step 2: Derive a functional equation relating the solutions to these problems to each other

• Step 3: Solve the functional equation• Step 4: Recover from the functional

equation a solution to your initial problem

Trouble is,

This is easier said than done

The Art of DP

hence

Most Famous Example(Shortest Path Problem)

• Given: a set of nodes (cities) and the lengths of the arcs connecting them

• Task: Find the shortest path from some given node to another given node.

1 2 3 4 5

6 7 8 9 10

11 12 13 14 15

1 2 1 2

1

2

10152010

1 2 1 2

110152010

121

Step 1: Create a Family of Related Problems

• Suppose we are given say m cities and have to determine the shortest path from city s to city d.

• Let f(j) := shortest distance from city s

to city j, j=1,2,3,…,m. Note: we now have m problems.

Step 2: Deriving a Functional Equation

• From the definition of f(j) it follows that f(j) = t(k,j) + f(k)

for some immediate predecessor k of j.• Hence,

f( j) =mink∈P ( j )

{t(k, j)+f(k)}

where t denotes the distance matrix and P(j) denotes the set of immediate predecessors of node j.

Example

f(8) = mink∈P (8)

{t(k,8)+f(k)}

1 2 3 4 5

6 7 8 9 10

11 12 13 14 15

1 2 1 2

1

2

10152010

1 2 1 2

110152010

121

=mint(3,8)+f(3)

t(7,8)+f(7)

⎧ ⎨ ⎩

⎫ ⎬ ⎭

=min15+f(3),2+f(7){ }

Step 3: Solve the FE

• Can be– Very easy– Easy– Difficult– Very difficult– Impossible

Example (naïve)1

13

D

E

A B

C

1

2

• f(A) = 0 (by definition?)• f(B) = t(A,B) + f(A) = 3 + 0 = 3• f(C) = t(B,C) + f(B) = 1 + 3 = 4• f(D) = t(B,D) + f(B) = 1 + 3 = 4• f(E) = min {t(C,E) + f(C),t(D,E) +

f(D)} = min {1+4,2+4} = 5

Step 4: Recover a solution

• Once the functional equation is solved, it is usually simple to recover an optimal solution to the original problem. (Hint: go in …. reverse)

Example (naïve)

1

1

3

D

E

A B

C

1

2

• Optimal solutions to the FE (going from A to E)

1

1

3

D

E

A B

C

1

2

• Optimal solution to the original problem (going from E to A)

So what is all the fuss about DP?

• The examples we have seen so far are extremely simple (in fact naïve)

• There are major complications

A (very) Famous Example

Task: Move pieces from left to right

Rules• One piece at a time• Large on small is not allowed

• Objective ??

Analysis

Analysis

Given n pieces• Move n-1 pieces to the intermediate position• Move the bottom one to the destination• Move the n-1 pieces from the intermediate

position to the destination

Formalism

Formalism

Let S(n,x,y) := Solution to a problem involving

moving n pieces from position x to position y.

Thus, S(1,x,y) := Move a piece from x to y.Theorem:• S(n,x,y) = S(n-1,x,~(x,y)) , S(1,x,y) , S(n-

1,~(x,y),y)

• Length [S(n,x,y)] = 2n - 1

Start

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

2n - 1 = 24 1 = 16 1 = 15

15

2n - 1 = 24 1 = 16 1 = 15

15

15

2n - 1 = 24 1 = 16 1 = 15

22nnThe Curse of

DimensionalityBellman [1957]

What’s next?

• A short visit to the tutOR site.• Hands-on with the Knapsack

Problem:– Conventional Formulation– DP functional equation