Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality

Nonlinear Optimization

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One

Decision Variable Models with Equality Constraints:

Lagrange Multipliers Interpretation of Lagrange Multiplier

Models Involving Inequality Constraints


Review of 1st Derivatives Notation:

y = f(x), dy/dx = f’(x) f(x) = c f’(x) = 0 f(x) = xn f’(x) = n*x(n-1)

f(x) = x f’(x) = 1*x0 = 1 f(x) = x5 f’(x) = 5*x4

f(x) = 1/x3 f(x) = x-3 f’(x) = -3*x4

f(x) = c*g(x) f’(x) = c*g’(x) f(x) = 10*x2 f’(x) = 20*x f(x) = u(x)+v(x) f’(x) = u’(x)+v’(x) f(x) = x2 - 5x f’(x) = 2x - 5


Review of 2nd Derivatives Notation:

y = f(x), d(f’(x))/dx = d2y/dx2 = f’’(x) f(x) = -x2 f’(x) = -2x f’’(x) = -2 f(x) = x-3 f’(x) = -3x-4 f’’(x) = 12x-5







Models with One Decision Variable

Requires 1st & 2nd derivative tests


1st & 2nd Derivative Tests Rule 1 (Necessary Condition):

df/dx = 0 Rule 2 (Sufficient Condition):

d2f/dx2 > 0 Minimum d2f/dx2 < 0 Maximum


Maximum Example Rule 1:

f(x) = y = -50 + 100x – 5x2

dy/dx = 100 – 10x = 0, x = 10 Rule 2:

d2y/dx2 = -10 Therefore, since d2y/dx2 < 0: f(x) has a Maximum at

x=10


Maximum Example – Graph Solution

050

100150200250300350400450500

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19

x

Y =

f(x)


Minimum Example Rule 1:

f(x) = y = x2 – 6x + 9 dy/dx = 2x - 6 = 0, x = 3

Rule 2: d2y/dx2 = 2

Therefore, since d2y/dx2 > 0: f(x) has a Minimum at x=3.


Minimum Example – Graph Solution

0

50

100

150

200

250

300

350

400

1 4 7 10 13 16 19 22 25 28 31 34 37

x

Y =

f(x)

3


Max & Min Example Rule 1:

f(x) = y = x3/3 – x2 dy/dx = f’(x) = x2 – 2x = 0; x = 0, 2

Rule 2: d2y/dx2 = f’’(x) = 2x – 2 = 0 2(0) – 2 = -2, f’’(x=0) = -2

Therefore, d2y/dx2 < 0: Maximum of f(x) at x=0

2(2) – 2 = 2, f’’(x=2) = 2 Therefore, d2y/dx2 > 0: Minimum of f(x) at x=2


Max & Min Example – Graph Solution

-8

-6

-4

-2

0

2

4

6

1 4 7 10 13 16 19 22 25 28 31 34 37

x

Y =

f(x)

20


Example: Cubic Cost Function Resulting in Quadratic 1st Derivative Rule 1:

f(x) = C = 10x3 – 200x2 – 30x + 15,000 dC/dx = f’(x)= 30x2 – 400x – 30 = 0

Quadratic Form: ax2 + bx + c

07.,4.13

)30(2

)]30)(30(42)^400[(400

2

]42^[

x

x

a

acbbx


Rule 2: d2y/dx2 = f’’(x) = 60x – 400 60(13.4) – 400 = 404 > 0

Therefore, d2y/dx2 > 0: Minimum of f(x) at x = 13.4

60(-.07) – 400 = -404.2 < 0 Therefore, d2y/dx2 < 0: Maximum of f(x) at x = -.07


05000

100001500020000250003000035000400004500050000

1 4 7 10 13 16 19 22 25 28 31 34 37

x = Units Produced

Cos

t $ (C

) = f(

x)Cubic Cost Function – Graph Solution

-.07 13.4


Economic Order Quantity – EOQ Assumptions:

Demand for a particular item is known and constant Reorder time (time from when the order is placed until the

shipment arrives) is also known The order is filled all at once, i.e. when the shipment

arrives, it arrives all at once and in the quantity requested Annual cost of carrying the item in inventory is

proportional to the value of the items in inventory Ordering cost is fixed and constant, regardless of the size

of the order


Economic Order Quantity – EOQ Variable Definitions:

Let Q represent the optimal order quantity, or the EOQ Ch represent the annual carrying (or holding) cost per

unit of inventory Co represent the fixed ordering costs per order D represent the number of units demanded annually


Economic Order Quantity – EOQ Note: If all the previous assumptions are

satisfied, then the number of units in inventory would follow the pattern in the graph below:

EOQ Model

Q

Time


Economic Order Quantity – EOQ At time = 0 after the initial delivery, the

inventory level would be Q. The inventory level would then decline, following the straight line since demand is constant. When the inventory just reaches zero, the next delivery would occur (since delivery time is known and constant) and the inventory would instantaneously return to Q. This pattern would repeat throughout the year.


Economic Order Quantity – EOQ Under these assumptions:

Average Inventory Level = Q/2 Annual Carrying (or Holding) Cost = (Q/2)*Ch

The annual ordering cost would be the number of orders times the ordering cost: (D/Q)* Co

Total Annual Cost = TC = (Q/2)*Ch+(D/Q)* Co


Economic Order Quantity – EOQ To find the Optimal Order Quantity, Q take the first

derivative of TC with respect to Q: (dTC/dQ) = (Ch/2) – DCoQ-2 = 0

Solving this for Q, we find: Q* = (2DCo/Ch)^(1/2)

Which is the Optimal Order Quaintly

Checking the second-order conditions (Rule 2 in our text), we have: (d2TC/dQ2)= (2DCo/Q3)

Which is always > 0, since all the quantities in the expression are positive. Therefore, Q* gives a minimum value for total cost (TC)


Restricted Interval Problems Step 1:

Find all the points that satisfy rules 1 & 2. These are candidates for yielding the optimal solution to the problem.

Step 2: If the optimal solution is restricted to a specified interval,

evaluate the function at the end points of the interval.

Step 3: Compare the values of the function at all the points found

in steps 1 and 2. The largest of these is the global maximum solution; the smallest is the global minimum solution.







Unconstrained Models with More Than One Decision Variable

Requires partial derivatives


Example Partial Derivatives If z = 3x2y3

∂z/∂x = 6xy3

∂z/∂y = 9y2x2

If z = 5x3 – 3x2y2 + 7y5

∂z/∂x = 15x2 – 6xy2

∂z/∂y = -6x2y + 35y4


2nd Partial Derivatives 2nd Partials

(∂/∂x) (∂z/∂x) = ∂2z/∂x2

(∂/dy) (∂z/∂y) = ∂2z/∂y2

Mixed Partials (∂/∂x) (∂z/∂y) = ∂2z/(∂x∂y) (∂/∂y) (∂z/∂x) = ∂2z/(∂y∂x)


Example 2nd Partial Derivatives If z = 7x3 + 9xy2 + 2y5

∂z/∂x = 21x2 + 9y2

∂z/∂y = 18xy + 10y4

∂2z/(∂y∂x) = 18y ∂2z/(∂x∂y) = 18y ∂2z/∂x2 = 42x ∂2z/∂y2 = 40y3


Partial Derivative Tests Rule 3 (Necessary Condition):

∂f/∂x1 = 0, ∂f/∂x2 = 0, Solve Simultaneously

Rule 4 (Sufficient Condition): If ∂2f/∂x1

2 > 0

And (∂2f/∂x12)*(∂2f/∂x2

2) – (∂2f/(∂x1∂x2))2 > 0 Then Minimum

If ∂2f/∂x12 < 0

And (∂2f/∂x12)*(∂2f/∂x2

2) – (∂2f/(∂x1∂x2))2 > 0 Then Maximum


Partial Derivative Tests Rule 4, continued:

If (∂2f/∂x12)*(∂2f/∂x2

2) – (∂2f/(∂x1∂x2))2 < 0 Then Saddle Point

If (∂2f/∂x12)*(∂2f/∂x2

2) – (∂2f/(∂x1∂x2))2 = 0 Then no conclusion


Partial Derivative Tests Rule 5 (Necessary Condition):

All n partial derivatives of an unconstrained function of n variables, f(x1, x2, …, xn), must equal zero at any local maximum or any local minimum point.







Lagrange Multipliers Nonlinear Optimization with an equality

constraint Max or Min f(x1, x2)

ST: g(x1, x2) = b

Form the Lagrangian Function: L = f(x1, x2) + λ[g(x1, x2) – b]


Lagrange Multipliers Rule 6 (Necessary Condition):

Optimization of an equality constrained function, 1st order conditions:

∂L/∂x1 = 0

∂L/∂x2 = 0

∂L/∂λ = 0


Lagrange Multipliers Rule 7 (Sufficient Condition):

If rule 6 is satisfied at a point (x*1, x*

2, λ*) apply conditions (a) and (b) of rule 4 to the Lagrangian function with λ fixed at a value of λ* to determine if the point (x*

1, x*2) is a local maximum or a local

minimum.


Lagrange Multipliers Rule 8 (Necessary Condition):

For the function of n variables, f(x1, x2, …, xn), subject to m constraints to have a local maximum or a local minimum at a point, the partial derivatives of the Langrangian function with respect to x1, x2, …, xn and λ1, λ2, …, λm must all equal zero at that point.


Interpretation of Lagrange Multipliers

The value of the Lagrange multiplier associated with the general model above is the negative of the rate of change of the objective function with respect to a change in b. More formally, it is negative of the partial derivative of f(x1, x2) with respect to b; that is, λ = - ∂f/∂b or ∂f/∂b = - λ








Step 1: Assume the constraint is not binding, and apply the procedures of

“Unconstrained Models with More Than One Decision Variable” to find the global maximum of the function, if it exists. (Functions that go to infinity do not have a global maximum). If this global maximum satisfies the constraint, stop. This is the global maximum for the inequality-constrained problem. If not, the constraint may be binding at the optimum. Record the value of any local maximum that satisfies the inequality constraint, and go on to Step 2.

Step 2: Assume the constraint is binding, and apply the procedures of “Models

with Equality Constraints” to find all the local maxima of the resulting equality-constrained problem. Compare these values with any feasible local maxima found in Step 1. The largest of these is the global maximum.

Documents

Nonlinear Optimization Review of Derivatives Models with One Decision Variable Unconstrained Models with More Than One Decision Variable Models with Equality