Optimization TechniquesOptimization Techniques

Optimal Problem Formulation

OptimizationOptimization

The idea behind “optimization” is to find the “best” solution from a domain of “possible” solutions.

Optimization methods provide mathematical tools that allow the search for this “best” solution to carried out in a rational and efficient way.

Before these tools can be applied the design problem needs to be recast in an appropriate form.

Conventional Optimization

• Translate a descriptive statement of the design problem into a mathematical statement for optimization using a five-step process

• Identify and define the problem’s design variables

• Identify a function for the problem that needs to be optimized

• Identify and define the problem’s constraints

Optimization in Design

Need Identified

Choose Design Variables

Formulate Constraints

Set up Variable Bounds

Choose an Optimization Algo

Obtain solutions

Formulate Objective Function

OptimizationOptimizationSet of all “workable”or “functional designs”x2 H2 : x 2 > c2

H1 : X1> c1

Set of all “acceptable”designs.

x1

*

Optimal DesignU(x1,x2) = Umax

Steps for optimization

Step 1: Project/problem statement.Step 2: Data and information collection.Step 3: Identification/definition of design

variables.Step 4: Identification of a criterion to be Step 4: Identification of a criterion to be

optimized.Step 5: Identification of constraints.

Step 1: Project/Problem Statement

The formulation process begins by developing a descriptive statement for theproject/problem, which is usually done by the project’s owner/sponsor. The statementdescribes the overall objectives of the project and the describes the overall objectives of the project and the requirements to be met.

Step 2: Data and Information Collection

Is all the information available to solve the problem?

To develop a mathematical formulation of the problem, we need to gather material problem, we need to gather material properties, performance requirements, resource limits, cost of raw materials, and other relevant information. In addition, most problems require the capability to analyze trial designs. Therefore, analysis procedures and analysis tools must be identified at this stage.

Step 3: Identification/Definition of Design Variables

• Identify a set of variables that describe the system, called design variables.

• They are referred to as optimization variables and are regarded as free because we should be able to assign any value to them. Different values for the variables produce different designs.

• The design variables should be independent of each other as far as possible. If they are dependent, then their values cannot be specified independently.

• The number of independent design variables specifies the design degrees of freedom for the problem.

For some problems, different sets of variables can be identified to describe the same system. The problem formulation will depend on the selected set. Once the design variables are given numerical values, we have a design of the system.

Step 3: Identification/Definition of Design Variables

• Design variables should be independent of each other as far as possible. If they are not, then there must be some equality constraints between them. Conversely, if there are equality constraints in the problem, then the design variables are dependent.

• A minimum number of design variables required to formulate a design optimization problem properly exists.formulate a design optimization problem properly exists.

• As many independent parameters as possible should be designated as design variables at the problem formulation phase. Later on, some of the variables can be assigned fixed values.

• A numerical value should be given to each variable once design variables have been defined to determine if a trial design of the system is specified.

Step 3: Identification/Definition of Design Variables

• Identify a set of variables that describe the system, called design variables.

• They are referred to as optimization variables and are regarded as free because we should be able to assign any value to them. Different values for the variables produce different designs.

• The design variables should be independent of each other as far as possible. If they are dependent, then their values cannot be specified independently.

• The number of independent design variables specifies the design degrees of freedom for the problem.

For some problems, different sets of variables can be identified to describe the same system. The problem formulation will depend on the selected set. Once the design variables are given numerical values, we have a design of the system.

Step 4: Identification of a Criterion to Be Optimized

How do I know that my design is the best?There can be many feasible designs for a system, and some are

better than others. To compare different designs, we must have a criterion. The criterion must be a scalar function whose numerical value can be obtained once a design is specified, i.e., it must be a function of the design variable vector x.

Such a criterion is usually called an objective function for the Such a criterion is usually called an objective function for the optimum design problem, which needs to be maximized or minimized depending on problem requirements.

A criterion that is to be minimized is usually called the cost function.

It is emphasized that a valid objective function must be influenced directly or indirectly by the variables of the design problem; otherwise, it is not a meaningful objective function. Note that an optimized design has the best value for the objective function.

Step 5: Identification of ConstraintsAll restrictions placed on a design are collectively called

constraints. The final step in the formulation process is to identify all constraints and develop expressions for them. Most realistic systems must be designed and fabricated within given resources and performance requirements.

• Linear and Nonlinear Constraints

• Feasible DesignA design meeting all requirements is called a feasible design

(acceptable or workable). An infeasible design (unacceptable) does not meet one or more of the requirements.

• Equality and Inequality Constraints

• Implicit ConstraintsWhen there are implicit functions in the problem formulation, it is

not possible to formulate the problem functions explicitly in terms of design variables alone. Instead, we must use some intermediate variables in the problem formulation.

Important definitions

Objective Function: � It represents the quantity (U) which is to be optimized (the

“objective”) as a function of one or more independent variables (x1, x2, x3…)

� The best form to put the objective function in depends on the optimization technique to be employed.optimization technique to be employed.

U = U( x1 , x2 , x3…) → Uopt

Important definitionsDesign Variables: The independent variables (x1, x2, x3…) that the objective

function depends on.� It is generally best to minimize the number of design variables

… the more variables the tougher the optimization will be.

Important definitions

Constraints:

� Relations which limit the possible (physical limitations) or the permissible (external constraints) solutions to the objective function.

� Constraints come in two mathematical “flavors.”

Equality Constraints: Often come from fundamental physics Equality Constraints: Often come from fundamental physics considerations (e.g. cons. of mass)

Inequality Constraints: Often from safety, cost, space, material strength limits etc.

� Generally Equality Constraints are easier to deal with than inequality constraints.

� it is desirable to reduce the number of constraints.

Distinction between Equality and In equality Constraints

Mathematical Formulation

Objective Function of n independent design variables:For U( x1, x2, x3…xn); Find Uopt

Equality Constraints:Gi( x1, x2, x3…)=0 ; i=1,2,…,m

Inequality Constraints:H (x , x , x …) < or > C ; j=1, 2 ,…, l. Hj(x1, x2, x3…) < or > Cj ; j=1, 2 ,…, l.

� If n > m → An Optimization problem results;

� If n = m → A unique solution exists…just solve all equations

simultaneously;

� If n < m → The problem is “overconstrained” no solution

which satisfies all of the constraints is possible.

Classification of Optimization Techniques

Calculus based Techniques� Lagrange Multipliers

Search Methods� Elimination Methods

Exhaustive

FibonacciFibonacci

golden section search

� “Hill Climbing” techniquesLattice Search

Steepest ascent

“Programming” methods� Linear Programming

� Geometric Programming

Search- Exhaustive

x2

H2 : X2> c2

H1 : X1> c1

x1

*

Optimal DesignU(x1,x2) = Umax

Note: None of the searchpoints exactly hits the optimum. The spacebetween search points isknown as the “interval of uncertainty”

* *

Search Lattice

1

3

* *

2

4

“Programming” MethodsThese methods have nothing to do with “Programming” in the sense that

you usually think of it!Linear Programming: Very powerful, but very limited!

� Applies only when the objective function and all constraints can be expressed as Linear Functions … Not in thermal/fluid systems

• Quadratic Programming (QP) ProblemsIf the objective function is a quadratic function and all constraint functions are linear

functions ofoptimization variables/ the problem is called a quadratic programmingfunctions ofoptimization variables/ the problem is called a quadratic programming

problem.Dynamic Programming: Related to optimizing a “process”

� Lets you find the “best” order to do steps in� useful in Project Management and Assembly optimization

Geometric Programming: Similar to Linear Programming, but now all functions must be polynomials.� Reduces the restrictions on linear programming quite a bit� Very useful where empirical correlations for system behavior are known� More difficult and computationally intensive than Linear Prog.

Functions

Optimization deals with functions. A function is simply a mapping from one space to another. (that is, a set of instructions describing how to get from one location to another)

BAf →: This is the range

This is the domain This is a function

This is the domain

A Bf

For any

( )xfy =

By∈andAx∈

Note: A function maps each value of x to one and only one value for y

x y

Examples of Single variable Function

For 53 += xy50 ≤≤x

]20,5[=B

( )xfDomain

205 ≤≤ y

Range

Ax∈

[ ]5,0=A ]20,5[=B

3

14

( ) 143 =f

y

53 += xyFor 50 ≤≤ x

20

Ran

ge

Y =14

Examples of Single variable Function

A straight line is a function.

x

0 5Domain

5

X =3

y { }53max +x

50 ≤≤ x20

Ran

ge

Optimization of a function

It involves finding the maximum value for y over an allowable range.

x0 5

Domain

5

Ran

ge

Here, the optimum occurs at x = 5 (y = 20)

− x5

1max

100 ≤≤ x

What is the solution to this optimization problem?

y

x105

There is no optimum because f(x) is discontinuous at x = 5

y

{ }x2max60 <≤ x

12

There is no optimum

What is the solution to this optimization problem?

x0 6

There is no optimum because the domain is open (that is, the maximum occurs at x = 6, but x = 6 is NOT in the domain!)

y

{ }xmax 0≥x12

What is the solution to this optimization problem?

x0

There is no optimum because the domain is unbounded (x is allowed to become arbitrarily large)

One-Variable case

The Weierstrass Theorem provides sufficient conditions for an optimum to exist, the conditions are shown in next slides

y

Here suppose f(x) is continuous over the domain of x.

{ })(max xf

Sufficient conditions guarantee a solution, but are not required

xThe domain for f(x) is closedand bounded.

x

{ })(max xfNecessary conditions are required for a solution to exist

Derivatives

( )

∆−∆+==

→∆ x

xfxxf

dx

dfxf

x

)()(lim'

0

Formally, the derivative of a function is defined a s

All you need to remember is the derivative represen ts a slope.

y

x

( )xf

y

0

( )xxf ∆+

xx ∆+x

)()( xfxxf −∆+

x∆

∆−∆+

x

xfxxf )()(Slope =

∆−∆+

x

xfxxf )()(

x

2)( xxf =

y 2x

Example:

∆−∆+=

x

xxx 22)(

x2

2x

x0

∆x

xxx

xxx

x

xxxxx ∆+=

∆∆+∆=

∆−∆+∆+= 2

22 2222

xxxx

22lim0

=∆+→∆

x

Unconstrained maximization

Strictly speaking, no problem is truly unconstrained. However, sometimes the constraints don’t influence the maximum.

First Order Necessary Conditions

If x* is a solution to the optimization problem:

0*)(' =xf

If x* is a solution to the optimization problem:

Min{ f(x) } or Max{ f(x) } then

How can we be sure we are at a minimum?

Secondary Order Necessary Conditions

If *x

{ })(max xfx

is a solution to the maximization problem:

then 0*)('' <xf 0*)('' <xf

If *x

{ })(min xfx

is a solution to the minimization problem:

then 0*)('' >xf

y

*x *x

y

x

The second derivative is the rate of change of the first derivative

x*x *x

0*)(' =xf 0*)(' =xf

0*)('' <xf 0*)(" >xf

Slope is increasingSlope is decreasing

Example1

The purpose of this project is to design a can to hold at least 400 ml of liquid, as well as to meet other design requirements (1 ml = 1cm3). The cans will be produced in the billions so it is desirable to minimize manufacturing costs. Since cost can be directly related to the surface area of the sheet metal, it is reasonable to minimize the amount of sheet metal required to fabricate the can. amount of sheet metal required to fabricate the can. Fabrication, handling, aesthetics, and shipping considerations impose the following restrictions on the size of the can:

The diameter should be no more than 8 cm and no less than 3.5 cm, whereas the height should be no more than 18 cm and no less than 8 cm.

Example 2

The goal of this project is to choose insulation thickness t to minimize the life-cycle cooling cost for a spherical tank. The cooling costs include the cost of installing and running the refrigeration equipment, and the cost of installing the insulation. Assume a 10-year life, installing the insulation. Assume a 10-year life, 10 percent annual interest rate, and no salvage value. The tank has already been designed having r (m) as its radius.

Quiz

Suppose that a company owns a corporate jet. The annual expenses are as follows:

� You pay your flight crew (pilot, co-pilot, and navigator a combined annual salary of $500,000.

� Annual insurance costs on the jet are $250,000

� Fuel/Supplies cost $1,500 per flight hour� Fuel/Supplies cost $1,500 per flight hour

� Per hour maintenance costs on the jet are proportional to the number of hours flown per year.

� Maintenance costs (per flight hour) = 1.5(Annual Flight Hours)

If you like to minimize the hourly cost of your jet, how many hours should company use it per year?

Quiz Sol

Let x = Number of Flight Hours

++

>x

xx5.1$1500$

000,750$min

0

xx

CostHourly 5.1$1500$000,750$

++=

> xx 0

First Order Necessary Conditions

hrsxx

7075.1

000,75005.1$

000,750$2

==⇒=+−

Quiz Sol

Hou

rly C

ost (

$)H

ourly

Cos

t ($)

Annual Flight Hours

Quiz Sol

Let x = Number of Flight Hours

++ x

xx5.1$1500$

000,750$min

First Order Necessary Conditions

−hrsx

x707

5.1

000,75005.1$

000,750$2

==⇒=+−

Second Order Necessary Conditions

0000,500,1$

3>

xFor X>0