Hae-Jin ChoiSchool of Mechanical Engineering,

Chung-Ang University

8. Numerical Approaches for Solving Optimization Problems

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-1-

If an optimization problem involves the objective/constraint functions that are not stated as explicit functions or too complicated to manipulate, we cannot solve it with the analytical approaches learned in the previous lectures.

Numerical approaches for optimization problems can be analogous to the numerical techniques, such as Lunge-Kutta method and Simpson rule, for mathematical solutions of differentiation and integration.

Numerical approaches are classified into several categories depending on the types of optimization problems

MATLAB toolbox, called ‘optimization toolbox’ is a useful tool for practical use of optimization techniques in various engineering optimization problems.

Introduction to Numerical Approach

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-2-

MATLAB Functions for Solving Opt. Prob.

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-3-

MATLAB GUI Tool for Optimization

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-4-

Elimination methods Unrestricted search Exhaustive search Dichotomous search Fibonacci method Golden section method

Interpolation method Quadratic interpolation method Cubic interpolation method Direct root methods

Techniques for One-variable Opt. Prob.

Golden section method

Quadratic interpolation method

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-5-

Direct search methods Random search method Grid search method Univariate method Pattern search methods

Decent methods Steepest Descent (Cauchy) method Newton’s method Quasi-Newton method

Techniques for Unconstrained Opt. Prob.

Grid search method

Steepest descent method

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-6-

Techniques for solving linear constrained problems.

Linear Programming (Linear objective function) Simplex Method Revised Simplex Method Dual Simplex Method

Quadratic Programming (Quadratic objective function) Kuhn-Tucker necessary condition +Simplex

Method

Techniques for Linear Constrained Prob.

Simplex method

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-7-

Direct methods Random search methods Sequential Linear Programming (SLP) Sequential Quadratic Programming (SQP) Most recent and popular method for constrained optimization problem

Convert original problem into stepwise Quadratic Programming

Linearization of the constraints

Solve QP for search direction and find new evaluation point

Iteratively process the above steps until converge to optimum point.

Indirect methods Interior penalty function method Exterior penalty function method

Techniques for Nonlinear Constrained Prob.

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-8-

Modern Methods – Genetic Algorithm (GA)

In a genetic algorithm, a population of which encode candidate solutions to an optimization problem, evolves toward better solutions. Traditionally, design variables are represented in binary as strings of 0s and 1s.

The evolution usually starts from a population of randomly generated individuals

In each generation, the fitness of every individual in the population is evaluated, multiple individuals are stochastically selected from the current population and modified (recombined and possibly randomly mutated) to form a new population. The new population is then used in the next iteration of the algorithm.

The algorithm terminates when either a maximum number of generations has been produced, or a satisfactory fitness level has been reached for the population. If the algorithm has terminated due to a maximum number of generations, a satisfactory solution may or may not have been reached.

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-9-

Modern Methods – Simulated Annealing (SA) The name and inspiration come from annealing in

metallurgy. The heat causes the atoms to become unstuck from their initial positions (a local minimum of the internal energy) and wander randomly through states of higher energy; the slow cooling gives them more chances of finding configurations with lower internal energy than the initial one.

By analogy with this physical process, each step of the SA algorithm attempts to replace the current solution by a new random solution near the current solution. The new solution may then be accepted with a probability of P = exp(-∆E/kT).

The choice between the previous and current solution is almost random when T is large, but increasingly selects the better or "downhill" solution (for a minimization problem) as T goes to zero. The allowance for "uphill" moves potentially saves the method from becoming stuck at local optima.

The algorithm is terminated when new solutions are repeatedly rejected with the higher temperature setting.

SCHOOL OF MECHANICAL ENG.CHUNG-ANG UNIVERSITY-10-

Topology optimization example A mathematical approach that optimizes material layout within a given

design space, for a given set of loads and boundary conditions such that the resulting layout meets a prescribed set of performance targets.

http://www.youtube.com/user/OSDELab#p/u/5/bvf6Dm_HF4A

Example of Numerical Optimization