Approximation Techniques for Simulation Optimizat

Approximation Techniques for Simulation-Optimization Frameworks

Vikram Ganesan Design for Six Sigma Master Black Belt

DFSS Lead Wheeled & Tracked Systems Engineering Business Planning & Resource Management,

General Dynamics Land Systems, 38500 Mound Road, Sterling Heights, Michigan 48310, USA.

Email: [email protected]

Engineering analyses often involve modeling system performance via techniques such as finite element methods and computational fluid dynamics, both of which entail heavy computational requirements. As a result, high fidelity analyses can become computationally prohibitive, thereby limiting optimization and design space explorations. Consequently, statistical approximation procedures are becoming increasingly popular for constructing simplified surrogate approximations or metamodels of these analytical codes. In both deterministic and probabilistic design optimization strategies, such approximation methods are crucial for modeling uncertainty and reducing the computational expense of probabilistic analyses. A variety of probabilistic methods have been developed to model and assess the effects of uncertainties by converting deterministic problem formulations into probabilistic formulations [1]. Recent advances in computational methods have promoted the application of such techniques to real multidisciplinary engineering design problems.

Response Surface Methodology. The Response Surface Methodology (RSM) is a very popular choice for constructing metamodels, especially in the aerospace industry and in structural design [2]. This method typically employs second-order polynomial models that are fit using least-square regression techniques. In general, a polynomial response surface model of order d, in n variables, is of the form

.x...xc....xxcxccyd

d1d12

2121

1j

nj...j1 1jj...jj

njj1 1jjj

nj1 1j1j0 ++++= (1)

Furthermore, for a function of n variables, a polynomial approximation of order d will have

+

++

= ddn

t1tn

1d

1t (2)

terms, including a constant value. The RSM framework offers the following benefits: 1. It smoothes the numerical noise present in the analysis. This noise can distort gradient

information and lead to artificial local minima in the design space when using an optimization approach.

2. It also allows the functional analysis code to be separated from the optimization routines. This permits flexibility in applying parallel computing technologies.

Approved for Public Release, Distribution Unlimited, GDLS Log # 2008-05

3. By replacing complex functional relationships with simple polynomials, the designer can more readily obtain information on design trade-offs, sensitivities to certain variables, and insights into highly constrained, nonconvex design spaces.

However, RSM models also have some drawbacks: 1. Since response surfaces are typically second-order polynomial models, they have limited

capability to accurately model nonlinear functions of arbitrary shape. Though higher-order models can be used to approximate nonlinear design spaces, instabilities may arise or may require a very large number of points to estimate all the coefficients in a polynomial equation.

2. Although sequential RSM approaches using move-limits or trust-region approaches somewhat help mitigate the above limitations, in multi-objective optimization problems, it is often difficult to isolate a small region that is accurately representable by a low-order polynomial model. The response region of interest will never be reduced to a sufficiently small neighborhood that permits a good approximation for all objectives in a multi-objective optimization problem [3]. Furthermore, difficulties arise in screening multi-objective problems involving a large number of variables [4].

3. Often-times, it is up to the designer to use an appropriate Design of Experiments (DOE) technique along with RSM concepts in a structured approach toward deriving good model representations.

Kriging Models. Kriging models show great promise for building accurate global approximations over potentially large regions of interest. Unlike RSM models, Kriging models have their origins in mining and geostatistical applications involving spatially and temporally correlated data. They are extremely flexible because of the wide range of spatial correlation functions that can be chosen to build the approximations, provided that sufficient sample data are available to capture the trends in the system response. As a result, Kriging models can approximate linear and nonlinear functions equally well. Furthermore, Kriging models can either honor the data by providing an exact interpolation of the data, or smooth the data by providing an inexact interpolation [5], [6]. A 56 variable helicopter-rotor structural design problem solution demonstrates how the flexibility of Kriging models permits such representations to be improved iteratively in regions of interest through an intelligent intervention of a design expert [7]. The limited use of Kriging models in engineering applications may be attributed to the lack of readily available software to fit Kriging models, or the additional effort involved in using a Kriging model as compared to a simple RSM model. Kriging models combine a global model plus localized departures via the functional form: ( ) ( ) ( )g x h x x= + (3) where,

g(x) is the unknown function of interest; h(x) is an approximation (usually polynomial) function, and (x) is the realization of a stochastic process with mean zero, variance 2 , and

nonzero covariance.


The term h(x) provides a base model for the design space, and is similar to a polynomial response surface. In many cases, however, h(x) is taken as simply a constant value. (x) creates localized deviations so that the Kriging model interpolates the sample data points; however, noninterpolative Kriging models can also be created to smooth noisy data. The covariance matrix of (x) is given by

sn

Cov [(xi), (x j)] = 2 [R(xi, x j)] (4) where, R(xi, xj) is the correlation function between any two of the sample data points xi and xj. Let R be the correlation matrix having components R(xi, xj). Note that R is an (

sn

sn sn ) symmetric matrix with ones along the diagonal. The correlation function R(xi, xj) is specified by the user and a variety of such functions exist. Often-times, a Gaussian correlation function of the following form is used.

2

1( , ) exp

n ji j ik k k

kR x x x x

= = (5)

where,

n is the number of design variables; k , k=1,,n, are the unknown correlation parameters used to fit the model, and ikx and jkx are the kth components of the sample points ix and jx , respectively.

In some cases, using a single correlation parameter gives sufficiently good results; however, in our approach, we use a different k for each design variable , k=1,,n. The predicted estimates, , of the response g(x) at untried values of x are then given by

kx)(~ xg

1( ) ( ) ( )Tg x r x R g h = + %% % (6) where,

g is the column vector of length sn that contains the sample values of the response; h is a column vector of length sn that is filled with ones (this assumes that h(x) is

taken as a constant); )(xrT is the transpose of the correlation vector, of length sn between an untried x

and the sampled data points { }snx given by x ,...,1

1 2( ) [ ( , ), ( , ),..., ( , )]nT sr x R x x R x x R x x= ; (7)

~ is estimated as ( ) gRhhRh TT 111~ = . (8)


The estimate of the variance, , between the underlying base model 2~ ~ and g is estimated using the following equation: 2 1[( ) ( )] /T sg h R g h n = % %% . (9) The maximum likelihood estimates for the k -parameters in (5) that are used to fit this Kriging model are obtained by solving the following problem: Maximize 2{ [ ( ) ] / 2 : 0}sn ln ln R +% , (10) where = (1,,n), and where both and 2~ R (the norm of the matrix R), are functions of . While any value of creates an interpolative Kriging model, the ideal Kriging model is found by solving the nonlinear optimization problem given by (10). Conclusions and Recommendations. In general, we tend to rely more on the RSM/ DOE technique to build metamodels, mainly due to the availability of these techniques in several commonly employed software packages. This approach is appropriate, and adequate for most scenarios that seek computational effectiveness; provided the model accounts for the statistical significance of the terms. In problems where a very close approximation of the data is desired, such as fatigue analyses, and test data models that involve significant variation in short time periods; the exact interpolation offered by Kriging models is more suitable. A sufficiently large data set for validating any surrogate model, before its use in optimization routines is always a worthwhile effort.


References 1. Simpson, T. W. Mauery, T. M., Krote, J. J. and Mistree F., Kriging Models for Global

Optimization in Simulation-Based Multidisciplinary Design Optimization, AIAA Journal, Vol. 39, No. 12, pp. 2233-2241, 2001.

2. Myers, R. H., Response Surface Methodology: Process and Product Optimization Using

Designed Experiments, John Wiley & Sons, Inc., New York, N.Y., 1995. 3. Barton, R. R., Metamodels for Simulation Input-Output Relations, Proceedings of the 1992

Winter Simulation Conference, edited by J. J. Swain, D. Goldsman, R. C. Crain, and J. R. Wilson, Institute of Electrical and Electronics Engineers, Arlington, Virginia, pp. 289-299, 1992.

4. Koch, P. N., Simpson T. W., Allen, J. K., and Mistree, F., Statistical Approximations for

Multidisciplinary Optimization: The Problem of Size, Journal of Aircraft (Special Multidisciplinary Design Optimization Issue), Vol. 36, No. 1, pp. 275-286, 1999.

5. Cressie, N. A. C., Statistics for Spatial Data, John Wiley & Sons, New York, N.Y., 1993. 6. Montes, P., Smoothing Noisy Data by Kriging with Nugget Effects, Wavelets, Images, and

Surface Fitting, edited by P.J. Laurent et al., A.K. Peters, Wellesley, Massachusetts, pp. 371-378, 1994.

7. Booker, A. J., Design and Analysis of Computer Experiments, Proceedings of the 7th

AIAA/ USAF/NASA/ISSMO Symposium on Multidisciplinary Analysis and Optimization, Vol. 1, AIAA, Reston, Virginia, pp.118-128, 1998.


Documents

Approximation Techniques for Simulation Optimizat