Multilevel Monte Carlo Metamodeling

Imry RosenbaumJeremy Staum

Outline

• What is simulation metamodeling?• Metamodeling approaches• Why use function approximation?• Multilevel Monte Carlo• MLMC in metamodeling

Simulation Metamodelling

• Simulation – Given input we observe .– Each observation is noisy.– Effort is measured by number of observations, .– We use simulation output to estimate the response surface .

• Simulation Metamodelling– Fast estimate of given any .– “what does the response surface look like?”

Why do we need Metamodeling

• What-if analysis– How things will change for different scenarios .– Applicable in financial, business and military settings.

• For example– Multi-product asset portfolios.– How product mix will change our business profit.

Approaches

• Regression• Interpolation• Kriging– Stochastic Kriging

• Kernel Smoothing

Metamodeling as Function Approximation

• Metamodeling is essentially function approximation under uncertainty.

• Information Based Complexity has answers for such settings.

• One of those answers is Multilevel Monte Carlo.

Multilevel Monte Carlo

• Multilevel Monte Carlo has been suggested as a numerical method for parametric integration.• Later the notion was extended to SDEs.• In our work we extend the multilevel notion to stochastic simulation metamodeling.

Multilevel Monte Carlo

• In 1998 Stefan Heinrich introduced the notion of multilevel MC. • The scheme reduces the computational cost of estimating a family of integrals.• We use the smoothness of the underlying function in order to enhance our estimate of the integral.

Example

• Let us consider and we want to computeFor all .

• We will fix a grid , estimate the respective integrals and interpolate.

Example Continued

We will use piecewise linear approximationWhere are the respective hat functions and are Monte Carlo estimate, i.e, . are iid uniform random variables.

Example Continued

• Let us use the root mean square norm as metric for error• It can be shown that under our assumption of smoothness that at the cost of .

Example Continued

• Let us consider a sequence of grids .

• We could represent our estimator as.

Where, is the estimation using the grid.• We define each one of our decision variables in terms

of M, as to keep a fair comparison.

Example Continued

level 0 LSquare root of

variance

Cost

• The variance reaches its maximum in the first level but the cost reaches its maximum in the last level.

Example Continued

• Let us now use a different number of observations in each level, thus the estimator will be

• We will use to balance between cost and variance.

Example Continued

level 0 LSquare root of

variance

Cost

• It follows that the square root of the variance is while the cost is .

• Previously, same variance at the cost of .

Generalization

• Let and be bounded open sets with Lipschitz boundary.

• We assume the Sobolev embedding condition • .

General Thm

Theorem 1 (Heinrich). Let Then there exist constants such that for each integer there is a choice of parameters such that the cost of computing is bounded by and for each with

Issues

• MLMC requires smoothness to work, but can we guarantee such smoothness?

• Moreover, the more dimensions we have the more smoothness that we will require.

• Is there a setting that will help with alleviating these concerns?

Answer

• The answer to our question came from the derivative estimation setting in Monte Carlo simulation.

• Derivative Estimation is mainly used in finance to estimate the Greeks of financial derivatives.

• Glasserman and Broadie presented a framework under which a pathwise estimator is unbiased.

• This framework will be suitable as well in our case.

Simulation MLMC

• Goal• Framework• Multi Level Monte Carlo Method• Computational Complexity• Algorithm• Results

Goal

• Our goal is to estimate the response surface • The aim is to minimize the total number of

observations used for the estimator. • Effort is relative to amount of precision we require.

Elements We will Need for the MLMC

• Smoothness provided us with the information how adjacent points behave.

• Our assumptions on the function will provide the same information.

• The choice of approximation and grid will allow to preserve this properties in the estimator.

The framework

• First we assume that our simulation output is a Holder continuous function of a random vector ,

• Therefore, there exist and such that for all in

Framework Continued…

• Next we assume that there exist a random variable, with a finite second moment such that

for all a.s.

• Furthermore, we assume that and that it is compact.

Behavior of Adjacnt Points

• bias of estimating using is

• It follows immediately that,

Multi Level Monte Carlo

• Let us assume that we have a sequence of grids with increasing number of points

• The experiment designed are structured such that the maximum distance between a point and point in the experiment design is , denoted by .

• Let denote an approximation of using the same at each design point.

Approximating the Response

MLMC Decomposition

• Let us rewrite the expectation of our approximation in the multilevel way

.

• Let us define the estimator of using m observations, .

MLMC Decomposition Continued

• Next we can write the estimator in the multilevel decomposition,

• Do we really have to use the same for all levels?

The MLMC estimator

• We will denote the MLMC estimator as

• Where

Multilevel Illustration

Δ

Multi Level MC estimators

• Let us denote

• We want to consider approximation of the form of

Approximation Reqierments

• We assume that for each there exist a window size >) which is . Such that for each, we have and for each

Bias and Variance of the Approximation

• Under these assumptions we can show that

• Our measure of error is Mean Integrated Square Error

• Next, we can use a theorem provided by Cliffe et al. to bound the computational complexity of the MLMC.

Computational Complexity Theorem

Theorem. Let denote a simulation response surface and , an estimator of it using replications for each design point. Suppose there exist such that , and

1. The computational cost of is bounded by

Theorem Continued…

Then for every there exist values of and for which the MSE of the MLMC estimator is bounded by with a total computation cost of

Multilevel Monte Carlo Algorithm

• The theoretical results need translation into practical settings.

• Out of simplicity we consider only the Lipschitz continuous setting.

Simplifying Assumptions

• The constants and stated in the theorem are crucial in deciding when to stop. However, in practice they will not be known to us.

• If we can deduce that .

Simplifying Assumptions Continued

• Hence, we can use as a pessimistic estimate of the bias at level . Thus, we will continue adding level until the following criterion is met

• However, due to its inherent variance we would recommend using the following stopping criteria

The algorithm

Black-Scholes

Black-Scholes continued

Conclusion

• Multilevel Monte Carlo provides an efficient metamodeling scheme.

• We eliminated the necessity for increased smoothness when dimension increase.

• Introduced a practical MLMC algorithm for stochastic simulation metamodeling.

Questions?