Customized sequential designs for random simulation experiments: Kriging metamodeling and bootstrapping

Abstract

This paper proposes a novel method to select an experimental design for interpolation in random simulation, especially discrete event simulation. (Though the paper focuses on Kriging, this design approach may also apply to other types of metamodels such as linear regression models.) Assuming that simulation requires much computer time, it is important to select a design with a small number of observations (or simulation runs). The proposed method is therefore sequential. Its novelty is that it accounts for the specific input/output behavior (or response function) of the particular simulation at hand; i.e., the method is customized or application-driven. A tool for this customization is bootstrapping, which enables the estimation of the variances of predictions for inputs not yet simulated. The new method is tested through two classic simulation models: example 1 estimates the expected steady-state waiting time of the M/M/1 queueing model; example 2 estimates the mean costs of a terminating (s, S) inventory simulation. For these simulations the novel design indeed gives better results than Latin Hypercube Sampling (LHS) with a prefixed sample of the same size.

Kriging Metamodeling in Simulation: A Review

“…The basics of bootstrapping are explained in [5] and [9]. To estimate the predictor variance in Kriging, [26] resamples-with replacement-the m i Independent and Identically Distributed (IID) observations. This sampling results in the bootstrapped average w i where the superscript is the usual symbol to denote a bootstrapped observation and i = 1; : : : ; n. From these n bootstrapped averages w i , the bootstrapped estimated optimal weights c 0 and the corresponding bootstrapped Kriging predictor y are computed.…”

“…In [11] and [26], Van Beers and I develop a sequential procedure for deterministic and random simulations respectively. These two procedures share the following steps.…”

Metamodel-Based Sensitivity Analysis: Polynomial Chaos Expansions and Gaussian Processes

“…More efficient criteria can be found in Bates et al [4]; van Beers and Kleijnen [62]; Le Gratiet and Cannamela [33].…”

Simulation Experiments in Practice: Statistical Design and Regression Analysis

“…Sequential procedures are proposed in (Kleijnen and Van Beers, 2004) and (Van Beers and Kleijnen, 2006). These two publications select the next factor combination to be simulated, where the simulation model may be either random or deterministicassuming the simulation I/O data are analysed through Kriging (instead of linear regression), which allows the simulation outputs to be correlated.…”