Controlling Sources of Inaccuracy in Stochastic Kriging

Abstract

Scientists and engineers commonly use simulation models to study real systems for which actual experimentation is costly, difficult, or impossible. Many simulations are stochastic in the sense that repeated runs with the same input configuration will result in different outputs. For expensive or time-consuming simulations, stochastic kriging (Ankenman et al., 2010) is commonly used to generate predictions for simulation model outputs subject to uncertainty due to both function approximation and stochastic variation. Here, we develop and justify a few guidelines for experimental design, which ensure accuracy of stochastic kriging emulators. We decompose error in stochastic kriging predictions into nominal, numeric, parameter estimation and parameter estimation numeric components and provide means to control each in terms of properties of the underlying experimental design. The design properties implied for each source of error are weakly conflicting and broad principles are proposed. In brief, space-filling properties "small fill distance" and "large separation distance" should balance with replication at distinct input configurations, with number of replications depending on the relative magnitudes of stochastic and process variability. Non-stationarity implies higher input density in more active regions, while regression functions imply a balance with traditional design properties. A few examples are presented to illustrate the results.

Replication or Exploration? Sequential Design for Stochastic Simulation Experiments

“…Processes (i.e., data generating mechanisms) benefiting from a heteroskedastic feature bring out the best in our sequential design schemes, demanding a greater degree of replication in high-noise regions relative to low-noise ones, confirming the intuition that replication becomes more valuable for separating signal from noise as the data get noisier (e.g., Wang and Haaland, 2017). However, the results we provide are just as valid in the homoskedastic setting, albeit with somewhat less flair.…”

“…Foreshadowing these developments, and utilizing the calculations detailed therein, we illustrate here the possibility that x N +1 = argmin x I N +1 (x) is a replicate. The conditions under which replication is advantageous, which we describe shortly in Section 3.1, have to our knowledge only been illustrated empirically (Boukouvalas, 2010), or conceptually (e.g., Wang and Haaland (2017) highlight that replication is more beneficial as the signal-to-noise ratio decreases, via upper bounds on the MSPE), or to bolster technical results (e.g., Plumlee and Tuo (2014) demand a sufficient degree of replication to ensure asymptotic efficiency). Figure 1: Illustration of the effect of noise variance on IMSPE optimization.…”

“…Ankenman et al (2010) demonstrated how replicates could facilitate signal isolation, via stochastic kriging (SK), and that accuracy could be improved without much extra computation by augmenting existing degrees of replication in stages (also see Liu and Staum, 2010;Quan et al, 2013;Mehdad and Kleijnen, 2018). Wang and Haaland (2017) showed that replicates have an important role in characterizing sources of inaccuracy in SK. Boukouvalas et al (2014) demonstrated the value of replication in (Fisher) information metrics, and Plumlee and Tuo (2014) provided asymptotic results favoring replication in quantile regression.…”

Order Priority Evaluation Based on Kriging Model Under Supply Chain Environment

“…Moreover, the proposed model can be easily extended to other areas of supply chain management for making appropriate decision to the managers, such as "Just-In-Time" cross-docking distribution system, average travel time for traffic system. Although Kriging model has been widely used in stochastic simulation [44], [45], sometimes it can"t cope well with noise from stochastic simulation, which motivates the development of stochastic Kriging [46]. How to measure stochastic factors and adopt stochastic Kriging model to determine the order priority is our future interest.…”

Multiresolution Functional ANOVA for Large-Scale, Many-Input Computer Experiments

“…*Note that due to memory limits, in these cases R max = 3 and D max = 3 are considered instead. Wang and Haaland (2018), along with n test = 10, 000 unique predictive locations randomly generated from a uniform distribution on [0, 1] d . Since the choice of tuning parameter λ in (2) can be particularly crucial in stochastic function emulation, we consider AIC, BIC and 10-fold CV as selection criteria.…”