Efficient budget allocation strategies for elementary effects method in stochastic simulation

Abstract

This paper focuses on extending the Morris' elementary effects method (MM) for sensitivity analysis/factor screening originated in the context of deterministic computer experiments to the stochastic simulation setting. Given a fixed simulation budget to expend, the main objective is to provide efficient and accurate estimates of main and interaction (or nonlinear) effects coined by the standard MM for characterizing the importance of each factor, despite the impact of simulation errors. Taking into account both the factor/input sampling uncertainty rooted in MM and the random errors inherent in a stochastic simulation, we develop efficient budget allocation strategies for implementing MM in this new context. Under each strategy proposed, we derive its corresponding optimal budget partition and optimal budget allocation rules. Numerical results corroborate the practical effectiveness of the proposed budget allocation strategies.

Cluster sampling for Morris method made easy

“…As explained in Section 3, the cluster-level and within-cluster sampling investigated here is similar in nature to the two-level nested sampling which has been studied in, example, Sun et al (2011) and Shi and Chen (2018). In contrast to only examining an equivalent term of Var(̂2 b ) as is done in Sun et al (2011) and Shi and Chen (2018), we note that the analysis of Var[̂2(N,W)] is more challenging, as the expression in ( 21) also involves the variance of̂2 w (i.e., the terms in the curly brackets on the right-hand side) and the covariance between̂2 b and̂2 w (i.e., the terms in the last four lines on the right-hand side).…”

“…Inspired by Sun et al (2011) and Shi and Chen (2018), we further investigate the asymptotically optimal budget allocation based on (21) when the budget C becomes large. Let us define the normalized variance , where is as given in (21).…”

“…For instance, Boukouvalas et al (2014) proposed a sequential method to enhance the computational efficiency of MM's original sampling design for identifying factors with nonlinear effects on the output; Campolongo et al (2007) and Pujol (2009) respectively proposed algorithms to generate factor combinations that spread out across the experimental region; Campolongo and Braddock (1999) and Cropp and Braddock (2002) proposed a new method to extend the original MM for estimation of the two‐factor interaction effects; and Fédou and Rendas (2015) studied a fast MM‐based mixed effects screening method that enables efficient estimation of the interaction graph of factors. More recently, Shi and Chen (2018) extended MM to the stochastic simulation setting; and Shi and Chen (2019) integrated a distribution‐free sequential multiple testing procedure with MM to control the Type‐I and Type‐II familywise error rates at the prescribed levels.…”

Elementary effects for models with dimensional inputs of arbitrary type and range: Scaling and trajectory generation

“…As far as we are aware, descriptions of the Elementary Effects method (see e.g. [18][19][20][21][22][23][24][25][26]) assume models are dimensionless with inputs taking real values on the unit interval. However, it is commonplace in practice (and in many environmental models) for models to have dimensional outputs, and for inputs to take values on arbitrary intervals or of different types (real, integer or Boolean).…”

Adaptive importance sampling for extreme quantile estimation with stochastic black box computer models

“…In either case, is hidden inside the black box computer model and thus, one cannot sample from its distribution, but can sample X only from p(x). Related types of simulation models also arise in several Emulators such as GP (Ba & Joseph, 2012;Bastos & O'Hagan, 2009;Oakley, 2004;Ranjan et al, 2008;Yang et al, 2007) Importance sampling and other variance reduction techniques (Cannamela et al, 2008;Chu & Nakayama, 2012;Glynn, 1996;Hesterberg, 1995;Kurtz & Song, 2013;Neddermeyer, 2009;Zhang, 1996) Stochastic black box computer model GP with nugget effect, stochatic krigging (Ankenman et al, 2010;Binois et al, 2019;Chen et al, 2012;Wang & Hu, 2015) Stochastic importance sampling (Choe, Byon, & Chen, 2015) other applications (Ankenman, Nelson, & Staum, 2010;Shi & Chen, 2018;Sun, Apley, & Staum, 2011).…”