Optimization-Based Calibration of Simulation Input Models

Abstract

Studies on simulation input uncertainty often built on the availability of input data. In this paper, we investigate an inverse problem where, given only the availability of output data, we nonparametrically calibrate the input models and other related performance measures of interest. We propose an optimization-based framework to compute statistically valid bounds on input quantities. The framework utilizes constraints that connect the statistical information of the real-world outputs with the input-output relation via a simulable map. We analyze the statistical guarantees of this approach from the view of data-driven robust optimization, and show how the guarantees relate to the function complexity of the constraints arising in our framework.We investigate an iterative procedure based on a stochastic quadratic penalty method to approximately solve the resulting optimization. We conduct numerical experiments to demonstrate our performance in bounding the input models and related quantities. Daley D, Servi L (1998) Moment estimation of customer loss rates from transactional data. International Journal of Stochastic Analysis 11(3):301-310. Dang CD, Lan G (2015) Stochastic block mirror descent methods for nonsmooth and stochastic optimization. SIAM Journal on Optimization 25(2):856-881. Delage E, Ye Y (2010) Distributionally robust optimization under moment uncertainty with application to data-driven problems. Operations Research 58(3):595-612. Donoho DL, Johnstone IM, Hoch JC, Stern AS (1992) Maximum entropy and the nearly black object. Journal of the Royal Statistical Society. Series B (Methodological) 41-81. Duchi J, Glynn P, Namkoong H (2016) Statistics of robust optimization: A generalized empirical likelihood approach. arXiv preprint arXiv:1610.03425 . Durrett R (2010) Probability: Theory and Examples (Cambridge university press). Esfahani PM, Kuhn D (2015) Data-driven distributionally robust optimization using the wasserstein metric: Performance guarantees and tractable reformulations. arXiv preprint arXiv:1505.05116 . Fan W, Hong LJ, Zhang X (2013) Robust selection of the best. Gao R, Kleywegt AJ (2016) Distributionally robust stochastic optimization with wasserstein distance. arXiv preprint arXiv:1604.02199 . Ghadimi S, Lan G (2013) Stochastic first-and zeroth-order methods for nonconvex stochastic programming. SIAM Journal on Optimization 23(4):2341-2368. Ghadimi S, Lan G (2015) Accelerated gradient methods for nonconvex nonlinear and stochastic programming. Mathematical Programming 1-41. Ghadimi S, Lan G, Zhang H (2016) Mini-batch stochastic approximation methods for nonconvex stochastic composite optimization. Mathematical Programming 155(1-2):267-305. Ghosh S, Lam H (2015a) Computing worst-case input models in stochastic simulation. Available at http://arxiv.org/pdf/1507.05609v1.pdf . Ghosh S, Lam H (2015b) Mirror descent stochastic approximation for computing worst-case stochastic input models. Proceedings of the 2015 Winter Simulation Conference, 425-436 (IEEE Press). 35 Ghosh S, Lam H (2015c) Robust...

Engineering Approaches for Addressing Opioid Use Disorder in the Community

“…Model calibration can be particularly challenging. Goeva et al (39) developed a nonparametric framework that uses constraints that connect the statistical information and optimizes over a quadratic penalty function. Run times for ABMs can also be significant due to their complexity.…”

“…When RCTs are infeasible or unethical to perform, quasi-experimental studies can be designed to compare outcomes of different groups that are exposed to different interventions and environmental conditions over time. Popular approaches include the use of difference in differences (DID), regression discontinuity, and instrumental variables (42). Perhaps the simplest of these approaches is DID.…”

Distributionally Constrained Stochastic Gradient Estimation Using Noisy Function Evaluations

“…Proof of Theorem 10. The proof is generalized from Goeva et al (2019b) that uses only an unbiased gradient estimator to deal with the bias in our zeroth-order estimator. We analyze the evolution of V (p k , p * ).…”

Calibrating Over-Parametrized Simulation Models: A Framework via Eligibility Set

“…These two steps are reiterated until the model is satisfactory. Though intuitive, this approach is ad hoc, potentially time-consuming and, moreover, there is no guarantee of a satisfactory model at the end (Goeva et al 2019). The ad-hoc-ness arises because just by locating model parameter values that match the simulated versus real outputs in terms of simple hypothesis tests, there is no guarantee that 1) there exists a unique set of parameter values that gives the match and 2) the simulation model is good enough for output dimensions different from the one being tested.…”