Calibrating a Stochastic, Agent-Based Model Using Quantile-Based Emulation

Abstract

In a number of cases, the Quantile Gaussian Process (QGP) has proven effective in emulating stochastic, univariate computer model output (Plumlee and Tuo, 2014). In this paper, we develop an approach that uses this emulation approach within a Bayesian model calibration framework to calibrate an agent-based model of an epidemic. In addition, this approach is extended to handle the multivariate nature of the model output, which gives a time series of the count of infected individuals. The basic modeling approach is adapted from Higdon et al. (2008), using a basis representation to capture the multivariate model output. The approach is motivated with an example taken from the 2015 Ebola Challenge workshop which simulated an ebola epidemic to evaluate methodology. of realizations can be averaged, preprocessed, or accounted for in the estimation scheme for the emulator. In other cases the distribution of computer model outcomes (at a given input setting) can be described by its mean and variance. One strategy is to model the output distribution as normal, conditional on the mean and variance and use a Gaussian process (GP) to model the mean and variance as a function of the inputs (Henderson et al., 2009;Marrel et al., 2012). Adapting similar strategies to non-Gaussian distributions has also been carried out -see Reich et al. (2012) for an example of modeling the exposure distribution as a mixture of normals, using a formulation that depends on space, time, and individual-level covariates.An alternative is the quantile kriging approach described in Plumlee and Tuo (2014). Here quantiles of the output distribution are estimated from the collection of realizations produced at each input setting. A GP is then fit to the augmented input space consisting of the original inputs combined with the quantile estimates. Once fit to the model output, this results in a GP whose realizations give continuous, univariate distributions at each input setting.In this paper we extend the quantile-kriging emulator to handle functional, stochastic computer model output. We also embed this new emulation approach within a Bayesian computer model calibration framework (Kennedy and O'Hagan, 2001;Higdon et al., 2008) to take on a challenging problem in characterizing epidemic behavior, using an ABM that models interaction and contact at the individual level. This statistical modeling is implemented in the software of GPMSA (Gattiker et al., 2016), taking advantage of thoughtful preprocessing of the epidemic observations and the simulation output. The Ebola Challenge ProblemThe 2015 ebola challenge problem (ebola-challenge.org) was organized by the Research and Policy for Infectious Disease Dynamics (RAPIDD) program at the National Institutes for Health (NIH). Its goal is to motivate the development, assessment, and comparison of different approaches for modeling, predicting, and managing infectious disease epidemics.The challenge involved multiple scenarios for the nation of Liberia, each simulating an epidemic patterned after the Ebola outbrea...

Optimizing spatial allocation of seasonal influenza vaccine under temporal constraints

“…For the gaussian likelihood (see S1 Appendix. Additional methods on Calibration methodology), we defined independent errors with a standard deviation of 20% around the calibration criteria [30]. While varying the standard deviation may affect the posterior distribution, it is to be noted that the MAP estimator remains unaffected.…”

“…We begin with the assumption that the ground truth of interest y is a noisy version of the simulation model η (⋅) at some unknown input parameter configuration . We use a gaussian error model, which are simple and adopted widely for many applications, including epidemics [30, 31]. We adopt importance sampling [32] scheme to produce posterior realizations of the calibration parameters.…”

Scalable Epidemiological Workflows to Support COVID-19 Planning and Response

“…An emulator is a statistical model that maps the input to the output of the simulation; it is cheap to run, and offers a way to quantify uncertainty for a deterministic system. To calibrate EpiHiper, a Gaussian Process [43], [46] emulator is used inside a Bayesian calibration framework for multivariate output [18], [29] to produce a set of plausible parameter configurations conditioned on the ground truth and associated uncertainty on the future predictions. The calibration task is carried out using the GPMSA framework [23] in Matlab.…”

“…Gaussianity assumption for the observation error E along with prior specifications complete the posterior of θ . Detailed derivation of the likelihood and posterior can be found in [18]. This posterior is explored via MCMC using the GPMSA [23] tool in Matlab. Metapopulation Model Calibration For each county c in a state, we model the observed time series of reported case counts as noisy realization from the underlying metapopulation model η ( θ ), with additive Gaussian noise.…”

“…The noise standard deviation is assumed to be 20% of the daily case counts. Assuming independence between counties, the joint likelihood can be written as the product of C multivariate gaussian pdfs: where, C is the total number of counties in the state, η ( c ) ( θ ) represents the simulation output for county c at parameter setting θ , and Σ ( c ) contains the error covariances for the same county [17]. Unlike Agent-Based Models, the metapopulation model is cheap to run, hence, calibration is carried out by directly simulating from the model in the Markov Chain Monte Carlo (MCMC) loop.…”