Calibration of Stochastic Computer Simulators Using Likelihood Emulation

Oakley, Jeremy E.; Youngman, Benjamin D.

doi:10.1080/00401706.2015.1125391

Cited by 15 publications

(8 citation statements)

References 28 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Currently, Bayesian calibration of microsimulation DMs might not be feasible on regular desktops or laptops. To circumvent current computational limitations from using Bayesian methods in calibrating microsimulation models, surrogate models -often called metamodels or emulators-have been proposed ( O’Hagan et al, 1999 ; O’Hagan, 2006 ; Oakley and Youngman, 2017 ). Surrogate models are statistical models like Gaussian processes ( Sacks et al, 1989a ; Sacks et al, 1989b ; Oakley and O’Hagan, 2002 ) or neural networks ( Hauser et al, 2012 ; Jalal et al, 2021 ) that aim to replace the relationship between inputs and outputs of the original microsimulation DM ( Barton et al, 1992 ; Kleijnen, 2015 ), which, once fitted, are computationally more efficient to run than the microsimulation DM.…”

Section: Discussionmentioning

confidence: 99%

Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models

et al. 2022

View full text Add to dashboard Cite

Background: We evaluated the implications of different approaches to characterize the uncertainty of calibrated parameters of microsimulation decision models (DMs) and quantified the value of such uncertainty in decision making.Methods: We calibrated the natural history model of CRC to simulated epidemiological data with different degrees of uncertainty and obtained the joint posterior distribution of the parameters using a Bayesian approach. We conducted a probabilistic sensitivity analysis (PSA) on all the model parameters with different characterizations of the uncertainty of the calibrated parameters. We estimated the value of uncertainty of the various characterizations with a value of information analysis. We conducted all analyses using high-performance computing resources running the Extreme-scale Model Exploration with Swift (EMEWS) framework.Results: The posterior distribution had a high correlation among some parameters. The parameters of the Weibull hazard function for the age of onset of adenomas had the highest posterior correlation of −0.958. When comparing full posterior distributions and the maximum-a-posteriori estimate of the calibrated parameters, there is little difference in the spread of the distribution of the CEA outcomes with a similar expected value of perfect information (EVPI) of $653 and $685, respectively, at a willingness-to-pay (WTP) threshold of $66,000 per quality-adjusted life year (QALY). Ignoring correlation on the calibrated parameters’ posterior distribution produced the broadest distribution of CEA outcomes and the highest EVPI of $809 at the same WTP threshold.Conclusion: Different characterizations of the uncertainty of calibrated parameters affect the expected value of eliminating parametric uncertainty on the CEA. Ignoring inherent correlation among calibrated parameters on a PSA overestimates the value of uncertainty.

show abstract

Section: Discussionmentioning

confidence: 99%

Characterization and Valuation of the Uncertainty of Calibrated Parameters in Microsimulation Decision Models

et al. 2022

View full text Add to dashboard Cite

show abstract

“…However there is no λ E since we replace this by a GP to account for heteroscedasticity. We adopt priors over the β * j that are very flat, as is often the case in the computer experiments literature (Oakley & Youngman 2017). Hence we take m j, * = 0 and s j, * = 10.…”

Section: Prior Specificationmentioning

confidence: 99%

Multilevel Emulation for Stochastic Computer Models with Application to Large Offshore Wind farms

Kennedy¹,

Henderson²,

Wilson³

2020

Preprint

View full text Add to dashboard Cite

Increasingly, stochastic computer models are being used in science and engineering to predict and understand complex phenomena. Despite the power of modern computing, these simulators are often too computationally costly to be of practical use due to their complexity. Hence the emulation of stochastic computer models is a problem of increasing interest. Many stochastic computer models can be run at different levels of complexity, which incurs a trade-off with simulation accuracy. More complex simulations are more expensive to run, but will often be correlated with less complex but cheaper to run versions. We present a heteroscedastic Gaussian process approach to emulation of stochastic simulators which utilises cheap approximations to a stochastic simulator, motivated by a stochastic reliability and maintenance model of a large offshore windfarm. The performance of our proposed methodology is demonstrated on two synthetic examples (a simple, tractable example and a predator-prey model) before being applied to the stochastic windfarm simulator.

show abstract

“…For stochastic simulators, many of the same techniques can be applied. The likelihood function now needs to be estimated, significantly increasing the difficulty, but progress is being made in this direction (Oakley and Youngman, 2014;Meeds and Welling, 2014;Wilkinson, 2014;Gutmann and Corander, 2015).…”

Section: Future Applicationsmentioning

confidence: 99%