Analyzing Stochastic Computer Models: A Review with Opportunities

Baker, Evan; Barbillon, Pierre; Fadikar, Arindam; Gramacy, Robert B.; Herbei, Radu; Higdon, David; Huang, Jiangeng; Johnson, Leah R.; Ma, Pulong; Mondal, Anirban; Pires, Bianica; Sacks, Jerome; Sokolov, Vadim

doi:10.48550/arxiv.2002.01321

Cited by 5 publications

(6 citation statements)

References 93 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…If the noise level is varying throughout the input space (i.e., heteroskedasticity), that process could be rather more involved (Binois et al, 2018). Surrogate modeling and active learning for stochastic simulation is still very much on the frontier of the computer experiments landscape (Baker et al, 2020). In such settings, the acquisition space must be extended to include the possibility of obtaining a replicate run, duplicating one of the n existing design elements (Binois et al, 2019).…”

Section: Discussionmentioning

confidence: 99%

Triangulation candidates for Bayesian optimization

Gramacy¹,

Sauer²,

Wycoff³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

Bayesian optimization is a form of sequential design: idealize input-output relationships with a suitably flexible nonlinear regression model; fit to data from an initial experimental campaign; devise and optimize a criterion for selecting the next experimental condition(s) under the fitted model (e.g., via predictive equations) to target outcomes of interest (say minima); repeat after acquiring output under those conditions and updating the fit. In many situations this "inner optimization" over the new-data acquisition criterion is cumbersome because it is non-convex/highly multi-modal, may be non-differentiable, or may otherwise thwart numerical optimizers, especially when inference requires Monte Carlo. In such cases it is not uncommon to replace continuous search with a discrete one over random candidates. Here we propose using candidates based on a Delaunay triangulation of the existing input design. In addition to detailing construction of these "tricands", based on a simple wrapper around a conventional convex hull library, we promote several advantages based on properties of the geometric criterion involved. We then demonstrate empirically how tricands can lead to better Bayesian optimization performance compared to both numerically optimized acquisitions and random candidate-based alternatives on benchmark problems.

show abstract

Section: Discussionmentioning

confidence: 99%

Triangulation candidates for Bayesian optimization

Gramacy¹,

Sauer²,

Wycoff³

2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Typically filed data such as physical measurement of a simulated object of survey Auld, Sokolov, et al 2012 can be used to calibrate parameters of a simulator. Baker et al 2020 provide an overview of Bayesian techniques for calibration and analysis of simulators.…”

Section: Surrogates and Optimisationmentioning

confidence: 99%

Bayesian Learning: A Selective Overview

Hsu¹,

Jeng²,

Murali³

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

This paper presents an overview of some of the concepts of Bayesian Learning. The number of scientific and industrial applications of Bayesian learning has been growing in size rapidly over the last few decades (Damien et al. 2013). This process has started with the wide use of Markov Chain Monte Carlo methods (S. Geman and D. Geman 1984; Gelfand and Smith 1990; Gamerman and Hedibert F Lopes 2006) that emerged as a dominant computational technique for Bayesian in the early 1990's. Since then Bayesian learning has spread well across several fields from robotics and machine learning to medical applications. This paper provides an overview of some of the widely used concepts and shows several applications. This is a paper based on the series of seminars given by students of a PhD course on Bayesian Learning at George Mason University. The course was taught in the Fall of 2021. Thus, the topics covered in the paper reflect the topics students selected to study.

show abstract

“…Other options include upper confidence bound (UCB) (Srinivas et al, 2010), knowledge gradient (Frazier, 2018), and entropy based criteria (Villemonteix et al, 2009b;Hennig and Schuler, 2012;Wang and Jegelka, 2017). Noisy simulators have their own set of challenges, see e.g., (Baker et al, 2020), and raise questions about selecting the right amount of replication. While not necessary per se, repeating experiments is the best option to separate signal from noise, and is beneficial in terms of computational speed, see e.g., Zhang et al, 2020).…”

Section: Introductionmentioning

confidence: 99%

A portfolio approach to massively parallel Bayesian optimization

Binois,

Collier,

Ozik

2021

Preprint

View full text Add to dashboard Cite

One way to reduce the time of conducting optimization studies is to evaluate designs in parallel rather than just one-at-a-time. For expensive-to-evaluate black-boxes, batch versions of Bayesian optimization have been proposed. They work by building a surrogate model of the black-box that can be used to select the designs to evaluate efficiently via an infill criterion. Still, with higher levels of parallelization becoming available, the strategies that work for a few tens of parallel evaluations become limiting, in particular due to the complexity of selecting more evaluations. It is even more crucial when the black-box is noisy, necessitating more evaluations as well as repeating experiments. Here we propose a scalable strategy that can keep up with massive batching natively, focused on the exploration/exploitation trade-off and a portfolio allocation. We compare the approach with related methods on deterministic and noisy functions, for mono and multiobjective optimization tasks. These experiments show similar or better performance than existing methods, while being orders of magnitude faster.

show abstract

Analyzing Stochastic Computer Models: A Review with Opportunities

Cited by 5 publications

References 93 publications

Triangulation candidates for Bayesian optimization

Triangulation candidates for Bayesian optimization

Bayesian Learning: A Selective Overview

A portfolio approach to massively parallel Bayesian optimization

Contact Info

Product

Resources

About