Bayesian optimization with output-weighted optimal sampling

Blanchard, Antoine; Sapsis, Themistoklis P.

doi:10.1016/j.jcp.2020.109901

Cited by 36 publications

(56 citation statements)

References 20 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…We have found that the likelihood ratio accelerates convergence of the sequential algorithm in a number of examples related to uncertainty quantification and rare-event prediction. The question of whether gains of similar proportions might be achieved in Bayesian optimization (where the focus is on learning the minimum of the objective function rather than the objective function itself or its statistics) is considered in [5].…”

Section: Discussionmentioning

confidence: 99%

See 1 more Smart Citation

Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification

Blanchard¹,

Sapsis²

2021

SIAM/ASA J. Uncertainty Quantification

Self Cite

View full text Add to dashboard Cite

We introduce a class of acquisition functions for sample selection that lead to faster convergence in applications related to Bayesian experimental design and uncertainty quantification. The approach follows the paradigm of active learning, whereby existing samples of a black-box function are utilized to optimize the next most informative sample. The proposed method aims to take advantage of the fact that some input directions of the black-box function have a larger impact on the output than others, which is important especially for systems exhibiting rare and extreme events. The acquisition functions introduced in this work leverage the properties of the likelihood ratio, a quantity that acts as a probabilistic sampling weight and guides the active-learning algorithm toward regions of the input space that are deemed most relevant. We demonstrate the proposed approach in the uncertainty quantification of a hydrological system as well as the probabilistic quantification of rare events in dynamical systems and the identification of their precursors in up to 30 dimensions.

show abstract

Section: Discussionmentioning

confidence: 99%

“…Since the focus of this work is on experimental design, this idea will not be pursued any further. But this issue is worth mentioning because it suggests that any successful BED strategy introduced in this paper has the potential of being equally successful in the context of BO [5].…”

mentioning

confidence: 98%

Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification

Blanchard¹,

Sapsis²

2021

SIAM/ASA J. Uncertainty Quantification

Self Cite

View full text Add to dashboard Cite

show abstract

“…We see that all bi-fidelity methods (BF-O and BF-Fn) achieve acceleration on the error reduction (to different extents) compared to the SF method. For the BF-Fn method, faster convergence is observed for larger n in the test range of n ∈ [1,15], but with much less benefit for n increasing from 10 to 15. The BF-O method provides the best result, in terms of the error e at cost c = 80, although the BF-O result is somewhat less accurate than the BF-F15 result for smaller c in the range of [25,50].…”

Section: Two-dimensional Stochastic Oscillatormentioning

confidence: 98%

“…EGRA [8] and many later improved variants [9,10,11]. Recently, new acquisition functions [12,13,14,15] have also been developed which focus on obtaining the overall probability density function (PDF) of the response with an emphasis on the extreme-value portion (which is our purpose in this paper instead of the exceeding probability for a single threshold).…”

Section: Introductionmentioning

confidence: 99%

Multi-fidelity Bayesian experimental design to quantify extreme-event statistics

Gong¹,

Pan²

2022

Preprint

View full text Add to dashboard Cite

In this work, we develop a multi-fidelity Bayesian experimental design framework to efficiently quantify the extreme-event statistics of an input-to-response (ItR) system with given input probability and expensive function evaluations. The key idea here is to leverage low-fidelity samples whose responses can be computed with a cost of a certain fraction of that for high-fidelity samples, in an optimized configuration to reduce the total computational cost. To accomplish this goal, we employ a multi-fidelity Gaussian process as the surrogate model of the ItR function, and develop a new acquisition based on which the optimized next sample can be selected in terms of its location in the sample space and the fidelity level. In addition, we develop an inexpensive analytical evaluation of the acquisition and its derivative, avoiding numerical integrations that are prohibitive for high-dimensional problems. The new method is tested in a bi-fidelity context for a series of synthetic problems with varying dimensions, low-fidelity model accuracy and computational costs. Comparing with the single-fidelity method and the bi-fidelity method with a pre-defined fidelity hierarchy, our method consistently shows the best (or among the best) performance for all the test cases. Finally, we demonstrate the superiority of our method in solving an engineering problem of estimating the extreme ship motion statistics in irregular waves, using computational fluid dynamics (CFD) with two different grid resolutions as the high and low fidelity models.

show abstract

“…Random forests [60] and support vector machines [61] are two other leading architectures for supervised learning. Bayesian methods are also widely used, especially for dynamical sys-tems [62]. Genetic programming has also been widely used to learn human-interpretable, yet flexible representations of data for modeling [16,[63][64][65] and control [4].…”

Section: The Architecturementioning

confidence: 99%

Applying machine learning to study fluid mechanics

Brunton

2021

Acta Mech. Sin.

View full text Add to dashboard Cite

This paper provides a short overview of how to use machine learning to build data-driven models in fluid mechanics. The process of machine learning is broken down into five stages: (1) formulating a problem to model, (2) collecting and curating training data to inform the model, (3) choosing an architecture with which to represent the model, (4) designing a loss function to assess the performance of the model, and (5) selecting and implementing an optimization algorithm to train the model. At each stage, we discuss how prior physical knowledge may be embedding into the process, with specific examples from the field of fluid mechanics. Graphic abstract

show abstract

Bayesian optimization with output-weighted optimal sampling

Cited by 36 publications

References 20 publications

Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification

Output-Weighted Optimal Sampling for Bayesian Experimental Design and Uncertainty Quantification

Multi-fidelity Bayesian experimental design to quantify extreme-event statistics

Applying machine learning to study fluid mechanics

Contact Info

Product

Resources

About