Constrained Bayesian Optimization with Max-Value Entropy Search

Perrone, Valerio; Shcherbatyi, Iaroslav; Jenatton, Rodolphe; Archambeau, Cédric; Seeger, Matthias

doi:10.48550/arxiv.1910.07003

Cited by 5 publications

(12 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…including the case of one constraint that is already shown in [25]. Although CMES is a reasonable criterion at a glance, we found that the approximation used in CMES results in the undesirable property and behavior, which we will discuss in detail at Section 3.2.…”

Section: Constrained Max-value Entropy Searchmentioning

confidence: 83%

“…Perrone et al [25] extended MES [38] to constrained optimization problem, which employs the approximation used in MES for the evaluation of the mutual information. However, their derivation is restricted to only one constraint, which severely limits the applicability of their method obviously.…”

Section: Constrained Max-value Entropy Searchmentioning

confidence: 99%

“…Hereafter, we omit the conditioning by D t−1 in mutual information, entropy, and probability when it is obvious from the context. Following the conventional MES and constrained MES [38,25], we transform and approximate the mutual information as below:…”

Section: Acquisition Function Of Cmesmentioning

confidence: 99%

“…In contrast, Max-value Entropy Search (MES) [38] greatly facilitates the computation compared with PES by using the information gain for the optimal value f * . Perrone et al [25] have proposed a MES-based CBO method called constrained MES (cMES), but it is restricted to only one constraint. For related works, we will discuss at Section 5 in more detail.…”

Section: Introductionmentioning

confidence: 99%

“…1. We first extend MES to multiple constraints, called CMES, by using the same approximation approach used in [38,25], and show the approximate information gain can be negative, which can cause irrational behavior.…”

Section: Introductionmentioning

confidence: 99%

See 4 more Smart Citations

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Takeno¹,

Tamura

Shitara

et al. 2021

Preprint

View full text Add to dashboard Cite

Recently, several Bayesian optimization (BO) methods have been extended to the expensive black-box optimization problem with unknown constraints, which is an important problem that appears frequently in practice. We focus on an information-theoretic approach called Max-value Entropy Search (MES) whose superior performance has been repeatedly shown in BO literature. Since existing MES-based constrained BO is restricted to only one constraint, we first extend it to multiple constraints, but we found that this approach can cause negative approximate values for the mutual information, which can result in unreasonable decisions. In this paper, we employ a different approximation strategy that is based on a lower bound of the mutual information, and propose a novel constrained BO method called Constrained Maxvalue Entropy Search via Information lower BOund (CMES-IBO). Our approximate mutual information derived from the lower bound has a simple closed-form that is guaranteed to be nonnegative, and we show that irrational behavior caused by the negative value can be avoided. Furthermore, by using conditional mutual information, we extend our methods to the parallel setting in which multiple queries can be issued simultaneously. Finally, we demonstrate the effectiveness of our proposed methods by benchmark functions and real-world applications to materials science.

show abstract

Section: Constrained Max-value Entropy Searchmentioning

confidence: 83%

Section: Constrained Max-value Entropy Searchmentioning

confidence: 99%

Section: Acquisition Function Of Cmesmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

Section: Introductionmentioning

confidence: 99%

See 3 more Smart Citations

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Takeno¹,

Tamura

Shitara

et al. 2021

Preprint

View full text Add to dashboard Cite

show abstract

Risk-Based Design Optimization via Scenario Generation and Genetic Programming Under Hybrid Uncertainties

Li,

Zhou,

Huang

et al. 2024

Journal of Computing and Information Science in Engineering

View full text Add to dashboard Cite

The design of complex systems often requires the incorporation of uncertainty optimization strategies to mitigate system failures resulting from multiple uncertainties during actual operation. Risk-based design optimization, as an alternative methodology that accounts for the balance between design cost and performance, has garnered significant attention and recognition. This paper proposes a risk design optimization method for tackling hybrid uncertainties via scenario generation and genetic programming. The hybrid uncertainties are quantified through scenario generation method to obtain risk assessment indicators. The genetic programming method is used to simulate the real output of the objective or constraints. To drive the optimization process, the sample-based discrete gradient expression is constructed, and the optimal scheme aligning the risk requirements is obtained. Three calculation examples of varying computing complexity are presented to verify the efficacy and usability of the suggested approach.

show abstract

Fair Bayesian Optimization

Perrone¹,

Donini²,

Zafar³

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Given the increasing importance of machine learning (ML) in our lives, algorithmic fairness techniques have been proposed to mitigate biases that can be amplified by ML. Commonly, these specialized techniques apply to a single family of ML models and a specific definition of fairness, limiting their effectiveness in practice. We introduce a general constrained Bayesian optimization (BO) framework to optimize the performance of any ML model while enforcing one or multiple fairness constraints. BO is a global optimization method that has been successfully applied to automatically tune the hyperparameters of ML models. We apply BO with fairness constraints to a range of popular models, including random forests, gradient boosting, and neural networks, showing that we can obtain accurate and fair solutions by acting solely on the hyperparameters. We also show empirically that our approach is competitive with specialized techniques that explicitly enforce fairness constraints during training, and outperforms preprocessing methods that learn unbiased representations of the input data. Moreover, our method can be used in synergy with such specialized fairness techniques to tune their hyperparameters. Finally, we study the relationship between hyperparameters and fairness of the generated model. We observe a correlation between regularization and unbiased models, explaining why acting on the hyperparameters leads to ML models that generalize well and are fair. 1 SigOpt (https://sigopt.com/); Cloud AutoML (https://www.blog.google/products/ google-cloud/cloud-automl-making-ai-accessible-every-business/); Optuna (https: //optuna.org/); Amazon AMT (https://docs.aws.amazon.com/sagemaker/latest/dg/ automatic-model-tuning.html).

show abstract

Constrained Bayesian Optimization with Max-Value Entropy Search

Cited by 5 publications

References 0 publications

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Sequential- and Parallel- Constrained Max-value Entropy Search via Information Lower Bound

Risk-Based Design Optimization via Scenario Generation and Genetic Programming Under Hybrid Uncertainties

Fair Bayesian Optimization

Contact Info

Product

Resources

About