Overview and Comparison of Gaussian Process-Based Surrogate Models for Mixed Continuous and Discrete Variables: Application on Aerospace Design Problems

Pelamatti, Julien; Brevault, Loïc; Balesdent, Mathieu; Talbi, El‐Ghazali; Guérin, Yannick

doi:10.1007/978-3-030-18764-4_9

Cited by 14 publications

(14 citation statements)

References 34 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Although less common than the standard continuous kernels, a few kernels allowing to characterize the covariance function between the values of a discrete unordered variable such as p exist in the literature. Different examples can be found in [64,65,66]. Given that the main focus of this work is not to optimize the modeling performance of a multi-output GP model, but rather to study its effect within the framework of robust optimization, a single discrete kernel parameterization known as the hypersphere decomposition [67] is considered for the remainder of the article.…”

Section: Gaussian Regression For a Multi-output Model Of The Constraintsmentioning

confidence: 99%

Coupling and selecting constraints in Bayesian optimization under uncertainties

Pelamatti¹,

Riche²,

Helbert³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

We consider chance constrained optimization where it is sought to optimize a function while complying with constraints, both of which are affected by uncertainties. The high computational cost of realistic simulations strongly limits the number of evaluations and makes this type of problems particularly challenging. In such a context, it is common to rely on Bayesian optimization algorithms. Assuming, without loss of generality, that the uncertainty comes from some of the inputs, it becomes possible to build a Gaussian process model in the joint space of design and uncertain variables. A two-step acquisition function is then used to provide, both, promising optimization variables associated to relevant uncertain samples. Our overall contribution is to correlate the constraints in the GP model and exploit this to optimally decide, at each iteration, which constraint should be evaluated and at which point. The coupled Gaussian model of the constraints relies on an output-as-input encoding. The constraint selection idea is developed by enabling that each constraint can be evaluated for a different uncertain input, thus improving the refinement efficiency. Constraints coupling and selection are gradually implement in 3 algorithm variants which are compared to a reference Bayesian approach. The results are promising in terms of convergence speed, accuracy and stability as observed on a 2, a 4 and a 27-dimensional problems.

show abstract

Section: Gaussian Regression For a Multi-output Model Of The Constraintsmentioning

confidence: 99%

Coupling and selecting constraints in Bayesian optimization under uncertainties

Pelamatti¹,

Riche²,

Helbert³

et al. 2022

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…Although we made a partial replacement with MAB, the GP with our constructed kernel enables the algorithm to learn the correlation between different variables. For integer variable, despite its discreteness, we exclude it from applying MAB since it rather acts as a continuous variable in the sense that both are quantitative [14,10,4]. If the candidate turns out to be the best evaluation, each α parameter in the beta distribution of the corresponding arms increases 4 (Figure 3 (b)) as a reward.…”

Section: Multi-armed Bandits On Qualitative Variablesmentioning

confidence: 99%

Adaptive Local Bayesian Optimization Over Multiple Discrete Variables

Kim¹,

Ahn²,

Kim³

et al. 2020

Preprint

View full text Add to dashboard Cite

In the machine learning algorithms, the choice of the hyperparameter is often an art more than a science, requiring labor-intensive search with expert experience. Therefore, automation on hyperparameter optimization to exclude human intervention is a great appeal, especially for the black-box functions. Recently, there have been increasing demands of solving such concealed tasks for better generalization, though the task-dependent issue is not easy to solve. The Black-Box Optimization challenge (NeurIPS 2020) required competitors to build a robust black-box optimizer across different domains of standard machine learning problems. This paper describes the approach of team KAIST OSI in a step-wise manner, which outperforms the baseline algorithms by up to +20.39%. We first strengthen the local Bayesian search under the concept of region reliability. Then, we design a combinatorial kernel for a Gaussian process kernel. In a similar vein, we combine the methodology of Bayesian and multi-armed bandit (MAB) approach to select the values with the consideration of the variable types; the real and integer variables are with Bayesian, while the boolean and categorical variables are with MAB. Empirical evaluations demonstrate that our method outperforms the existing methods across different tasks.

show abstract

“…However, when dealing with actual engineering design problems, it may occur that some combinations of discrete design variables are not physically feasible or can not be modeled, in which case the previously complete parameterization of the discrete kernel can not be applied. An example of this issue can be found in the modeling of a rocket engine propulsive performance, in which not all the combinations of reductant and oxidant result in a feasible combustion process [16]. For these reasons, the complete parameterization of the discrete kernel is not considered in this paper.…”

Section: Discrete Kernel Complete Parameterizationmentioning

confidence: 99%

“…In this case, the covariance is characterized dimension-wise rather than category-wise, the number of hyperparameters required to define the r matrices T s is therefore equal to k=r k=1 b k (b k + 1)/2. When compared to the complete hypersphere decomposition of the discrete kernel presented in Section 3.1, the dimension-wise variant offers a better scaling with the discrete dimension of the problem that is being modeled in terms of number of hyperparameters, but as a trade-off provides a theoretically less accurate modeling of the correlation between the various discrete categories of the problem [16]. Furthermore, due to the fact that the kernel is defined dimension-wise, it is not necessary for all the problem categories to be represented in the training data set in order to train the hyperparameters.…”

Section: Heteroscedastic Dimension-wise Hypersphere Decompositionmentioning

confidence: 99%

“…However, it must also be noticed that because each discrete variable is only parameterized by 2 hyperparameters θ and p, the surrogate model may present poor modeling performances in the presence of discrete variables with large numbers of discrete levels. For the same reason, the simultaneous presence of correlation and anti-correlation trends described by the same discrete variable might be more difficult to model when compared to the parameterizations described in the previous paragraphs [16].…”

Section: Compound Symmetry Parameterizationmentioning

confidence: 99%

See 1 more Smart Citation

Efficient global optimization of constrained mixed variable problems

et al. 2018

Self Cite

View full text Add to dashboard Cite

Due to the increasing demand for high performance and cost reduction within the framework of complex system design, numerical optimization of computationally costly problems is an increasingly popular topic in most engineering fields. In this paper, several variants of the Efficient Global Optimization algorithm for costly constrained problems depending simultaneously on continuous decision variables as well as on quantitative and/or qualitative discrete design parameters are proposed. The adaptation that is considered is based on a redefinition of the Gaussian Process kernel as a product between the standard continuous kernel and a second kernel representing the covariance between the discrete variable values. Several parameterizations of this discrete kernel, with their respective strengths and weaknesses, are discussed in this paper. The novel algorithms are tested on a number of analytical test-cases and an aerospace related design problem, and it is shown that they require fewer function evaluations in order to converge towards the neighborhoods of the problem optima when compared to more commonly used optimization algorithms.

show abstract

Overview and Comparison of Gaussian Process-Based Surrogate Models for Mixed Continuous and Discrete Variables: Application on Aerospace Design Problems

Cited by 14 publications

References 34 publications

Coupling and selecting constraints in Bayesian optimization under uncertainties

Coupling and selecting constraints in Bayesian optimization under uncertainties

Adaptive Local Bayesian Optimization Over Multiple Discrete Variables

Efficient global optimization of constrained mixed variable problems

Contact Info

Product

Resources

About