Bayesian optimization with approximate set kernels

Kim, Jungtaek; McCourt, Michael; You, Tae; Kim, Saehoon; Choi, Seungjin

doi:10.1007/s10994-021-05949-0

Cited by 9 publications

(12 citation statements)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Therefore, it is straightforward to use it in traditional Bayesian optimisation for single or multi-objective optimisation problems. In a recent work [14], the correlation was extended to sets and a set kernel was proposed. Given a kernel (x, x ′ ) between two decision vectors, the set kernel between two sets and ′ is defined as:…”

Section: Gaussian Processes Over Setsmentioning

confidence: 99%

“…It is worthy to mention that the computational complexity of the set kernel is ( 2 s | | 2 ), which makes it computationally expensive compared to the traditional kernel. In [14], the authors proposed an approximation of the set kernel to alleviate the computational cost. In this work, we do not use such approximation and assume that the computational cost of building models is significantly lower than expensive objective evaluation.…”

Section: Gaussian Processes Over Setsmentioning

confidence: 99%

“…A recent proposal has extended the use of Gaussian processes (GPs) over sets and applied them in Bayesian optimisation [14]. In this paper, we apply this proposal for the set-based optimisation of the hypervolume and indicators and evaluate its performance against non set-based Bayesian optimisation methods.…”

Section: Introductionmentioning

confidence: 99%

See 2 more Smart Citations

Maximising hypervolume and minimising ϵ-indicators using Bayesian optimisation over sets

Chugh

López-Ibáñez

2021

Proceedings of the Genetic and Evolutionary Computation Conference Companion

View full text Add to dashboard Cite

Bayesian optimisation methods have been widely used to solve problems with computationally expensive objective functions. In the multi-objective case, these methods have been successfully applied to maximise the expected hypervolume improvement of individual solutions. However, the hypervolume, and other unary quality indicators such as multiplicative -indicator, measure the quality of an approximation set and the overall goal is to find the set with the best indicator value. Unfortunately, the literature on Bayesian optimisation over sets is scarce. This work uses a recent set-based kernel in Gaussian processes and applies it to maximise hypervolume and minimise -indicators in Bayesian optimisation over sets. The results on benchmark problems show that maximising hypervolume using Bayesian optimisation over sets gives a similar performance than non-set based methods. The performance of using indicator in Bayesian optimisation over sets needs to be investigated further. The set-based method is computationally more expensive than the non-set-based ones, but the overall time may be still negligible in practice compared to the expensive objective functions. CCS CONCEPTS• Mathematics of computing → Probabilistic algorithms; • Computing methodologies → Supervised learning by regression.

show abstract

Section: Gaussian Processes Over Setsmentioning

confidence: 99%

Section: Gaussian Processes Over Setsmentioning

confidence: 99%

See 1 more Smart Citation

Maximising hypervolume and minimising ϵ-indicators using Bayesian optimisation over sets

Chugh

López-Ibáñez

2021

Proceedings of the Genetic and Evolutionary Computation Conference Companion

View full text Add to dashboard Cite

show abstract

“…Bayesian optimization on structured domains, which is distinct from a vector-based Bayesian optimization (Moćkus et al, 1978;Jones et al, 1998), attracts considerable attention from Bayesian optimization community, due to its potential on novel practical applications such as sensor set selection (Garnett et al, 2010), hyperparameter optimization (Hutter et al, 2011;Wang et al, 2016), aero-structural problem (Baptista and Poloczek, 2018), clustering initialization (Kim et al, 2019), and neural architecture search (Oh et al, 2019). This line of research discovers a new formulation of Bayesian optimization tasks, by defining surrogate models and acquisition functions on structured domains.…”

Section: Related Workmentioning

confidence: 99%

“…However, depending on problems, input variables can be of different structures. For instance, recent work includes Bayesian optimization over sets (Garnett et al, 2010;Kim et al, 2019;Buathong et al, 2020), over combinatorial inputs (Hutter et al, 2011;Baptista and Poloczek, 2018), or over graph-structured inputs (Cui and Yang, 2018;Oh et al, 2019).…”

Section: Introductionmentioning

confidence: 99%

Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes

Kim¹,

Choi²,

Cho³

2020

Preprint

Self Cite

View full text Add to dashboard Cite

Bayesian optimization is a popular method for solving the problem of global optimization of an expensive-to-evaluate black-box function. It relies on a probabilistic surrogate model of the objective function, upon which an acquisition function is built to determine where next to evaluate the objective function. In general, Bayesian optimization with Gaussian process regression operates on a continuous space. When input variables are categorical or discrete, an extra care is needed. A common approach is to use one-hot encoded or Boolean representation for categorical variables which might yield a combinatorial explosion problem. In this paper we present a method for Bayesian optimization in a combinatorial space, which can operate well in a large combinatorial space. The main idea is to use a random mapping which embeds the combinatorial space into a convex polytope in a continuous space, on which all essential process is performed to determine a solution to the black-box optimization in the combinatorial space. We describe our combinatorial Bayesian optimization algorithm and present its regret analysis. Numerical experiments demonstrate that our method outperforms existing methods.

show abstract

CMD: controllable matrix decomposition with global optimization for deep neural network compression

et al. 2022

View full text Add to dashboard Cite

Bayesian optimization with approximate set kernels

Cited by 9 publications

References 11 publications

Maximising hypervolume and minimising ϵ-indicators using Bayesian optimisation over sets

Maximising hypervolume and minimising ϵ-indicators using Bayesian optimisation over sets

Combinatorial Bayesian Optimization with Random Mapping Functions to Convex Polytopes

CMD: controllable matrix decomposition with global optimization for deep neural network compression

Contact Info

Product

Resources

About