Efficient multi-criteria optimization on noisy machine learning problems

Koch, Peter; Wagner, Tobias; Emmerich, Michael; Bäck, Thomas; Konen, Wolfgang

doi:10.1016/j.asoc.2015.01.005

Cited by 37 publications

(21 citation statements)

References 52 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…On the other hand, the inherent complexity of standard implementations of Bayesian methods limits the number of values of the objective functions which can be processed. The modest number of iterations seems sufficient for the solution of various applied problems, e.g., up to 60 evaluations of values of the objective functions in [24], 100 function evaluations in [27], and 200 functions evaluations in [20]. However, such a number of function values is not sufficient for the appropriate representation of the Pareto front of the considered three-objective optimization problem.…”

Section: 2 Optimization Algorithmmentioning

confidence: 99%

Multi-objective optimization and decision visualization of batch stirred tank reactor based on spherical catalyst particles

Žilinskas

Baronas

Litvinas

et al. 2019

NAMC

View full text Add to dashboard Cite

This paper presents a Bayesian approach rooted algorithm oriented to the properties of multi-objective optimization problems. The performance of the developed algorithm is compared with several other multi-objective optimization algorithms. The approach is applied to the multiobjective optimization of a batch stirred tank reactor based on spherical catalyst microreactors. The microbioreactors are computationally modeled by a two-compartment model based on reaction–diffusion equations containing a nonlinear term related to the Michaelis–Menten enzyme kinetics. A two-stage visualization procedure based on the multi-dimensional scaling is proposed and applied for the visualization of trade-off solutions and for the selection of favorable configurations of the bioreactor.

show abstract

Section: 2 Optimization Algorithmmentioning

confidence: 99%

Multi-objective optimization and decision visualization of batch stirred tank reactor based on spherical catalyst particles

Žilinskas

Baronas

Litvinas

et al. 2019

NAMC

View full text Add to dashboard Cite

show abstract

“…Given (x i , f (x i )), i = 1, · · · , t, the incumbent Pareto-front approximation becomes P max,t = Non-Dominated subset of {f (x i )|i ∈ {1, ..., t}} and its hypervolume replaces f max,t in the definition of the EI. The EHVI was first proposed in [21], and since then it has been used in Evolutionary Algorithms for airfoil optimization [4] and quantum control [22], as well as in multi-criterion generalizations of Efficient Global Optimization for applications in fluid dynamics [23], event controllers in wastewater treatment [1], efficient algorithm tuning [5], electrical component design [8], and bio-fuel power-generation [3]. In all of these applications, the bi-objective EHVI was used.…”

Section: Relevance and Related Workmentioning

confidence: 99%

“…In the context of SAMCO, the Expected Hypervolume Improvement (EHVI) is frequently used as an infill or pre-selection criterion [1][2][3][4][5][6][7][8], and is a straightforward generalization of the single-objective expected improvement (EI). It is called multiple times because, in each iteration of SAMCO, either an optimum of the EI needs to be found, as in the case of Bayesian global optimization 1 [10], or, in surrogate-assisted evolutionary algorithms, it is used to pre-assess the quality of the individuals of a larger population.…”

Section: Introductionmentioning

confidence: 99%

Computing 3-D Expected Hypervolume Improvement and Related Integrals in Asymptotically Optimal Time

Yang

Emmerich

Deutz

et al. 2017

Lecture Notes in Computer Science

View full text Add to dashboard Cite

Abstract. The Expected Hypervolume Improvement (EHVI) is a frequently used infill criterion in surrogate-assisted multi-criterion optimization. It needs to be frequently called during the execution of such algorithms. Despite recent advances in improving computational efficiency, its running time for three or more objectives has remained inwhere d is the number of objective functions and n is the size of the incumbent Pareto-front approximation. This paper proposes a new integration scheme, which makes it possible to compute the EHVI in Θ(n log n) optimal time for the important three-objective case (d = 3). The new scheme allows for a generalization to higher dimensions and for computing the Probability of Improvement (PoI) integral efficiently. It is shown, both theoretically and empirically, that the hidden constant in the asymptotic notation is small. Empirical speed comparisons were designed between the C++ implementations of the new algorithm (which will be in the public domain) and those recently published by competitors, on randomly-generated non-dominated fronts of size 10, 100, and 1000. The experiments include the analysis of batch computations, in which only the parameters of the probability distribution change but the incumbent Pareto-front approximation stays the same. Experimental results show that the new algorithm is always faster than the other algorithms, sometimes over 10 4 times faster.

show abstract

“…Compared to evolutionary multi-objective optimization algorithms (EMOAs), MOBGO requires only a small budget of function evaluations to achieve a similar result with respect to hypervolume indicator, and it has already been used in real-world applications to solve expensive evaluation problems [40]. According to the authors' knowledge, BGO was used for the first time in the context of airfoil optimization in [27], and then applied in the field of biogas plant controllers [16], detection in water quality management [41], machine learning algorithm configuration [23], and structural design optimization [33].…”

Section: Introductionmentioning

confidence: 99%

Efficient computation of expected hypervolume improvement using box decomposition algorithms

et al. 2019

Self Cite

View full text Add to dashboard Cite

In the field of multi-objective optimization algorithms, multi-objective Bayesian Global Optimization (MOBGO) is an important branch, in addition to evolutionary multiobjective optimization algorithms (EMOAs). MOBGO utilizes Gaussian Process models learned from previous objective function evaluations to decide the next evaluation site by maximizing or minimizing an infill criterion. A commonly used criterion in MOBGO is the Expected Hypervolume Improvement (EHVI), which shows a good performance on a wide range of problems, with respect to exploration and exploitation. However, so far, it has been a challenge to calculate exact EHVI values efficiently. This paper proposes an efficient algorithm for the exact calculation of the EHVI for in a generic case. This efficient algorithm is based on partitioning the integration volume into a set of axis-parallel slices. Theoretically, the upper bound time complexities can be improved from previously O(n 2 ) and O(n 3 ), for two-and three-objective problems respectively, to Θ(n log n), which is asymptotically optimal. This article generalizes the scheme in higher dimensional cases by utilizing a new hyperbox decomposition technique, which is proposed by Dächert et al., EJOR, 2017. It also utilizes a generalization of the multilayered integration scheme that scales linearly in the number of hyperboxes of the decomposition. The speed comparison shows that the proposed algorithm in this paper significantly reduces computation time. Finally, this decomposition technique is applied in the calculation of the Probability of Improvement (PoI).

show abstract

Efficient multi-criteria optimization on noisy machine learning problems

Cited by 37 publications

References 52 publications

Multi-objective optimization and decision visualization of batch stirred tank reactor based on spherical catalyst particles

Multi-objective optimization and decision visualization of batch stirred tank reactor based on spherical catalyst particles

Computing 3-D Expected Hypervolume Improvement and Related Integrals in Asymptotically Optimal Time

Efficient computation of expected hypervolume improvement using box decomposition algorithms

Contact Info

Product

Resources

About