Mixed Integer Evolution Strategies for Parameter Optimization

Abstract

Evolution strategies (ESs) are powerful probabilistic search and optimization algorithms gleaned from biological evolution theory. They have been successfully applied to a wide range of real world applications. The modern ESs are mainly designed for solving continuous parameter optimization problems. Their ability to adapt the parameters of the multivariate normal distribution used for mutation during the optimization run makes them well suited for this domain. In this article we describe and study mixed integer evolution strategies (MIES), which are natural extensions of ES for mixed integer optimization problems. MIES can deal with parameter vectors consisting not only of continuous variables but also with nominal discrete and integer variables. Following the design principles of the canonical evolution strategies, they use specialized mutation operators tailored for the aforementioned mixed parameter classes. For each type of variable, the choice of mutation operators is governed by a natural metric for this variable type, maximal entropy, and symmetry considerations. All distributions used for mutation can be controlled in their shape by means of scaling parameters, allowing self-adaptation to be implemented. After introducing and motivating the conceptual design of the MIES, we study the optimality of the self-adaptation of step sizes and mutation rates on a generalized (weighted) sphere model. Moreover, we prove global convergence of the MIES on a very general class of problems. The remainder of the article is devoted to performance studies on artificial landscapes (barrier functions and mixed integer NK landscapes), and a case study in the optimization of medical image analysis systems. In addition, we show that with proper constraint handling techniques, MIES can also be applied to classical mixed integer nonlinear programming problems.

Quadcriteria Optimization of Binary Classifiers: Error Rates, Coverage, and Complexity

“…First, other multiobjective and many-objectives optimization algorithms of high potential for the anti-spam filtering type of problems will be studied and explored (e.g. MOEA/D and NSGA-III), and also tailor made approaches for classification such as CH-EMOA [1,8] or mixed integer optimization [2]. Secondly, analysis of the rules that reveal highest contributions for the classification process will be addressed, in order to assess not only quantitative classifier complexity, but also to explore the nature of the rules most frequently present in the best solutions.…”

A Super-Structure Based Optimisation Approach for Building Spatial Designs

“…Gaussian mutation with individual step sizes is applied to the continuous parameters. Their step sizes are mutated by the local learning rate τ 1 = 1/ 2 √ N dims and global learning rate τ 2 = 1/ √ 2 × N dims as in Li et al [7]. Implementation wise a global value g 1 is drawn from a Gaussian distribution g 1 = G(0, 1) for every individual.…”

“…Evolution Strategies (ESs) were developed by Ingo Rechenberg and Hans-Paul Schwefel at the TU Berlin in the 60s and are especially well suited for solving engineering design problems [6]. They are interesting for this work, as they can deal with discrete as well as with continuous design variables, as outlined in Li et al [7]. In Figure 1 the main loop of a (µ + λ)-ES is summarised.…”

A memetic algorithm for cardinality-constrained portfolio optimization with transaction costs

“…This subordinate optimization problem does not have cardinality or turnover constraints, which means that it can be solved using standard QP solvers. The main advantage of this pure combinatorial encoding compared to mixed encodings like those used in [50], [22] and [8], where chromosomes with both discrete and continuous components are used, resides in the fact that the GA can focus on solving the combinatorial optimization problem of finding the optimal subset of assets and the trades to be performed without having to handle the continuous constraints. This separation has been shown to increase the performance and effectiveness of cardinality-constrained portfolio selection algorithms [31] [42] [41].…”