Learning hybridization strategies in evolutionary algorithms

Abstract

Evolutionary Algorithms are powerful optimization techniques which have been applied to many different problems, from complex mathematical functions to real-world applications. Some studies report performance improvements through the combination of different evolutionary approaches within the same hybrid algorithm. However, the mechanisms used to control this combination of evolutionary approaches are not as satisfactory as would be desirable. In most cases, there is no feedback from the algorithm nor any regulatory component that modifies the participation of each evolutionary approach in the overall search process. In some cases, the algorithm makes use of some information for an on-line adaptation of the participation of each algorithm. In this paper, the use of Reinforcement Learning (RL) is proposed as a mechanism to control how the different evolutionary approaches contribute to the overall search process. In particular, three learning policies based on one of the state-of-the-art RL algorithms, Q-Learning, have been considered and used to control the participation of each algorithm by learning the best-response mixed strategy. To test this approach, a benchmark made up of six large-scale (500 dimensions) continuous optimization functions has been considered. The experimentation carried out has proved that RL control mechanisms successfully learn optimal patterns for the combination of Evolutionary Algorithms in most of the proposed functions, being able to improve the performance of both individual and non RL hybrid algorithms.

Benchmarking a hybrid DE-RHC algorithm on real world problems

“…When applied to the DE algorithm, this strategy prevents the stagnation problems of the DE by reducing the excessive difference between the best individual and the rest of the population. In [17] a quite different approach is followed. In this work, a hybrid algorithm combines several Genetic Algorithms in such a way that their contribution to the overall search is learned through the use of Reinforcement Learning techniques.…”

Bayesian Classifiers Applied to the Tennessee Eastman Process

“…Recently, a hybrid adaptive algorithm that is capable of combining several evolutionary approaches has appeared. This new method is named MOS and uses the mechanisms offered by evolutionary heuristics to create the offspring for the next generation …”

“…Each of these mechanisms capable of producing a new offspring is called a technique. Specifically, a technique may be defined as a particular evolutionary model, with an appropriate encoding, using specific operators (if needed), and configured with its necessary parameters. …”

“…The calculation of the amount of new individuals created in each generation, for n different offspring sampling methods, is obtained using a participation function. These functions can carry out simple static assignments or, more interestingly, dynamic adjustments according to a quality measure that evaluates how good the offspring of each technique is from the point of view of that measure …”