A general-purpose tunable landscape generator

Abstract

Abstract-The research literature on metaheuristic and evolutionary computation has proposed a large number of algorithms for the solution of challenging real-world optimization problems. It is often not possible to study theoretically the performance of these algorithms unless significant assumptions are made on either the algorithm itself or the problems to which it is applied, or both. As a consequence, metaheuristics are typically evaluated empirically using a set of test problems. Unfortunately, relatively little attention has been given to the development of methodologies and tools for the large-scale empirical evaluation and/or comparison of metaheuristics.In this paper, we propose a landscape (test-problem) generator that can be used to generate optimization problem instances for continuous, bound-constrained optimization problems. The landscape generator is parameterized by a small number of parameters, and the values of these parameters have a direct and intuitive interpretation in terms of the geometric features of the landscapes that they produce. An experimental space is defined over algorithms and problems, via a tuple of parameters for any specified algorithm and problem class (here determined by the landscape generator). An experiment is then clearly specified as a point in this space, in a way that is analogous to other areas of experimental algorithmics, and more generally in experimental design. Experimental results are presented, demonstrating the use of the landscape generator. In particular, we analyze some simple, continuous estimation of distribution algorithms, and gain new insights into the behavior of these algorithms using the landscape generator.Index Terms-Continuous optimization, empirical algorithm analysis, estimation of distribution algorithm, test-problem generator.

Empirical Review of Standard Benchmark Functions Using Evolutionary Global Optimization

“…Of multimodal nature, with a single, well-defined global minimum. To have better control over these criteria when generating benchmark functions, a few "search landscape generators" have been proposed in recent years [31][32][33]. The simplest and most flexible of these is the one based on randomly distributed Gaussians [31].…”

“…We would like to emphasize already at this point that our intention in using GRUNGE is not to re-iterate known results from Ref. [31] and similar work, but to directly contrast the OGOLEM behavior displayed in section III with its different behavior in the GRUNGE benchmark. This shows strikingly that the rather uniform results in section III are not a feature of OGOLEM but rather a defect of that benchmark function class.…”

When Does Dependency Modelling Help? Using a Randomized Landscape Generator to Compare Algorithms in Terms of Problem Structure

“…This includes the development of large-scale competitions and associated sets of benchmark test problems (e.g at recent Genetic and Evolutionary Computation Conferences (GECCO) and Congress on Evolutionary Computation (CEC)). Several different types of test problems have been used for the evaluation of algorithms, including constructed analytical functions, real-world problem instances or simplified versions of real-world problems and problem/landscape generators [3,4,5]. Different problem types have their own characteristics, however it is usually the case that complementary insights into algorithm behaviour result from conducting larger experimental studies using a variety of different problem types.…”

“…Different problem types have their own characteristics, however it is usually the case that complementary insights into algorithm behaviour result from conducting larger experimental studies using a variety of different problem types. Max-Set of Gaussians (MSG) [3] is a randomised landscape generator that specifies test problems as a weighted sum of Gaussian functions. By specifying the number of Gaussians and the mean and covariance parameters for each component, a variety of test landscape instances can be generated.…”

“…This suggests objective fitness function landscapes with correlation structure or ridges in their contours. In [3] parameterisations of the MSG generator are described that generate localised dependencies, with peaks either uniformly distributed around the space or in a "big valley" structure. However, these rarely lead to global dependency structure and cannot be controlled directly from the generator parameters.…”