Test Function Generators for Assessing the Performance of PSO Algorithms in Multimodal Optimization

Barrera, Julio; Coello, Carlos A. Coello

doi:10.1007/978-3-642-17390-5_4

Cited by 12 publications

(10 citation statements)

References 18 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Besides a huge variety of different meta-heuristic optimization algorithms, also a large number of exotic benchmark problems and collections have been developed, and used in various competitions [40]- [49] Many of these benchmark problems are artificial-landscapebased problems, which are difficult to relate to real-world problem instances and vice versa [22], [50], [51].In many cases they are even difficult to relate to other synthetic problem instances. Also, from a theoretical perspective, commonly used benchmark problems have been criticized for regularities and simplicity [52], [53]. Even more fundamental issues are the substantial lack of generalization value [27] and the lack of systematism in state-of-the-art benchmark-based comparative assessments [24], [54].…”

Section: Benchmarking and Scientific Algorithm Performance Analysismentioning

confidence: 99%

Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges

Sala

Müller

2020

2020 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

Research on new optimization algorithms is often funded based on the motivation that such algorithms might improve the capabilities to deal with real-world and industrially relevant optimization challenges. Besides a huge variety of different evolutionary and metaheuristic optimization algorithms, also a large number of test problems and benchmark suites have been developed and used for comparative assessments of algorithms, in the context of global, continuous, and black-box optimization. For many of the commonly used synthetic benchmark problems or artificial fitness landscapes, there are however, no methods available, to relate the resulting algorithm performance assessments to technologically relevant real-world optimization problems, or vice versa. Also, from a theoretical perspective, many of the commonly used benchmark problems and approaches have little to no generalization value. Based on a mini-review of publications with critical comments, advice, and new approaches, this communication aims to give a constructive perspective on several open challenges and prospective research directions related to systematic and generalizable benchmarking for black-box optimization.

show abstract

Section: Benchmarking and Scientific Algorithm Performance Analysismentioning

confidence: 99%

Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges

Sala

Müller

2020

2020 IEEE Congress on Evolutionary Computation (CEC)

View full text Add to dashboard Cite

show abstract

“…However, the relevance of the resulting functions is debatable. Similar test function generators are described in [1].…”

Section: Related Workmentioning

confidence: 99%

“…e la er is the derived from Eq. (1). e sample that optimizes the in ll criterion will be evaluated by the objective function and the result is used to update the surrogate model.…”

Section: Performance Analysismentioning

confidence: 99%

Simulation-based test functions for optimization algorithms

Zaefferer

Fischbach

Naujoks

et al. 2017

Proceedings of the Genetic and Evolutionary Computation Conference

View full text Add to dashboard Cite

When designing or developing optimization algorithms, test functions are crucial to evaluate performance. O en, test functions are not su ciently di cult, diverse, exible or relevant to real-world applications. Previously, test functions with real-world relevance were generated by training a machine learning model based on real-world data. e model estimation is used as a test function. We propose a more principled approach using simulation instead of estimation. us, relevant and varied test functions are created which represent the behavior of real-world tness landscapes. Importantly, estimation can lead to excessively smooth test functions while simulation may avoid this pitfall. Moreover, the simulation can be conditioned by the data, so that the simulation reproduces the training data but features diverse behavior in unobserved regions of the search space. e proposed test function generator is illustrated with an intuitive, one-dimensional example. To demonstrate the utility of this approach it is applied to a protein sequence optimization problem. is application demonstrates the advantages as well as practical limits of simulation-based test functions. CCS CONCEPTS • eory of computation → Mathematical optimization; Gaussian processes; •Computing methodologies → Modeling and simulation;

show abstract

“…In [18] the topic of test function generators for assessing the performance of MHAs on multimodal functions was discussed. Authors highlighted, that many of the currently available test functions in the specialized literature are too simple, and show regularities such as: symmetry, uniform spacing of optima, and centered optima.…”

Section: Introductionmentioning

confidence: 99%

Global optimization test problems based on random field composition

2016

View full text Add to dashboard Cite

The development and identification of effective optimization algorithms for non-convex real-world problems is a challenge in global optimization. Because theoretical performance analysis is difficult, and problems based on models of real-world systems are often computationally expensive, several artificial performance test problems and test function generators have been proposed for empirical comparative assessment and analysis of metaheuristic optimization algorithms. These test problems however often lack the complex function structures and forthcoming difficulties that can appear in real-world problems. This communication presents a method to systematically build test problems with various types and degrees of difficulty. By weighted composition of parameterized random fields, challenging test functions with tunable function features such as, variance contribution distribution, interaction order, and nonlinearity can be constructed. The method is described, and its applicability to optimization performance analysis is described by means of a few basic examples. The method aims to set a step forward in the systematic generation of global optimization test problems, which could lead to a better understanding of the performance of optimization algorithms on problem types with particular characteristics. On request an introductive MATLAB implementation of a test function generator based on the presented method is available. This is the author's accepted manuscript version of an article published in: Sala R., Baldanzini N., Pierini M., (2017). Global optimization test problems based on random field composition, Optimization Letters 11(4) 699-713,The final version is available online at: http://dx.

show abstract

Test Function Generators for Assessing the Performance of PSO Algorithms in Multimodal Optimization

Cited by 12 publications

References 18 publications

Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges

Benchmarking for Metaheuristic Black-Box Optimization: Perspectives and Open Challenges

Simulation-based test functions for optimization algorithms

Global optimization test problems based on random field composition

Contact Info

Product

Resources

About