Evolving a Nelder–Mead Algorithm for Optimization with Genetic Programming

Abstract

We used genetic programming to evolve a direct search optimization algorithm, similar to that of the standard downhill simplex optimization method proposed by Nelder and Mead ( 1965 ). In the training process, we used several ten-dimensional quadratic functions with randomly displaced parameters and different randomly generated starting simplices. The genetically obtained optimization algorithm showed overall better performance than the original Nelder-Mead method on a standard set of test functions. We observed that many parts of the genetically produced algorithm were seldom or never executed, which allowed us to greatly simplify the algorithm by removing the redundant parts. The resulting algorithm turns out to be considerably simpler than the original Nelder-Mead method while still performing better than the original method.

Analytic Continued Fractions for Regression: A Memetic Algorithm Approach

“…To do these optimizations, we used a modified version of Nelder-Mead algorithm recently proposed by Fajfar et al (2016) to optimize the coefficients of a model. Nelder-Mead methods, also known as the downhill simplex algorithm, is a derivative-free nonlinear optimization algorithm known for its simplicity and relatively good empirical performance (Singer & Nelder, 2009).…”

“…While our memetic algorithm does individual search optimization with a modified Nelder-Mead algorithm (Fajfar et al, 2016), the nature of the nonlinear optimization problem indicates that more powerful solvers could later be used. We have chosen to start with a Nelder-Mead solver as a way of ensuring and promoting reproducibility, keeping the core memetic algorithm as simple as possible (Sun & Moscato, 2019) for this first in-depth test of performance of the new representation in real-world problems.…”

Grammatical Evolution-based Analog Circuit Synthesis

“…This is done using several manipulation techniques. Using experience and advice from other experiments [17,5,18] we take the best individuals from the current generation and move them into the next until we fill one tenth of it (so in our case of 300 individuals per generation we allow the best 30 individuals to proceed into the next one). Since we already evaluated these individuals, we will not need to do so again.…”

Hybrid evolutionary programming using adaptive Lévy mutation and modified Nelder–Mead method

“…The Nelder-Mead method is good at exploitation whereas EP is good at exploration. Therfore the combination of these two methods naturally has attracted researchers' interests [14,10,25,15,28]. Based on the Nelder-Mead method [30], we design an exploitation operator, called a modified Nelder-Nead exploitation operator.…”