Robust optimization through neuroevolution

Abstract

We propose a method for evolving neural network controllers robust with respect to variations of the environmental conditions (i.e. that can operate effectively in new conditions immediately, without the need to adapt to variations). The method specifies how the fitness of candidate solutions can be evaluated, how the environmental conditions should vary during the course of the evolutionary process, which algorithm can be used, and how the best solution can be identified. The obtained results show how the method proposed is effective and computational tractable. It allows to improve performance on an extended version of the double-pole balancing problem, outperform the best available human-designed controllers on a car racing problem, and generate effective solutions for a swarm robotic problem. The comparison of different algorithms indicates that the CMA-ES and xNES methods, that operate by optimizing a distribution of parameters, represent the best options for the evolution of robust neural network controllers.

Efficacy of Modern Neuro-Evolutionary Strategies for Continuous Control Optimization

“…The state of the sensors, the activation of the neural network, the force applied to the cart, and the position and velocity of the cart and of the poles are updated every 0.02 s. Unlike the MuJoCo locomotion tasks, this problem necessarily requires memory. For more details, see Pagliuca et al ( 2018 ) and Pagliuca and Nolfi ( 2019 ).…”

“…The seventh problem is the Swarm foraging problem (Pagliuca and Nolfi, 2019 ) in which a group of 10 simulated MarXbots (Bonani et al, 2010 ) should explore their environment so to maximize the number of food elements collected and transported to a nest. The robots are located in a flat square arena of 5 × 5 m, surrounded by walls, which contains a nest, i.e., a circular area with diameter of 0.8 m painted in gray.…”

“…The action state includes four values encoding the desired speeds of the left and right robot's wheels, and the state of the blue and red LEDs located on the frontal and rear side of the robot. For other details, see Pagliuca and Nolfi ( 2019 ).…”

“…Unlike the MuJoCo locomotion tasks, this problem necessarily requires memory. For more details, see Pagliuca, Milano and Nolfi (2018) and Pagliuca and Nolfi (2019).…”

Adaptability of Improved NEAT in Variable Environments

“…In these tests, NEAT has been tested against newer GAs, which always perform better in these adaptability problems. This is concerning for the relevance of NEAT, as if there are better GAs, no one will use NEAT as an effective GA [9,10,12].…”