Morpho Evolution With Learning Using a Controller Archive as an Inheritance Mechanism

Abstract

Most work in evolutionary robotics centres on evolving a controller for a fixed body-plan. However, previous studies suggest that simultaneously evolving both controller and bodyplan could open up many interesting possibilities. However, the joint optimisation of body-plan and control via evolutionary processes can be challenging in rich morphological spaces. This is because offspring can have body-plans that are very different from either of their parents, leading to a potential mismatch between the structure of an inherited neural controller and the new body. To address this, we propose a framework that combines an evolutionary algorithm to generate body-plans and a learning algorithm to optimise the parameters of a neural controller. The topology of this controller is created once the body-plan of each offspring has been generated. The key novelty of the approach is to add an external archive for storing learned controllers that map to explicit 'types' of robots (where this is defined with respect to the features of the body-plan). By initiating learning from a controller with an appropriate structure inherited from the archive, rather than from a randomly initialised one, we show that both the speed and magnitude of learning increases over time when compared to an approach that starts from scratch, using two tasks and three environments. The framework also provides new insights into the complex interactions between evolution and learning.

The Effects of Learning in Morphologically Evolving Robot Systems

“…Similar research has been done recently with evolvable morphologies ( Cheney et al, 2014 ; Lipson et al, 2016 ; Nygaard et al, 2017 ; Cheney et al, 2018 ; Nygaard et al, 2018 ; Wang et al, 2019 ; Liao et al, 2019 ; De Carlo et al, 2020 ; Zhao et al, 2020 ; Goff et al, 2021 ; Gupta et al, 2021 ; Medvet et al, 2021 ; Pagliuca and Nolfi, 2021 ). Among all the studies, several approaches have been proposed to mitigate the body and brain mismatch effect on the population.…”

“… Goff et al (2021) proposed the use of an external archive to store previously discovered controllers which are the highest-performing controllers found for the type. Then they provide learning on these controllers.…”

“…The bodies are encoded in a Compositional Pattern Producing Network (CPPN) which was introduced in ( Stanley, 2007 ). It has been demonstrated that with this encoding it is possible to evolve complex patterns for a variety of tasks ( Clune et al, 2009 ), ( Haasdijk et al, 2010 ), ( Clune and Lipson, 2011 ), ( Auerbach and Bongard, 2014 ), ( Goff et al, 2021 )].…”

Lamarck's Revenge: Inheritance of Learned Traits Can Make Robot Evolution Better

“…Previous research on artificial evolution combined with learning is mostly limited to Darwinian systems [ 14,32,35,51 ] and the Baldwin effect [ 28,29 ]. The few existing studies on Lamarckian evolution can be divided into three categories.…”

Recombination and Novelty in Neuroevolution: A Visual Analysis

“…NEAT is, therefore, a complex algorithmic system. Successful applications of NEAT include evolving neural network controllers for robots [29,37], evolving both controllers and morphology [3,11], and evolving innovative video game content [12,34]. Neuroevolution has also been used in biochemistry [9], geosciences [36], and to address open questions in natural evolution [5].…”