Robot coverage control by evolved neuromodulation

Harrington, Kyle; Awa, Emmanuel; Cussat‐Blanc, Sylvain; Pollack, Jordan

doi:10.1109/ijcnn.2013.6706784

Cited by 7 publications

(8 citation statements)

References 39 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In this work we use an evolutionary algorithm to optimize gene-regulatory networks (GRNs) that dynamically tune the parameters of a reinforcement learning (RL) algorithm. This geneticallyregulated neuromodulation (GRNM) extends previous results that showed GRNM can enhance the learning of agents beyond traditional fixed parameter RL [15]. We apply GRNM to additional problem classes, and show that GRNs evolved Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.…”

Section: Introductionsupporting

confidence: 57%

“…An extensive review of computational models of neuromodulation can be found in [12], and some recent models are reviewed in [19]. In this study we extend work on the evolution of neuromodulation [15], focusing on the relationship between evolved neuromodulatory GRNs and reinforcement learning.…”

Section: Neuromodulationmentioning

confidence: 99%

“…Maze uses a discrete state-space, Parameter value Population size 500 Initial duplicates 19 Speciation threshold 0. 15 Species min size 15 Selection 3-player tournament Crossover rate 0.2 Mutation rate 0.8 Table 1: GRNEAT parameters used for evolution. and agents only receive a one-shot reward upon completion.…”

Section: Problemsmentioning

confidence: 99%

See 2 more Smart Citations

Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning

Cussat‐Blanc

Harrington

2015

Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

In this paper, we use a gene regulatory network (GRN) to regulate a reinforcement learning controller, the StateAction-Reward-State-Action (SARSA) algorithm. The GRN serves as a neuromodulator of SARSA's learning parameters: learning rate, discount factor, and memory depth. We have optimized GRNs with an evolutionary algorithm to regulate these parameters on specific problems but with no knowledge of problem structure. We show that geneticallyregulated neuromodulation (GRNM) performs comparably or better than SARSA with fixed parameters. We then extend the GRNM SARSA algorithm to multi-task problem generalization, and show that GRNs optimized on multiple problem domains can generalize to previously unknown problems with no further optimization.

show abstract

Section: Introductionsupporting

confidence: 57%

Section: Neuromodulationmentioning

confidence: 99%

See 1 more Smart Citation

Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning

Cussat‐Blanc

Harrington

2015

Proceedings of the 2015 Annual Conference on Genetic and Evolutionary Computation

Self Cite

View full text Add to dashboard Cite

show abstract

“…Recent work has begun to make a connection between learning and genetic regulation. In [64], the authors studied a robot navigation problem with a robot controlled by a temporal-difference reinforcement learning agent. By introducing a neuromodulatory system governed by a GRN to control the agent's learning and memory, the robot was able to outperform traditional reinforcement learning.…”

Section: Neuromodulationmentioning

confidence: 99%

Artificial Gene Regulatory Networks—A Review

2019

Self Cite

View full text Add to dashboard Cite

In nature, gene regulatory networks are a key mediator between the information stored in the DNA of living organisms (their genotype) and the structural and behavioral expression this finds in their bodies, surviving in the world (their phenotype). They integrate environmental signals, steer development, buffer stochasticity, and allow evolution to proceed. In engineering, modeling and implementations of artificial gene regulatory networks have been an expanding field of research and development over the past few decades. This review discusses the concept of gene regulation, describes the current state of the art in gene regulatory networks, including modeling and simulation, and reviews their use in artificial evolutionary settings. We provide evidence for the benefits of this concept in natural and the engineering domains.

show abstract

“…Now, GRNs are becoming ubiquitous models in artificial life and robotics. GRNs are the basis of a number of developmental models [3]- [5], as controllers of virtual and real robots [6]- [10], and neuromodulators of learning behavior [11], [12]. Other related methods of encoding reaction networks are commonly applied to similar problem domains [13]- [15].…”

mentioning

confidence: 99%

Gene Regulatory Network Evolution Through Augmenting Topologies

Cussat‐Blanc

Harrington

Pollack

2015

IEEE Trans. Evol. Computat.

Self Cite

View full text Add to dashboard Cite

Artificial gene regulatory networks (GRNs) are biologically inspired dynamical systems used to control various kinds of agents, from the cells in developmental models to embodied robot swarms. Most recent work uses a genetic algorithm (GA) or an evolution strategy in order to optimize the network for a specific task. However, the empirical performances of these algorithms are unsatisfactory. This paper presents an algorithm that primarily exploits a network distance metric, which allows genetic similarity to be used for speciation and variation of GRNs. This algorithm, inspired by the successful neuroevolution of augmenting topologies algorithm's use in evolving neural networks and compositional pattern-producing networks, is based on a specific initialization method, a crossover operator based on gene alignment, and speciation based upon GRN structures. We demonstrate the effectiveness of this new algorithm by comparing our approach both to a standard GA and to evolutionary programming on four different experiments from three distinct problem domains, where the proposed algorithm excels on all experiments.

show abstract

Robot coverage control by evolved neuromodulation

Cited by 7 publications

References 39 publications

Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning

Genetically-regulated Neuromodulation Facilitates Multi-Task Reinforcement Learning

Artificial Gene Regulatory Networks—A Review

Gene Regulatory Network Evolution Through Augmenting Topologies

Contact Info

Product

Resources

About