-Intrinsically Motivated Reinforcement Learning (IMRL) has been proposed as a framework within which agents exploit "internal reinforcement" to acquire general-purpose building-block behaviors ("skills") which can be later combined for solving several specific tasks. The architectures so far proposed within this framework are limited in that: (1) they use hardwired "salient events" to form and train skills, and this limits agents' autonomy; (2) they are applicable only to problems with abstract states and actions, as grid-world problems. This paper proposes solutions to these problems in the form of a hierarchical reinforcement-learning architecture that: (1) exploits Evolutionary Robotics techniques so to allow the system to autonomously discover "salient events"; (2) uses neural networks so to allow the system to cope with continuous states and noisy environments. The viability of the proposed approach is demonstrated with a simulated robotic scenario.
Abstract. Studying the role played by evolution and learning in adaptive behavior is a very important topic in artificial life research. This paper investigates the interplay between learning and evolution when agents have to solve several different tasks, as it is the case for real organisms but typically not for artificial agents. Recently, an important thread of research in machine learning and developmental robotics has begun to investigate how agents can solve different tasks by composing general skills acquired on the basis of internal motivations. This work presents a hierarchical, neural-network, actor-critic architecture designed for implementing this kind of intrinsically motivated reinforcement learning in real robots. We compare the results of several experiments in which the various components of the architecture are either trained during lifetime or evolved through a genetic algorithm. The most important results show that systems using both evolution and learning outperform systems using either one of the two, and that, among the former, systems evolving internal reinforcers for learning building-block skills have a higher evolvability than those directly evolving the related behaviors.
scite is a Brooklyn-based organization that helps researchers better discover and understand research articles through Smart Citations–citations that display the context of the citation and describe whether the article provides supporting or contrasting evidence. scite is used by students and researchers from around the world and is funded in part by the National Science Foundation and the National Institute on Drug Abuse of the National Institutes of Health.