Robots that can adapt like animals

Abstract

As robots leave the controlled environments of factories to autonomously function in more complex, natural environments 1,2,3 , they will have to respond to the inevitable fact that they will become damaged 4,5 . However, while animals can quickly adapt to a wide variety of injuries, current robots cannot "think outside the box" to find a compensatory behavior when damaged: they are limited to their pre-specified selfsensing abilities, can diagnose only anticipated failure modes 6 , and require a pre-programmed contingency plan for every type of potential damage, an impracticality for complex robots 4,5 . Here we introduce an intelligent trial and error algorithm that allows robots to adapt to damage in less than two minutes, without requiring self-diagnosis or pre-specified contingency plans. Before deployment, a robot exploits a novel algorithm to create a detailed map of the space of high-performing behaviors: This map represents the robot's intuitions about what behaviors it can perform and their value. If the robot is damaged, it uses these intuitions to guide a trial-and-error learning algorithm that conducts intelligent experiments to rapidly discover a compensatory behavior that works in spite of the damage. Experiments reveal successful adaptations for a legged robot injured in five different ways, including damaged, broken, and missing legs, and for a robotic arm with joints broken in 14 different ways. This new technique will enable more robust, effective, autonomous robots, and suggests principles that animals may use to adapt to injury.Robots have transformed the economics of many industries, most notably manufacturing 7 , and have the power to deliver tremendous benefits to society, such as in search and rescue 8 , disaster response 9 , health care 3 , and transportation 10 . They are also invaluable tools for scientific exploration, whether of distant planets 1,4 or deep oceans 2 . A major obstacle to their widespread adoption in more complex environments outside of factories is their fragility 4,5 : Robots presently pale in comparison to natural animals in their ability to invent compensatory behaviors after an injury (Fig. 1A).Current damage recovery in robots typically involves two phases: self-diagnosis, and then selecting the best, pre-designed contingency plan 11,12,13,14 . Such self-diagnosing robots are expensive, because self-monitoring sensors are expensive, and are difficult to design, because robot engineers cannot foresee every possible situation: this approach often fails either because the diagnosis is incorrect 12,13 or because an appropriate contingency plan is not provided 14 . Injured animals respond differently: they learn by trial and error how to compensate for damage (e.g. learning which limp minimizes pain) 15,16 . Similarly, trial-and-error learning algorithms could allow robots to creatively discover compensatory behaviors without being limited to their designers' assumptions about how damage may occur and how to compensate for each damage type. However, state-of-the-art ...

Behavioral Diversity Generation in Autonomous Exploration through Reuse of Past Experience

“…In practice, in the experimental contexts considered in this paper, the performance and robustness of this model is competitive with more complex approaches. This inverse model is not completely unreasonable in biological organisms (Loeb, 2012), and that related algorithms implemented as part of the SAGG-Random architecture such as in Baranes and Oudeyer (2013) and Moulin-Frier et al (2014), as well as other variations such as the MAP-Elite algorithm (Cully et al, 2015), have yielded good results in diverse robotics contexts.…”

“…Motor motions can of course be part of behavior and contribute to diversity, when adequate sensors and sensory primitives are used to encode them in S. This is, for instance, the case in the behavioral descriptors used by the MAP-Elites algorithms (Cully et al, 2015).…”

“…Goal-directed exploration (Oudeyer and Kaplan, 2007) implementations have been proposed in Baranes and Oudeyer (2010), Jamone et al (2011), andRolf et al (2011), and as part of the SAGG-RIAC architecture (Baranes and Oudeyer, 2013) and have been shown to be effective in exploring sensorimotor spaces with large motor spaces. These methods for goal babbling as well as related methods such as MAP-Elite (Cully et al, 2015) have also been shown to efficiently generate forms of behavioral diversity.…”

“…Goal-directed exploration (Oudeyer and Kaplan, 2007;Baranes and Oudeyer, 2010;Jamone et al, 2011;Rolf et al, 2011;Baranes and Oudeyer, 2013), as well as related methods such as MAPElite (Cully et al, 2015), has been shown to be effective at creating behavioral diversity in large sensorimotor spaces. However, these methods only consider a single task.…”

Abstraction as a Mechanism to Cross the Reality Gap in Evolutionary Robotics

“…With the use of a closed loop low-level control system, evolution would abstract away from the low-level sensor inputs and actuator outputs. Some resent work has shown promising results in bridging the reality gap using abstracted methods [5,7]. This however may come at a price, as abstraction "hides" the properties of the raw sensory inputs and motor actions to the controller, it may have as a disadvantage that the potential for SMC by the robot is reduced.…”