How robot morphology and training order affect the learning of multiple behaviors

Abstract

Abstract-Automatically synthesizing behaviors for robots with articulated bodies poses a number of challenges beyond those encountered when generating behaviors for simpler agents. One such challenge is how to optimize a controller that can orchestrate dynamic motion of different parts of the body at different times. This paper presents an incremental shaping method that addresses this challenge: it trains a controller to both coordinate a robot's leg motions to achieve directed locomotion toward an object, and then coordinate gripper motion to achieve lifting once the object is reached. It is shown that success is dependent on the order in which these behaviors are learned, and that despite the fact that one robot can master these behaviors better than another with a different morphology, this learning order is invariant across the two robot morphologies investigated here. This suggests that aspects of the task environment, learning algorithm or the controller dictate learning order more than the choice of morphology.

A hox gene inspired generative approach to evolving robot morphology

“…The need for more advanced skills is gradually introduced, and in this aspect the evaluation is conceptually similar to the scaffolding schedules used in [2].…”

Beyond black-box optimization: a review of selective pressures for evolutionary robotics

“…These authors called these approaches "behavioral complexification" [74,128], "behavior chaining" [14], "dynamic scaffolding" [18] or "incremental shaping" [3,18]. Gomez et al [74] thus worked on a prey-capture task that was parameterized with the prey speed and the delay before starting the pursuit; they defined ten ordered sub-tasks of increasing difficulty.…”

“…Gomez et al [74] thus worked on a prey-capture task that was parameterized with the prey speed and the delay before starting the pursuit; they defined ten ordered sub-tasks of increasing difficulty. Bongard et al investigated automatic difficulty tuning with a task in which a simulated legged robot has to grab an object and lift it [14,3,19,18]. They proposed two different algorithms: a hill climber in which difficulty is decreased when too many individuals fail and increased when they often succeed [14,3], and a variant of the Age-Layered Population Structure (ALPS) algorithm [86,84,21,18].…”

“…Bongard et al investigated automatic difficulty tuning with a task in which a simulated legged robot has to grab an object and lift it [14,3,19,18]. They proposed two different algorithms: a hill climber in which difficulty is decreased when too many individuals fail and increased when they often succeed [14,3], and a variant of the Age-Layered Population Structure (ALPS) algorithm [86,84,21,18]. Bongard et al highlighted that the order in which behaviors are evolved critically impacts the performance of the evolutionary process [14,3].…”

“…They proposed two different algorithms: a hill climber in which difficulty is decreased when too many individuals fail and increased when they often succeed [14,3], and a variant of the Age-Layered Population Structure (ALPS) algorithm [86,84,21,18]. Bongard et al highlighted that the order in which behaviors are evolved critically impacts the performance of the evolutionary process [14,3]. They also noted that starting with changes in the morphology (morphological scaffolding) can synergize with changes in the environment (environmental scaffolding), but only if they occur in this specific order [19].…”