Full-Body Postural Control of a Humanoid Robot with Both Imitation Learning and Skill Innovation

Abstract

In this paper, we present a novel methodology to obtain imitative and innovative postural movements in a humanoid based on human demonstrations in a di®erent kinematic scale. We collected motion data from a group of human participants standing up from a chair. Modeling the human as an actuated 3-link kinematic chain, and by de¯ning a multiobjective reward function of zero moment point and joint torques to represent the stability and e®ort, we computed reward pro¯les for each demonstration. Since individual reward pro¯les show vari-ability across demonstrating trials, the underlying state transition probabilities were modeled using a Markov chain. Based on the argument that the reward pro¯les of the robot should show the same temporal structure of those of the human, we used di®erential evolution to compute a trajectory that ¯ts all humanoid constraints and minimizes the di®erence between the robot reward pro¯le and the predicted pro¯le if the robot imitates the human. Therefore, robotic imitation involves developing a policy that results in a temporal reward structure, matching that of a group of human demonstrators across an array of demonstrations. Skill innovation was achieved by optimizing a signed reward error after imitation was achieved. Experimental results using the humanoid HOAP-3 are shown.

Behavior sequencing based on demonstrations: a case of a humanoid opening a door while walking

“…Furthermore, some recent studies suggested that the main difference between humans and chimpanzees is the ability of over-imitation [3]. Our proposal of using a reward profile to solve the correspondence problem in order to transfer a complex behavior from a human to a humanoid is based on these previous neuroscience works and previous experiments performed in a real humanoid standing up from a chair [17]. However, we are not sure of what is the internal objective function that the brain is optimizing Figure 10.…”

“…The generality comes from the definition of the reward profile. In fact, any behavior can be modeled, from simple ones as in [17] to complex behaviors. The preference in the selection of a predefined reward function over a learned function like in inverse reinforcement learning [29,30] does not affect the general idea of comparing the behavior of a human and a robot in a common domain, which is the reward domain.…”

“…This work is an improvement of previous papers [16,17]. In [16], we proposed to imitate a simple behavior like standing up from a chair using human demonstrations.…”

“…Once the candidate trajectory is generated and the behavior that the robot should use is known using (15), the associated reward is computed using (17). The optimization process is performed using Differential Evolution algorithm [25] with (11) and (12) as cost functions.…”

“…Using that idea, we proposed a method of imitation learning of a single behavior in a small humanoid robot using the reward as a common space of comparison [16] and later, we improved that idea by making a robot imitate a single human behavior and also innovate a new one which better fits its internal constraints and kinematic structure [17]. Starting from there, we extend the work by proposing in this paper a new human-robot LfD framework where a complex sequence of behaviors, which involves manipulation and locomotion, takes place.…”