Real-time imitation of human whole-body motions by humanoids

Abstract

Abstract-In this paper, we present a system that enables humanoid robots to imitate complex whole-body motions of humans in real time. In our approach, we use a compact human model and consider the positions of the endeffectors as well as the center of mass as the most important aspects to imitate. Our system actively balances the center of mass over the support polygon to avoid falls of the robot, which would occur when using direct imitation. For every point in time, our approach generates a statically stable pose. Hereby, we do not constrain the configurations to be in double support. Instead, we allow for changes of the support mode according to the motions to imitate. To achieve safe imitation, we use retargeting of the robot's feet if necessary and find statically stable configurations by inverse kinematics. We present experiments using human data captured with an Xsens MVN motion capture system. The results show that a Nao humanoid is able to reliably imitate complex whole-body motions in real time, which also include extended periods of time in single support mode, in which the robot has to balance on one foot.

Adaptive whole-body manipulation in human-to-humanoid multi-contact motion retargeting

“…However, our algorithm is compatible with real-time tracking, as long as there is a mechanism for contact event detection. One caveat is that (as mentioned in [15]), contact and CoM stabilization add delay to the tracking, so movements with contact changes occurring in rapid succession may be difficult to track. However, we can compensate for the delay by "buffering" the human demonstration trajectory.…”

“…We formulate a generalization of the method proposed in Koenemann et al [15], where the robot tracks a normalized offset (denoted as φ) between its support surfaces. We define a support point p i as the center of a support surface that is in contact with the fixed environment.…”

“…In existing motion retargeting methods, the dynamics of manipulated objects are assumed to be external perturbations [14] or negligible [15] (note that in [14] the character is rooted to the ground and hence the manipulated objects do not affect its balance). Vaillant et al [16] introduced the idea of incorporating external objects and other characters into a single QP formulation, which allows explicit optimization over physically and behaviorally coupled systems.…”

First-person tele-operation of a humanoid robot

“…The NAO and the Kinect are also used in [5] to record movement and reproduce it in order to support exercise therapy. Another tele-operation project is [6] which uses a full body motion tracking suit and the NAO for teleoperation tasks. In [7] it is introduced an approach which uses a full body motion tracking suit for tele-operation while using machine learning to train neural networks to map sensor data to joint space.…”

A Balance Feedback Human Machine Interface for humanoid teleoperation in dynamic tasks

“…However none can provide the necessary feedback that triggers the human operator's awareness of complete whole-body synchronization. Common policies explore the use of Motion Capture Systems (MoCap) in order to track body posture and control the robotic platform [3], [4]. Other strategies focus on transmitting simple high-level commands that are translated into end-effector (hands, feet or limbs) coordinates [5].…”