First-person tele-operation of a humanoid robot

Abstract

Abstract-Remote control of robots is often necessary to complete complex unstructured tasks in environments that are inaccessible (e.g. dangerous) for humans. Tele-operation of humanoid robots is often performed trough motion tracking to reduce the complexity deriving from manually controlling a high number of DOF. However, most commercial motion tracking apparatus are expensive and often uncomfortable. Moreover, a limitation of this approach is the need to maintain visual contact with the operated robot, or to employ a second human operator to independently maneuver a camera. As a result, even performing simple tasks heavily depends on the skill and synchronization of the two operators. To alleviate this problem we propose to use augmented-reality to provide the operator with first-person vision and a natural interface to directly control the camera, and at the same time the robot. By integrating recent off-the-shelf technologies, we provide an affordable and intuitive environment composed of Microsoft Kinect, Oculus Rift and haptic SensorGlove to tele-operate in first-person humanoid robots. We demonstrate on the humanoid robot iCub that this set-up allows to quickly and naturally accomplish complex tasks.

Deep Imitation Learning for Complex Manipulation Tasks from Virtual Reality Teleoperation

“…On the other hand, Virtual Reality teleoperation allows for a direct mapping of observations and actions between the teacher and the robot and does not suffer from the above correspondence issues [3], while also leveraging the natural manipulation instincts that the human teacher possesses. In a non-learning setting, VR teleoperation has been recently explored for controlling humanoid robots [40], [41], [42], for simulated dexterous manipulation [43], and for communicating motion intent [44]. Existing use cases of VR for learning policies have so far been limited to collecting waypoints of low-dimensional robot states [45], [46].…”

Scaling Robot Supervision to Hundreds of Hours with RoboTurk: Robotic Manipulation Dataset through Human Reasoning and Dexterity

“…The phone transmits its changes in position and its current absolute orientation over WebRTC to the teleoperation server. We additionally provide users with the ability to enable and disable control of the robot so that users can re-position their arm and phone for better ergonomics similar in nature to interfaces for surgical robots [9]. Coordination Server.…”

Robust Real-Time Whole-Body Motion Retargeting from Human to Humanoid

“…The first online retargeting works were for the majority related exclusively to upper body movements [15], [16], [17]. Frequently, the retargeting of the upper-body joints, which is important for manipulation, is done independently from the one of the legs, crucial for balancing and locomotion [10], [18].…”