A Modular Software Framework for Eye–Hand Coordination in Humanoid Robots

Abstract

We describe our software system enabling a tight integration between vision and control modules on complex, high-DOF humanoid robots. This is demonstrated with the iCub humanoid robot performing visual object detection and reaching and grasping actions. A key capability of this system is reactive avoidance of obstacle objects detected from the video stream while carrying out reach-and-grasp tasks. The subsystems of our architecture can independently be improved and updated, for example, we show that by using machine learning techniques we can improve visual perception by collecting images during the robot's interaction with the environment. We describe the task and software design constraints that led to the layered modular system architecture.

Enhancing a Neurocognitive Shared Visuomotor Model for Object Identification, Localization, and Grasping With Learning From Auxiliary Tasks

“…Conventional approaches for visuomotor applications like vision-based grasping are usually modular. Each module is independent and expert-designed, e.g., [24]. Developmental robotics takes a different approach, relying on emerging abilities and learning of complex skills through interaction with .…”

Machine Learning and Cognitive Robotics: Opportunities and Challenges

“…Besides, there are several platforms for humanoids and other robots in studies reported in ref. [42][43][44]. Details of Pioner3-AT bender robotic platform are provided in ref.…”

Enhancing a Neurocognitive Shared Visuomotor Model for Object Identification, Localization, and Grasping With Learning From Auxiliary Tasks

“…Conventional approaches for visuomotor applications like vision-based grasping are usually modular. Each module is independent and expert-designed, e.g., [24]. Developmental robotics takes a different approach, relying on emerging abilities and learning of complex skills through interaction with Fig.…”