Learning From Humans How to Grasp: A Data-Driven Architecture for Autonomous Grasping With Anthropomorphic Soft Hands

Abstract

Soft hands are robotic systems that embed compliant elements in their mechanical design. This enables an effective adaptation with the items and the environment, and ultimately, an increase in their grasping performance. These hands come with clear advantages in terms of ease-to-use and robustness if compared with classic rigid hands, when operated by a human. However, their potential for autonomous grasping is still largely unexplored, due to the lack of suitable control strategies. To address this issue, in this letter, we propose an approach to enable soft hands to autonomously grasp objects, starting from the observations of human strategies. A classifier realized through a deep neural network takes as input the visual information on the object to be grasped, and predicts which action a human would perform to achieve the goal. This information is hence used to select one among a set of human-inspired primitives, which define the evolution of the soft hand posture as a combination of anticipatory action and touch-based reactive grasp. The architecture is completed by the hardware component, which consists of an RGB camera to look at the scene, a 7-DoF manipulator, and a soft hand. The latter is equipped with inertial measurement units at the fingernails for detecting contact with the object. We extensively tested the proposed

Hands in the Real World

“…These methods help to overcome the more narrow grasping primitives used as kinematic reference during the design phase of the hand (Bonilla et al, 2014 ). Such opportunities can emerge from both the observation of humans performing actions with robotic hands and data driven methods (Bonilla et al, 2014 ; Pacchierotti et al, 2014 ; Della Santina et al, 2019 ). Enabling touch sensing capabilities additionally creates new avenues of control, playing, in such a way, an important role as in humans.…”

Modeling Human Motor Skills to Enhance Robots’ Physical Interaction

“…This uncertainty can be naturally compensated by the ability of soft hands to locally adapt to unknown environments. Following this approach, part of our effort has been devoted to the development of a human inspired multi-modal, multilayer architecture that combines feedforward components, predicted by a Deep Neural Network, with reactive sensor-triggered actions (more details in [5]). Humans are able to accomplish very complex grasps by employing a vast range of different strategies [7].…”

“…Technical details on training and validation are here omitted, the interested reader is invited to refer to [5].…”

Soft-Grasping With an Anthropomorphic Robotic Hand Using Spiking Neurons

“…An anthropomorphic robotic hand enables further investigation of the neural response of grasping motions [15], or evaluation of the different affordances [16], or the use of synergies from human demonstration for grasping control [17]. Other approaches explore human-like grasping using deep learning (DL) to create representations using autoencoders [18], for extensive training in simulation for in-hand manipulation [19], or with a combination of an object classifier with reactive and anticipatory motor primitives [20]. But despite recent successes, DL also has some drawbacks.…”