Leveraging Contact Forces for Learning to Grasp

Abstract

Grasping objects under uncertainty remains an open problem in robotics research. This uncertainty is often due to noisy or partial observations of the object pose or shape. To enable a robot to react appropriately to unforeseen effects, it is crucial that it continuously takes sensor feedback into account. While visual feedback is important for inferring a grasp pose and reaching for an object, contact feedback offers valuable information during manipulation and grasp acquisition. In this paper, we use model-free deep reinforcement learning to synthesize control policies that exploit contact sensing to generate robust grasping under uncertainty. We demonstrate our approach on a multi-fingered hand that exhibits more complex finger coordination than the commonly used twofingered grippers. We conduct extensive experiments in order to assess the performance of the learned policies, with and without contact sensing. While it is possible to learn grasping policies without contact sensing, our results suggest that contact feedback allows for a significant improvement of grasping robustness under object pose uncertainty and for objects with a complex shape.

Improved Learning of Robot Manipulation Tasks Via Tactile Intrinsic Motivation

“…Our main insight is that sense of touch can guide robots to take actions that result in manipulation of an object. To implement the sense of touch, we extend our state-space with force measurements from tactile sensors positioned at the endeffector [13], [14], [15]. We introduce a touch-based intrinsic reward function to direct exploration towards the states where the robot touches the object.…”

“…It is rich in information and free of external disturbances, compared to visual sensing that is influenced by occlusions and poor lighting conditions. While the majority of DRL works do not consider tactile feedback for learning robotic manipulation tasks, a few have shown the benefits and potential of considering this additional sensory modality [13], [14], [15], [18], [19].…”

“…We extend our state space with force measurements from tactile sensors positioned at the end-effector [13], [14], [15]. The original state space consists of a mixture between proprioceptive state information, i.e., joint positions and joint velocities, and relevant information about the manipulation object, such as the object's velocity and the position x obj , which we will use in the reward function.…”

“…In our method (see Figure 1), we add force sensors to the end-effector and extend the state space with force measurements, similarly to other work [13], [14], [15]. We then go beyond prior work and introduce an intrinsic reward to guide the exploration.…”

Review of Deep Reinforcement Learning-Based Object Grasping: Techniques, Open Challenges, and Recommendations

“…Recently, learning physical object representations for robot manipulation has been performed based on DensePhysNet system that actively executes a sequence of dynamic interactions (e.g., sliding and colliding) [284]. Also, Merzic et al [285] have attempted to generate robust grasping under uncertainty based on synthesized control policies that exploit contact sensing, where they have utilized model-free deep reinforcement learning with exploiting Trust Region Policy Optimization (TRPO). Learningbased approaches to grasp planning are preferred over analytical methods due to their ability to better generalize to new, partially observed objects.…”