Active vision for dexterous grasping of novel objects

Abstract

How should a robot direct active vision so as to ensure reliable grasping? We answer this question for the case of dexterous grasping of unfamiliar objects. By dexterous grasping we simply mean grasping by any hand with more than two fingers, such that the robot has some choice about where to place each finger. Such grasps typically fail in one of two ways, either unmodeled objects in the scene cause collisions or object reconstruction is insufficient to ensure that the grasp points provide a stable force closure. These problems can be solved more easily if active sensing is guided by the anticipated actions. Our approach has three stages. First, we take a single view and generate candidate grasps from the resulting partial object reconstruction. Second, we drive the active vision approach to maximise surface reconstruction quality around the planned contact points. During this phase, the anticipated grasp is continually refined. Third, we direct gaze to improve the safety of the planned reach to grasp trajectory. We show, on a dexterous manipulator with a camera on the wrist, that our approach (80.4% success rate) outperforms a randomised algorithm (64.3% success rate).

Multi-View Picking: Next-best-view Reaching for Improved Grasping in Clutter

“…For example, ten Pas et al [2] showed that computing grasp poses using a fused point cloud from many viewpoints along a predefined trajectory resulted in a 9% increase in grasp success rate compared to using a point cloud collected from two static cameras. Rather than rely on a fixed data collection routine, Arruda et al [3] use an active perception approach to choose viewpoints which specifically aid point cloud reconstruction near potential finger contact points in an efficient manner.…”

“…1). Unlike previous works in active perception for grasping which employ objectspecific heuristics [4] or a secondary task such as point cloud reconstruction [2,3,9], our approach directly uses entropy in the grasp pose estimation to influence control.…”

2.5D Image-based Robotic Grasping

“…Current work mainly concerns about computer vision data. FCNs and CNNs have been applied successfully to the grasp prediction [2] [8] [14] [15] based on 2D or 3D (including 2D+depth image or pointcloud) computer vision. Furthermore, heterogeneous sensor modalities are popular these years.…”

Viewpoint optimization for aiding grasp synthesis algorithms using reinforcement learning

“…On the other hand, very few works utilize active vision that use partial object data. In [31], active vision is used to refine the surface reconstruction of the grasp location candidates, which results in a more reliable grasp execution. [32] addresses the problem of grasping unknown objects in the presence of occlusion.…”