Mind the gap - robotic grasping under incomplete observation

Abstract

We consider the problem of grasp and manipulation planning when the state of the world is only partially observable. Specifically, we address the task of picking up unknown objects from a table top. The proposed approach to object shape prediction aims at closing the knowledge gaps in the robot's understanding of the world. A completed state estimate of the environment can then be provided to a simulator in which stable grasps and collision-free movements are planned.The proposed approach is based on the observation that many objects commonly in use in a service robotic scenario possess symmetries. We search for the optimal parameters of these symmetries given visibility constraints. Once found, the point cloud is completed and a surface mesh reconstructed.Quantitative experiments show that the predictions are valid approximations of the real object shape. By demonstrating the approach on two very different robotic platforms its generality is emphasized.

Xtru3D: Single-View 3D Object Reconstruction from Color and Depth Data

“…and the standard deviation is 0.96mm. Taking into account that the objects are similar to the set used in [7], our extrusion approach provides a similar e ectiveness in comparison with earlier symmetry method and a signi cant improvement for large objects. With our method, the mean error is less than 5mm in all objects, independently of their size while in [7] the average error is less than 7mm…”

“…The processing time of the whole algorithm is currently less than 2 seconds on a 2Ghz computer for a point cloud with less than 30000 points, signi cantly improving computation time achieved in [7] with a similar number of points.…”

“…Second, we propose a re nement step that takes advantage of the complementarity of the depth and color images by carefully initializing a graph-cut based color segmentation with the depth data. Finally, a quantitative evaluation of the accuracy of the reconstructed meshes is performed on a set of 12 common use objects, showing that the e ectiveness of our proposed method is comparable to the most recent approach using symmetries [7]. Some preliminary experiments for grasping applications are also conducted using the OpenRAVE simulator [10].…”

Dexterous grasping under shape uncertainty

“…4(e)-(h), when the training data points are sampled from the whole object point cloud, the variance or shape uncertainty is generally very small on the whole surface except the parts with sparse or with even no data points, such as the bottom of the jug. In robotic grasping tasks, due to the occlusion [48,49] or non-reachability from tactile exploration [21], it is usually the case that some parts of the object are not perceivable and point-clouds exhibit holes. To evaluate our method under missing data points, we use MeshLab 3 to simulate partial view of point clouds with a fixed camera view, and then obtain object point cloud from that virtual camera.…”

“…Few works have considered object shape uncertainty by integrating planning and control. However, in real robotic grasping tasks due to the, for example, occlusion problems [48,49,22] or non-reachability from tactile exploration [21], object shape uncertainty is inevitable. In this work, we thus integrate object uncertainty in grasp planning and control.…”