Three-dimensional object reconstruction of symmetric objects by fusing visual and tactile sensing

Abstract

In this work, we propose to reconstruct a complete three-dimensional (3-D) model of an unknown object by fusion of visual and tactile information while the object is grasped. Assuming the object is symmetric, a first hypothesis of its complete 3-D shape is generated. A grasp is executed on the object with a robotic manipulator equipped with tactile sensors. Given the detected contacts between the fingers and the object, the initial full object model including the symmetry parameters can be refined. This refined model will then allow the planning of more complex manipulation tasks. The main contribution of this work is an optimal estimation approach for the fusion of visual and tactile data applying the constraint of object symmetry. The fusion is formulated as a state estimation problem and solved with an iterated extended Kalman filter. The approach is validated experimentally using both artificial and real data from two different robotic platforms.

Ergodic Exploration Using Binary Sensing for Nonparametric Shape Estimation

“…Performance for estimation in R 3 depends on the robot dynamical constraints as well as the sensor area coverage. 1,2,6,11,30. Trajectories start at the magenta square.…”

“…T ACTILE sensing is often associated with shape estimation [1]- [4] and mapping problems [5] in conditions where visual sensing may be limited. In some instances, tactile sensing is used to supplement vision-based sensing to improve shape estimates [6], [7]. The richness of touch as a sensing modality is underscored by the development of novel tactile sensors [8]- [11] for use in a myriad of applications ranging from robot-assisted tumor detection [7], to texture recognition, and feature localization [12], [13].…”

AGILITY - Dynamic full body locomotion and manipulation with autonomous legged robots

“…Furthermore, MPI-IS studied how different sensor modalities can be exploited or fused for increasing the robustness of grasping [37,38,42]. This research has been evaluated on different manipulation platforms.…”

Tactile Mapping and Localization from High-Resolution Tactile Imprints

“…However, in the context of robotic manipulation, occlusions are often unavoidable specially when the robot actively interacts and manipulates objects. To mitigate this problem, some works have explored tactile sensing to better estimate object properties or motion [10,11,12,13,14]. Most of these works heavily rely on visual information and tend to use tactile information for some form of contact detection.…”