Localizing the object contact through matching tactile features with visual map

Abstract

Abstract-This paper presents a novel framework for integration of vision and tactile sensing by localizing tactile readings in a visual object map. Intuitively, there are some correspondences, e.g., prominent features, between visual and tactile object identification. To apply it in robotics, we propose to localize tactile readings in visual images by sharing same sets of feature descriptors through two sensing modalities. It is then treated as a probabilistic estimation problem solved in a framework of recursive Bayesian filtering. Feature-based measurement model and Gaussian based motion model are thus built. In our tests, a tactile array sensor is utilized to generate tactile images during interaction with objects and the results have proven the feasibility of our proposed framework.

Tactile Object Recognition with Semi-Supervised Learning

“…7 that shows an overall accuracy of 83.3% was obtained. It can be noticed that most objects can be recognized successfully; however, some are still assigned to wrong labels, especially those with similar local shapes, e.g., different pliers (6,7,8) or scissors (11,12). These objects are expected to be further classified by utilizing distribution of their local features in future work.…”

“…In the view of hardware, multiple tactile sensors have been developed using various sensing principles [1]- [5], e.g., capacitive, piezoelectric, optical, resistive and ultrasonic etc. In the meantime, some researchers contributed to decoding the conveyed message by the tactile sensors to retrieve the information of interacted object, such as contact locations [6], [7], slippage [8], object pose [9] and surface textures [10]- [12]. As defined in [13], the object characteristics can mainly be divided into two categories, i.e., material and geometric properties.…”

Object exploration using vision and active touch

“…Early work in the field focussed on using passive stereo vision and active touch to explore object contours and features [1]- [3]. Following work has combined touch and 3D vision to rapidly label surface features [17], map object surfaces [18], match tactile features to visual maps [19] and nest visual and tactile control loops to improve surface exploration [20], [21].…”

Recent progress on tactile object recognition

“…In the study by Prats et al, 53 the author developed a system which integrates the visual, tactile, and force to perform the task of complicated opening doors. Luo et al 54 investigated the correspondence between visual and tactile features.…”