Using 3D Convolutional Neural Networks for Tactile Object Recognition with Robotic Palpation

Abstract

In this paper, a novel method of active tactile perception based on 3D neural networks and a high-resolution tactile sensor installed on a robot gripper is presented. A haptic exploratory procedure based on robotic palpation is performed to get pressure images at different grasping forces that provide information not only about the external shape of the object, but also about its internal features. The gripper consists of two underactuated fingers with a tactile sensor array in the thumb. A new representation of tactile information as 3D tactile tensors is described. During a squeeze-and-release process, the pressure images read from the tactile sensor are concatenated forming a tensor that contains information about the variation of pressure matrices along with the grasping forces. These tensors are used to feed a 3D Convolutional Neural Network (3D CNN) called 3D TactNet, which is able to classify the grasped object through active interaction. Results show that 3D CNN performs better, and provide better recognition rates with a lower number of training data.

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

“…Traditional computer vision and machine learning methods have been used to face the problem of tactile object recognition [14]. The work presented in [15] uses a feature-based matching technique from tactile images obtained with GelSight, while deep learning algorithms were proposed in [16], [17]. Nevertheless, other researchers faced the tactile object recognition problem by considering dynamic tactile information.…”

Embodied tactile perception and learning

“…Sohn et al [83] applied deep learning to large-scale electronic skin tactile perception. By using various contact forces, the obtained tactile spatio-temporal sequence information was integrated and a 3D CNN was designed to realize object recognition [42]. CNNs are mostly designed for images; thus, good results can be expected on tactile sensors based on optical measurement.…”

“…Wang et al [41] placed four contacting points on the palm of the designed service robot. Pastor et al [42] integrated a high‐resolution tactile array containing 1400 tactile sensing units on the palm of a robotic hand, which collaboratively worked with the under‐actuated fingers. The palm has a limited contact with the object during the operation; hence, it is greatly affected by the hand’s operating configuration.…”

Neural Networks Predicting Microbial Fuel Cells Output for Soft Robotics Applications

“…A widely used approach is therefore to use bio-inspired learning algorithms for control of soft robots ( Wilson et al, 2016 ; Thuruthel et al, 2018 ). Strategies include learning the inverse kinematics of soft actuators ( Thuruthel et al, 2016 ), predictive control ( Thuruthel et al, 2019 ), and mapping sensor outputs to real world values ( Pastor et al, 2019 ).…”