Research on object classification based on visual-tactile fusion

Abstract

As two modes of direct contact between robots and external environment, visual and tactile play a critical role in improving robot perception ability. In the real environment, it is difficult for the robot to achieve high accuracy when classifying objects only by a single mode (visual or tactile). In order to improve the classification accuracy of robots, a novel visual-tactile fusion method is proposed in this paper. Firstly, the ResNet18 is selected as the backbone network to extract visual features. To improve the accuracy of object localization and recognition in the visual network, the Position-Channel Attention Mechanism (PCAM) block is added after conv3 and conv4 of ResNet18. Then, the four-layer one-dimensional convolutional neural network is used to extract tactile features, and the extracted tactile features are fused with visual features at the feature layer. Finally, the experimental results demonstrate that compared with the existing methods, on the self-made dataset VHAC-52, the proposed method has improved the AUC and ACC by 1.60% and 1.47%, respectively.