Gradient adaptive sampling and multiple temporal scale 3D CNNs for tactile object recognition

Abstract

Tactile object recognition (TOR) is very important for the accurate perception of robots. Most of the TOR methods usually adopt uniform sampling strategy to randomly select tactile frames from a sequence of frames, which will lead to a dilemma problem, i.e., acquiring the tactile frames with high sampling rate will get lots of redundant data, while the low sampling rate will miss important information. In addition, the existing methods usually adopt single time scale to construct TOR model, which will induce that the generalization capability is not enough for processing the tactile data generated under different grasping speeds. To address the first problem, a novel gradient adaptive sampling (GAS) strategy is proposed, which can adaptively determine the sampling interval according to the importance of tactile data, therefore, the key information can be acquired as much as possible when the number of tactile frames is limited. To handle the second problem, a multiple temporal scale 3D convolutional neural networks (MTS-3DCNNs) model is proposed, which downsamples the input tactile frames with multiple temporal scales (MTSs) and extracts the MTS deep features, and the fused features have better generalization capability for recognizing the object grasped with different speed. Furthermore, the existing lightweight network ResNet3D-18 is modified to obtain a MR3D-18 network which can match the tactile data with smaller size and prevent the overfitting problem. The ablation studies show the effectiveness of GAS strategy, MTS-3DCNNs, and MR3D-18 networks. The comprehensive comparisons with advanced methods demonstrate that our method is SOTA on two benchmarks.

Mining local and global spatiotemporal features for tactile object recognition

“…To demonstrate the overall effectiveness of our model, we conducted a comprehensive quantitative comparison with 5 methods on the MIT-STAG dataset (Sundaram et al, 2019), i.e., STAG (Sundaram et al, 2019), Smart-hand (Wang et al, 2021), ResNet10-v1 (Zhang et al, 2021, Tactile-ViewGCN (Sharma et al, 2022), and GAS-MR3D (Qian et al, 2023c), and 5 methods on the iCub dataset (Soh et al, 2012), i.e., DS (Soh andDemiris, 2014), GS (Soh and Demiris, 2014), STORK-GP (Soh et al, 2012), STAG (Sundaram et al, 2019), and GAS-MR3D (Qian et al, 2023c).…”

“…Bottcher et al (2021) Recently, the another category of methods has achieved remarkable results and has become mainstream. Qian et al (2023c). used a gradient adaptive sampling (GAS) strategy to process the acquired tactile data and subsequently fed the data into a 3D CNN network to extract multiple scale temporal features.…”