A Virtual Tactile Sensor with Adjustable Precision and Size for Object Recognition

Rouhafzay, Ghazal; Creţu, Ana-Maria

doi:10.1109/civemsa.2018.8439966

Cited by 3 publications

(5 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In deep CNN architectures, convolutional layers with larger kernels are usually placed in earlier layers to extract general features from data, such as color and edges, and are not particular to a specific dataset [ 28 ]. Features from the later layers, closer to the output, are of higher level and are mostly updated to adapt the network to achieve a specific task.…”

Section: Classification Results and Discussionmentioning

confidence: 99%

“…The reason can be found in the deep architecture of Resnet50 that allocates smaller weight updates to each layer, and thus, the average normalized weight differences are lower. In deep CNN architectures, convolutional layers with larger kernels are usually placed in earlier layers to extract general features from data, such as color and edges, and are not particular to a specific dataset [28]. Features from the later layers, closer to the output, are of higher level and are mostly updated to adapt the network to achieve a specific task.…”

Section: Classification Results and Discussionmentioning

confidence: 99%

“…The FSR tactile sensor, as well as an example of the acquired tactile image, are illustrated in Figure 1 a,b. A virtual counterpart of this sensor was also developed to simulate the acquisition of tactile data from 3D object models [ 28 ] and allow to study the impact of tactile imprints’ quality and size as well as to plan the real acquisition of data. The virtual sensor allows modifying both the number of sensing elements and the distance between them.…”

Section: Datasets and Data Processingmentioning

confidence: 99%

“…This is justified by the fact that the quality of tactile imprints is better when the imprint is captured in the direction of the local normal on the surface of the object. Distances between the virtual object surface and sensing elements on the plane are computed and normalized in the range of 0 to 1 to generate a tactile image similar to the example in Figure 1 d. Additional details about this virtual sensor are provided in [ 28 ]. Tactile imprints of a resolution of 128 by 128 are simulated for experiments in this paper.…”

Section: Datasets and Data Processingmentioning

confidence: 99%

“…Distances between the virtual object surface and sensing elements on the plane are computed and normalized in the range of 0 to 1 to generate a tactile image similar to the example in Figure 1d. Additional details about this virtual sensor are provided in [28]. Tactile imprints of a resolution of 128 by 128 are simulated for experiments in this paper.…”

Section: Fsr Tactile Array and High-resolution Simulated Fsr Sensormentioning

confidence: 99%

See 4 more Smart Citations

Transfer of Learning from Vision to Touch: A Hybrid Deep Convolutional Neural Network for Visuo-Tactile 3D Object Recognition

Rouhafzay

Creţu

Payeur

2020

Sensors

Self Cite

View full text Add to dashboard Cite

Transfer of learning or leveraging a pre-trained network and fine-tuning it to perform new tasks has been successfully applied in a variety of machine intelligence fields, including computer vision, natural language processing and audio/speech recognition. Drawing inspiration from neuroscience research that suggests that both visual and tactile stimuli rouse similar neural networks in the human brain, in this work, we explore the idea of transferring learning from vision to touch in the context of 3D object recognition. In particular, deep convolutional neural networks (CNN) pre-trained on visual images are adapted and evaluated for the classification of tactile data sets. To do so, we ran experiments with five different pre-trained CNN architectures and on five different datasets acquired with different technologies of tactile sensors including BathTip, Gelsight, force-sensing resistor (FSR) array, a high-resolution virtual FSR sensor, and tactile sensors on the Barrett robotic hand. The results obtained confirm the transferability of learning from vision to touch to interpret 3D models. Due to its higher resolution, tactile data from optical tactile sensors was demonstrated to achieve higher classification rates based on visual features compared to other technologies relying on pressure measurements. Further analysis of the weight updates in the convolutional layer is performed to measure the similarity between visual and tactile features for each technology of tactile sensing. Comparing the weight updates in different convolutional layers suggests that by updating a few convolutional layers of a pre-trained CNN on visual data, it can be efficiently used to classify tactile data. Accordingly, we propose a hybrid architecture performing both visual and tactile 3D object recognition with a MobileNetV2 backbone. MobileNetV2 is chosen due to its smaller size and thus its capability to be implemented on mobile devices, such that the network can classify both visual and tactile data. An accuracy of 100% for visual and 77.63% for tactile data are achieved by the proposed architecture.

show abstract

Section: Classification Results and Discussionmentioning

confidence: 99%

Section: Classification Results and Discussionmentioning

confidence: 99%

Section: Datasets and Data Processingmentioning

confidence: 99%

Section: Datasets and Data Processingmentioning

confidence: 99%

Section: Fsr Tactile Array and High-resolution Simulated Fsr Sensormentioning

confidence: 99%

See 3 more Smart Citations