A Review of Tactile Information: Perception and Action Through Touch

Li, Qiang; Kroemer, Oliver; Su, Zhe; Veiga, Filipe; Kaboli, Mohsen; Ritter, Helge

doi:10.1109/tro.2020.3003230

Cited by 159 publications

(66 citation statements)

References 236 publications

(301 reference statements)

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…The main reason is that each approach uses different sensing modalities and hardware, considering different properties and objects (see Table I). Li et al have recently published a review that covers the most relevant sensor types and solutions of the state-of-art in the field of tactile perception [53].…”

Section: Resultsmentioning

confidence: 99%

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

Pastor

García‐González

Gandarias

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

Recent advances in the field of intelligent robotic manipulation pursue providing robotic hands with touch sensitivity. Haptic perception encompasses the sensing modalities encountered in the sense of touch (e.g., tactile and kinesthetic sensations). This letter focuses on multimodal object recognition and proposes analytical and data-driven methodologies to fuse tactile-and kinesthetic-based classification results. The procedure is as follows: a three-finger actuated gripper with an integrated high-resolution tactile sensor performs squeeze-andrelease Exploratory Procedures (EPs). The tactile images and kinesthetic information acquired using angular sensors on the finger joints constitute the time-series datasets of interest. Each temporal dataset is fed to a Long Short-term Memory (LSTM) Neural Network, which is trained to classify in-hand objects. The LSTMs provide an estimation of the posterior probability of each object given the corresponding measurements, which after fusion allows to estimate the object through Bayesian and Neural inference approaches. An experiment with 36-classes is carried out to evaluate and compare the performance of the fused, tactile, and kinesthetic perception systems. The results show that the Bayesian-based classifiers improves capabilities for object recognition and outperforms the Neural-based approach.

show abstract

Section: Resultsmentioning

confidence: 99%

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

Pastor

García‐González

Gandarias

et al. 2021

IEEE Robot. Autom. Lett.

View full text Add to dashboard Cite

show abstract

“…As described in [ 1 ], “hand movement data acquisition is used in many engineering applications”. The use of sensory gloves has been considered for many purposes, such as sign language recognition [ 2 , 3 ], hand posture monitoring [ 4 , 5 ], computer-generated (typically virtual reality or augmented vision) environments [ 6 ], tactile sensing [ 7 , 8 , 9 , 10 ], force-sensing for biomedical purposes [ 11 , 12 , 13 ], fitness exercises tracking [ 14 ], Sensing Finger Tapping in Piano Playing [ 15 ], teleoperation [ 16 ], rehabilitation [ 17 , 18 , 19 ] and many others. However, the use of convolutional neural networks (CNNs) to provide intelligence to these gloves is indeed a significant improvement in the recognition capacity of these devices.…”

Section: Related Workmentioning

confidence: 99%

Smart System with Artificial Intelligence for Sensory Gloves

Cerro¹,

Latasa²,

Guerra³

et al. 2021

Sensors

View full text Add to dashboard Cite

This paper presents a new sensory system based on advanced algorithms and machine learning techniques that provides sensory gloves with the ability to ensure real-time connection of all connectors in the cabling of a cockpit module. Besides a microphone, the sensory glove also includes a gyroscope and three accelerometers that provide valuable information to allow the selection of the appropriate signal time windows recorded by the microphone of the glove. These signal time windows are subsequently analyzed by a convolutional neural network, which indicates whether the connection of the components has been made correctly or not. The development of the system, its implementation in a production industry environment and the results obtained are analyzed.

show abstract

“…Vision-based recognition of an object is the commonly adopted approach; however, several research studies show incorporating a variety of sensory modalities is the key to further enhance the robotic capabilities in recognizing the multisensory object properties (see Bohg et al, 2017 ; Li et al, 2020 for a review). Previous work has shown that robots can recognize objects using non-visual sensory modalities such as the auditory (Torres-Jara et al, 2005 ; Sinapov et al, 2009 ; Luo et al, 2017 ; Eppe et al, 2018 ; Jin et al, 2019 ; Gandhi et al, 2020 ), the tactile (Sinapov et al, 2011b ; Bhattacharjee et al, 2012 ; Fishel and Loeb, 2012 ; Kerzel et al, 2019 ), and the haptic sensory modalities (Natale et al, 2004 ; Bergquist et al, 2009 ; Braud et al, 2020 ).…”

Section: Related Workmentioning

confidence: 99%

A Framework for Sensorimotor Cross-Perception and Cross-Behavior Knowledge Transfer for Object Categorization

et al. 2020

View full text Add to dashboard Cite

From an early age, humans learn to develop an intuition for the physical nature of the objects around them by using exploratory behaviors. Such exploration provides observations of how objects feel, sound, look, and move as a result of actions applied on them. Previous works in robotics have shown that robots can also use such behaviors (e.g., lifting, pressing, shaking) to infer object properties that camera input alone cannot detect. Such learned representations are specific to each individual robot and cannot currently be transferred directly to another robot with different sensors and actions. Moreover, sensor failure can cause a robot to lose a specific sensory modality which may prevent it from using perceptual models that require it as input. To address these limitations, we propose a framework for knowledge transfer across behaviors and sensory modalities such that: (1) knowledge can be transferred from one or more robots to another, and, (2) knowledge can be transferred from one or more sensory modalities to another. We propose two different models for transfer based on variational auto-encoders and encoder-decoder networks. The main hypothesis behind our approach is that if two or more robots share multi-sensory object observations of a shared set of objects, then those observations can be used to establish mappings between multiple features spaces, each corresponding to a combination of an exploratory behavior and a sensory modality. We evaluate our approach on a category recognition task using a dataset in which a robot used 9 behaviors, coupled with 4 sensory modalities, performed multiple times on 100 objects. The results indicate that sensorimotor knowledge about objects can be transferred both across behaviors and across sensory modalities, such that a new robot (or the same robot, but with a different set of sensors) can bootstrap its category recognition models without having to exhaustively explore the full set of objects.

show abstract

A Review of Tactile Information: Perception and Action Through Touch

Cited by 159 publications

References 236 publications

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

Bayesian and Neural Inference on LSTM-Based Object Recognition From Tactile and Kinesthetic Information

Smart System with Artificial Intelligence for Sensory Gloves

A Framework for Sensorimotor Cross-Perception and Cross-Behavior Knowledge Transfer for Object Categorization

Contact Info

Product

Resources

About