Artificial haptic recognition through human manipulation of objects

Miralles, David; Parés, Carlota; Garrofé, Guillem; Soto, Alberto; Llauradó, Albert; Serra, Gerard; Falcó, Àlex; Beeck, Hans Op de; Masson, Haemy Lee

doi:10.32470/ccn.2019.1240-0

Cited by 2 publications

(2 citation statements)

References 0 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Our haptic object recognition system achieves an average accuracy of 89.63% after just 8 seconds of manipulation (average time for best accuracy in humans (20)), as shown in Fig. 1.…”

Section: Haptic Data and Object Recognitionmentioning

confidence: 85%

See 1 more Smart Citation

Cross-Modal Environment Self-Adaptation During Object Recognition in Artificial Cognitive Systems

Miralles

Garrofé

Parés

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

The cognitive connection between the senses of touch and vision is probably the best-known case of cross-modality. Recent discoveries suggest that the mapping between both senses is learned rather than innate. These evidences open the door to a dynamic cross-modality that allows individuals to adaptively develop within their environment. Mimicking this aspect of human learning, we propose a new cross-modal mechanism that allows artificial cognitive systems (ACS) to adapt quickly to unforeseen perceptual anomalies generated by the environment or by the system itself. In this context, visual recognition systems have advanced remarkably in recent years thanks to the creation of large-scale datasets together with the advent of deep learning algorithms. However, such advances have not occurred on the haptic mode, mainly due to the lack of two-handed dexterous datasets that allow learning systems to process the tactile information of human object exploration. This data imbalance limits the creation of synchronized multimodal datasets that would enable the development of cross-modality in ACS during object exploration. In this work, we use a multimodal dataset recently generated from tactile sensors placed on a collection of objects that capture haptic data from human manipulation, together with the corresponding visual counterpart. Using this data, we create a cross-modal learning transfer mechanism capable of detecting both sudden and permanent anomalies in the visual channel and still maintain visual object recognition performance by retraining the visual mode for a few minutes using haptic information. Here we show the importance of cross-modality in perceptual awareness and its ecological capabilities to self-adapt to different environments.

show abstract

“…Our haptic object recognition system achieves an average accuracy of 89.63% after just 8 seconds of manipulation (average time for best accuracy in humans (20)), as shown in Fig. 1.…”

Section: Haptic Data and Object Recognitionmentioning

confidence: 85%

“…As illustrated in Fig. 1, the objects are six similar shapes we have digitally created and 3D printed (20,21).…”

Section: Haptic Data and Object Recognitionmentioning

confidence: 99%

Cross-Modal Environment Self-Adaptation During Object Recognition in Artificial Cognitive Systems

Miralles

Garrofé

Parés

et al. 2021

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

Multi-modal self-adaptation during object recognition in an artificial cognitive system

Miralles

Garrofé

Parés

et al. 2022

Sci Rep

Self Cite

View full text Add to dashboard Cite

The cognitive connection between the senses of touch and vision is probably the best-known case of multimodality. Recent discoveries suggest that the mapping between both senses is learned rather than innate. This evidence opens the door to a dynamic multimodality that allows individuals to adaptively develop within their environment. By mimicking this aspect of human learning, we propose a new multimodal mechanism that allows artificial cognitive systems (ACS) to quickly adapt to unforeseen perceptual anomalies generated by the environment or by the system itself. In this context, visual recognition systems have advanced remarkably in recent years thanks to the creation of large-scale datasets together with the advent of deep learning algorithms. However, this has not been the case for the haptic modality, where the lack of two-handed dexterous datasets has limited the ability of learning systems to process the tactile information of human object exploration. This data imbalance hinders the creation of synchronized datasets that would enable the development of multimodality in ACS during object exploration. In this work, we use a multimodal dataset recently generated from tactile sensors placed on a collection of objects that capture haptic data from human manipulation, together with the corresponding visual counterpart. Using this data, we create a multimodal learning transfer mechanism capable of both detecting sudden and permanent anomalies in the visual channel and maintaining visual object recognition performance by retraining the visual mode for a few minutes using haptic information. Our proposal for perceptual awareness and self-adaptation is of noteworthy relevance as can be applied by any system that satisfies two very generic conditions: it can classify each mode independently and is provided with a synchronized multimodal data set.

show abstract

Artificial haptic recognition through human manipulation of objects

Cited by 2 publications

References 0 publications

Cross-Modal Environment Self-Adaptation During Object Recognition in Artificial Cognitive Systems

Cross-Modal Environment Self-Adaptation During Object Recognition in Artificial Cognitive Systems

Multi-modal self-adaptation during object recognition in an artificial cognitive system

Contact Info

Product

Resources

About