Visuo-haptic integration in object identification using novel objects

Desmarais, Geneviève; Meade, Melissa E.; Wells, Taylor; Nadeau, Mélanie

doi:10.3758/s13414-017-1382-x

Cited by 16 publications

(18 citation statements)

References 35 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, this may be, our results support Kennedy (1993) that people deprived of vision can accurately process 2D spatial information about objects at a similar level as sighted individuals, in our case by making use of a combination of information based on contours and textures. In a series of experiments, Desmarais et al (2017) consistently demonstrated that haptic and visual object identification are based on similar processes which may possibly share memory representations that contain shape information.…”

Section: Discussionmentioning

confidence: 96%

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

2020

View full text Add to dashboard Cite

A high level of variability in the capacity of visually impaired children to accurately identify tactile images is reported in the literature, with on average rather low percentages of correct naming responses. However, most of these studies used raised-line drawings as stimuli to be explored and named. The present experiment investigated whether blind children of 3 to 8 years of age would demonstrate a satisfactory ability to name the elements making up tactile images when tested in an experimental setting similar to their natural reading conditions. Textured tactile images taken from genuine illustrated tactile books for young children were used, and the participants received information about the title of the book or listened the text that accompanied each picture before exploration, as it would occur in a natural reading setting. The results showed that their naming scores were indeed higher than previously reported at equivalent ages and did not differ from those of age-matched sighted children. These scores were positively impacted by haptic practice in blind children and correlated with the use of some specific exploratory procedures. The blind children benefited from information provided before exploration, as did their sighted counterparts. However, only in the former did the condition in which full information was provided influence the way the children organized their exploration. The haptic identification scores increased with age regardless of visual status, with the exploration times decreasing in the blind children, while the reverse trend was observed in the sighted children. These results are discussed at the light of the image-mediation model of haptics, suggesting that during the age period considered in the present experiment, blind children would progressively learn to process haptic information directly, thus leading to a decrease of exploration times, while sighted children would learn to translate haptic information into a visual image used to retrieve semantic information, involving an increase of their exploration times.

show abstract

Section: Discussionmentioning

confidence: 96%

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

2020

View full text Add to dashboard Cite

show abstract

“…While we think of softness as primarily a mechanical property that can be perceived through touch (Klatzky & Lederman, 1987;Cellini et al, 2013;Okamoto et al, 2013;Di Luca, 2014;Higashi et al, 2019;Kitada et al, 2019;Cavdan et al, 2019;Dovencioglu et al, 2020;Xu et al, 2020) softness can also be judged visually (Drewing et al, 2009;Giesel & Zaidi, 2013;Baumgartner et al, 2013;Bouman et al, 2013;Bi et al, 2018;Schmid & Doerschner, 2018). This latter ability is most likely acquired through countless multisensory interactions with objects in the environment, where simultaneous activation of visual and haptic senses leads to strong associations across modalities (Lacey et al, 2010;Yildirim & Jacobs, 2013;Desmarais et al, 2017). For example, while exploring a type of fabric (e.g.…”

Section: Discussionmentioning

confidence: 99%

“…The copyright holder for this preprint this version posted January 22, 2021. ; https://doi.org/10.1101/2021.01.22.427730 doi: bioRxiv preprint environment, where simultaneous activation of visual and haptic senses leads to strong associations across modalities (Lacey et al, 2010;Yildirim & Jacobs, 2013;Desmarais et al, 2017). For example, while exploring a type of fabric (e.g.…”

Section: Discussionmentioning

confidence: 99%

Materials in action: The look and feel of soft

Cavdan

Drewing

Doerschner

2021

Preprint

View full text Add to dashboard Cite

The softness of objects can be perceived through several senses. For instance, to judge the softness of our cat’s fur, we do not only look at it, we also run our fingers in idiosyncratic ways through its coat. Recently, we have shown that haptically perceived softness covaries with the compliance, viscosity, granularity, and furriness of materials (Dovencioglu et al.,2020). However, it is unknown whether vision can provide similar information about the various aspects of perceived softness. Here, we investigated this question in an experiment with three conditions: in the haptic condition, blindfolded participants explored materials with their hands, in the visual-static condition participants were presented with close-up photographs of the same materials, and in the visual-dynamic condition participants watched videos of the hand-material interactions that were recorded in the haptic condition. After haptically or visually exploring the materials participants rated them on various attributes. Our results show a high overall perceptual correspondence between the three experimental conditions. With a few exceptions, this correspondence tended to be strongest between haptic and visual-dynamic conditions. These results are discussed with respect to information potentially available through the senses, or through prior experience, when judging the softness of materials.

show abstract

“…Other researchers study different exploratory procedures that humans apply for tactile object recognition [2] and reveal the superiority of tactile perception in presence of vision [4]. Demarais et al [6] studied the performance of visual, tactile, and bimodal exploration of objects for both learning and testing procedures for object identification.…”

Section: State-of-the-artmentioning

confidence: 99%

An Application of Deep Learning to Tactile Data for Object Recognition under Visual Guidance

Rouhafzay

Creţu

2019

Sensors

View full text Add to dashboard Cite

Drawing inspiration from haptic exploration of objects by humans, the current work proposes a novel framework for robotic tactile object recognition, where visual information in the form of a set of visually interesting points is employed to guide the process of tactile data acquisition. Neuroscience research confirms the integration of cutaneous data as a response to surface changes sensed by humans with data from joints, muscles, and bones (kinesthetic cues) for object recognition. On the other hand, psychological studies demonstrate that humans tend to follow object contours to perceive their global shape, which leads to object recognition. In compliance with these findings, a series of contours are determined around a set of 24 virtual objects from which bimodal tactile data (kinesthetic and cutaneous) are obtained sequentially and by adaptively changing the size of the sensor surface according to the object geometry for each object. A virtual Force Sensing Resistor array (FSR) is employed to capture cutaneous cues. Two different methods for sequential data classification are then implemented using Convolutional Neural Networks (CNN) and conventional classifiers, including support vector machines and k-nearest neighbors. In the case of conventional classifiers, we exploit contourlet transformation to extract features from tactile images. In the case of CNN, two networks are trained for cutaneous and kinesthetic data and a novel hybrid decision-making strategy is proposed for object recognition. The proposed framework is tested both for contours determined blindly (randomly determined contours of objects) and contours determined using a model of visual attention. Trained classifiers are tested on 4560 new sequential tactile data and the CNN trained over tactile data from object contours selected by the model of visual attention yields an accuracy of 98.97% which is the highest accuracy among other implemented approaches.

show abstract

Visuo-haptic integration in object identification using novel objects

Cited by 16 publications

References 35 publications

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

Identification of Textured Tactile Pictures in Visually Impaired and Blindfolded Sighted Children

Materials in action: The look and feel of soft

An Application of Deep Learning to Tactile Data for Object Recognition under Visual Guidance

Contact Info

Product

Resources

About