Visual and Haptic Integration in the Estimation of Softness of Deformable Objects

Abstract

Softness perception intrinsically relies on haptic information. However, through everyday experiences we learn correspondences between felt softness and the visual effects of exploratory movements that are executed to feel softness. Here, we studied how visual and haptic information is integrated to assess the softness of deformable objects. Participants discriminated between the softness of two softer or two harder objects using only-visual, only-haptic or both visual and haptic information. We assessed the reliabilities of the softness judgments using the method of constant stimuli. In visuo-haptic trials, discrepancies between the two senses' information allowed us to measure the contribution of the individual senses to the judgments. Visual information (finger movement and object deformation) was simulated using computer graphics; input in visual trials was taken from previous visuo-haptic trials. Participants were able to infer softness from vision alone, and vision considerably contributed to bisensory judgments (∼35%). The visual contribution was higher than predicted from models of optimal integration (senses are weighted according to their reliabilities). Bisensory judgments were less reliable than predicted from optimal integration. We conclude that the visuo-haptic integration of softness information is biased toward vision, rather than being optimal, and might even be guided by a fixed weighting scheme.

Multisensory Softness

“…However, the reliability of judgments, as measured by the standard deviations of individual's estimates, was not improved by the addition of visual information. Further evidence for non-optimal integration of visual and haptic softness was found by Cellini et al (2013). These studies collectively suggest that visual and haptic softness cues may not be integrated optimally, particularly when the two inputs are not in congruence.…”

“…If performance with two sources of information is higher than the best single-source, the brain must be taking advantage of the redundancy in the sensory estimates (the signature of sensory integration, Ernst 2006). For compliance perception, the outcome of multimodal integration has been shown to be close to the statistical optimum (maximal reduction in sensory noise) in some cases (Di Luca 2011), but not in others Cellini et al 2013). Deviation from optimality seems to occur when conflicts are introduced by the experimental manipulation.…”

“…A second type of conflict that can lead to non-integration, which does not follow the MLE scheme, is when the visual information about the amount of indentation is manipulated Cellini et al 2013). This situation is discussed in detail in Chap.…”

“…Information about resistive force is carried by touch and proprioception modalities and again other modalities could provide information about force (i.e., vision see Chap. 2 and Cellini et al 2013). Thus, integration could occur at the level of position and force information, even before a compliance estimate is obtained.…”

Interaction between static visual cues and force-feedback on the perception of mass of virtual objects

“…Little is known about how visual and haptic information interact when we estimate material properties. Most of the work investigating the influence of haptic-visual interaction on material properties has focused on roughness [Guest and Spence 2003;Jones and O'Neil 1985;Lederman et al 1986;Tiest and Kappers 2007] and compliance [Cellini et al 2013;Kuschel et al 2010;Massimiliano 2011;. A few studies investigate whether haptic and visual representation of material properties and categories are similar [Baumgartner et al 2013;Gaißert et al 2010;Lederman et al 1986;Overvliet and Soto-Faraco 2011;Tiest and Kappers 2007;Whitaker et al 2008].…”