Identification of walked-upon materials in auditory, kinesthetic, haptic, and audio-haptic conditions

Abstract

Locomotion generates multisensory information about walked-upon objects. How perceptual systems use such information to get to know the environment remains unexplored. The ability to identify solid (e.g., marble) and aggregate (e.g., gravel) walked-upon materials was investigated in auditory, haptic or audio-haptic conditions, and in a kinesthetic condition where tactile information was perturbed with a vibromechanical noise. Overall, identification performance was better than chance in all experimental conditions and for both solids and the better identified aggregates. Despite large mechanical differences between the response of solids and aggregates to locomotion, for both material categories discrimination was at its worst in the auditory and kinesthetic conditions and at its best in the haptic and audio-haptic conditions. An analysis of the dominance of sensory information in the audio-haptic context supported a focus on the most accurate modality, haptics, but only for the identification of solid materials. When identifying aggregates, response biases appeared to produce a focus on the least accurate modality--kinesthesia. When walking on loose materials such as gravel, individuals do not perceive surfaces by focusing on the most accurate modality, but by focusing on the modality that would most promptly signal postural instabilities.

Footstep sounds synthesis: Design, implementation, and evaluation of foot–floor interactions, surface materials, shoe types, and walkers’ features

“…The statistical analysis shows that the four typologies of surface materials (solid, aggregate, liquid, and hybrid) were very well discriminated. This result perfectly parallels that of previous studies performed involving real stimuli listened by participants walking on real surface materials, recordings of real walks, as well as synthesized stimuli [27,28].…”

“…On a separate note, Giordano and colleagues studied the recognition of walked-upon ground materials [27]. Participants were presented with recordings of their own walking sounds using rubber-sole shoes on solid (vinyl, wood, ceramic, marble) and aggregate materials (very small gravel, small gravel, medium gravel, and large gravel), and were asked to identify them in a forced choice task with the same eight materials.…”

Spectral and temporal cues for perception of material and action categories in impacted sound sources

“…This led the authors to question whether material discrimination in general is based on auditory cues or whether the representations of material categories relies instead primarily on visual or haptic experience. Giordano et al (2012), studying walked-upon materials, confirmed that material identification based on haptic feedback was indeed more accurate than in auditory perception. However, multiple impact sounds (rattling, bouncing, scattering, dropping, etc.)…”

Designing presence for real locomotion in immersive virtual environments: an affordance-based experiential approach

“…Particular attention was devoted to the semantic congruence, especially for the case of footfloor interactions (Giordano et al 2012;Turchet and Serafin 2014). In addition, cross-modal correspondences between stimulus features in different sensory modalities were taken into account as a factor relevant for multisensory integration [for a review, see (Spence 2011)] and, therefore, related to presence (Harvey and Sanchez-Vives 2005).…”