The vivid tactile experience from vision and auditory: Clues from multisensory channel integration

Abstract

Virtual reality creates an immersive experience for users by providing visual, auditory, and tactile information. However, tactile feedback still faces many technical bottlenecks, which limit natural interaction in the virtual reality environment. Pseudo-haptic technology based on multi-sensory illusion can enhance and enrich tactile perception with the help of information from other channels, which is one of the effective ways to optimize tactile perception in the virtual reality environment. In order to explore the problem in a more targeted way, the study focuses on roughness perception among different dimensions of tactile perception. We discuss the multi-sensory channel integration of visual, auditory, and tactile in roughness perception, then analyze how the visual cues (density of surface texture, light and shadow, control display rate) and auditory cues (pitch/frequency, loudness) affect roughness perception, moreover, summarize the methods of changing roughness perception by manipulating these cues. Finally, we discuss the differences of visual, auditory, and tactile information between virtual reality environment and realworld in representation and perceptual integration when using pseudo-haptic feedback technology, proposing practical solutions and future research directions to improve the tactile experience.

Decoding the Pattern Between Architecture and Emotion

“…Not only is the visual system the dominant sensory system, but also it is capable of receiving information related to the other sensory channels. For instance, roughness perception is a function of visual information perceived (WAN et al, 2022). More empirical study shows the role of the visual cortex in the perception of tactile information (Preston, 1999).…”