Moving sounds enhance the visually-induced self-motion illusion (circular vection) in virtual reality

Abstract

While rotating visual and auditory stimuli have long been known to elicit self-motion illusions ("circular vection"), audiovisual interactions have hardly been investigated. Here, two experiments investigated whether visually induced circular vection can be enhanced by concurrently rotating auditory cues that match visual landmarks (e.g., a fountain sound). Participants sat behind a curved projection screen displaying rotating panoramic renderings of a market place. Apart from a no-sound condition, headphone-based auditory stimuli consisted of mono sound, ambient sound, or low-/high-spatial resolution auralizations using generic head-related transfer functions (HRTFs). While merely adding nonrotating (mono or ambient) sound showed no effects, moving sound stimuli facilitated both vection and presence in the virtual environment. This spatialization benefit was maximal for a medium (20 • × 15 • ) FOV, reduced for a larger (54 • × 45 • ) FOV and unexpectedly absent for the smallest (10 • × 7.5 • ) FOV. Increasing auralization spatial fidelity (from low, comparable to five-channel home theatre systems, to high, 5 • resolution) provided no further benefit, suggesting a ceiling effect. In conclusion, both self-motion perception and presence can benefit from adding moving auditory stimuli. This has important implications both for multimodal cue integration theories and the applied challenge of building affordable yet effective motion simulators.

Comparing leaning-based motion cueing interfaces for virtual reality locomotion

“…The observed trend is consistent with the literature in that motion cueing can often enhance perception of self-motion [7], [11], [12], [25], [29], [46]. However, the lack of significant benefit of motion cueing on self-motion perception in the current study suggests the need for further development of these prototype-like interfaces.…”

“…Moreover, researchers found vestibular and proprioceptive cues were tightly coupled, with interaction between motion detection and heading direction change playing a key role in motion perception [26]- [28]. Auditory cues, specifically with approaching and receding sound objects [29], [30], as well as tactile and biomechanical cues arranged in a matrix pattern around the torso [31] were also found to stimulate self-motion illusions.…”

Vertical illusory self-motion through haptic stimulation of the feet

“…Commonly experienced implicit motion clues include engine sounds or the subtle vibrations experienced during commute with a motorized vehicle. Notable, it would seem that explicit motion cues relies on bottom-up factors while explicit motion cues largely achieve their significance do to the perceivers expectations to, and interpretation of, the stimuli, that is, top-down factors [8]. With that being said, it does seem plausible that haptic stimuli also may qualify as a bottom-up factor.…”

“…Moreover, the experiments involving haptic feedback have generally focused on whether this form of stimuli positively influences an illusion of movement facilitated by stimulation of another modality [8,13].…”

“…Notably their results also showed that the auditory-tactile simulation of a vehicle engine was as effective as illusions induced via auditory feedback including explicit motion cues, i.e., moving sound fields. Riecke et al [8] similarly describe an experiment investigating whether physical vibrations of the perceivers' seat and footrest enhance visually induced circular vection. They found that the addition of this form of vibrotactile feedback entailed a slight, yet significant, enhancement of the self-motion illusion.…”