The effects of substitute multisensory feedback on task performance and the sense of presence in a virtual reality environment

Abstract

Objective and subjective measures of performance in virtual reality environments increase as more sensory cues are delivered and as simulation fidelity increases. Some cues (colour or sound) are easier to present than others (object weight, vestibular cues) so that substitute cues can be used to enhance informational content in a simulation at the expense of simulation fidelity. This study evaluates how substituting cues in one modality by alternative cues in another modality affects subjective and objective performance measures in a highly immersive virtual reality environment. Participants performed a wheel change in a virtual reality (VR) environment. Auditory, haptic and visual cues, signalling critical events in the simulation, were manipulated in a factorial design. Subjective ratings were recorded via questionnaires. The time taken to complete the task was used as an objective performance measure. The results show that participants performed best and felt an increased sense of immersion and involvement, collectively referred to as ‘presence’, when substitute multimodal sensory feedback was provided. Significant main effects of audio and tactile cues on task performance and on participants' subjective ratings were found. A significant negative relationship was found between the objective (overall completion times) and subjective (ratings of presence) performance measures. We conclude that increasing informational content, even if it disrupts fidelity, enhances performance and user’s overall experience. On this basis we advocate the use of substitute cues in VR environments as an efficient method to enhance performance and user experience.

Presence Promotes Performance on a Virtual Spatial Cognition Task: Impact of Human Factors on Virtual Reality Assessment

“…Besides this tricky question, it has been noted that it is often complicated to find a causation direction: does the individual perform better because he feels more present or does he feel more present because he performs better (Nash et al, 2000)? Even when putting causation or direction aside, reviews and experimental studies do not allow us to assert anything on the relationship between presence and performance: associations are sometimes found, often weak (Witmer and Singer, 1994;Slater et al, 1996;Pausch et al, 1997;Stanney et al, 2002;Youngblut and Huie, 2003;Stevens and Kincaid, 2015;Cooper et al, 2018) and sometimes not found at all (Ma and Kaber, 2006;Pallamin and Bossard, 2016).…”

Narrative and Gaming Experience Interact to Affect Presence and Cybersickness in Virtual Reality

“…While some groups have reported that the sense of presence in VR can increase cybersickness (e.g., Lin et al, 2002;Ling et al, 2013;Liu & Uang, 2011), we recently argued (Weech et al, 2019) that the balance of evidence suggests an inverse relationship between presence and cybersickness (e.g., Busscher et al, 2011;Cooper et al, 2016;Kim et al, 2005;Knight & Arns, 2006;Milleville-Pennel & Charron, 2014;Witmer & Singer, 1998). Several groups have claimed that this association results from a disruptive effect of cybersickness symptoms on presence, such that nausea and discomfort reorient attention away from a simulated environment (Bahit et al, 2016;Cobb et al, 1999;Nichols et al, 2000;Stróżak et al, 2018;Wilson et al, 1997;Witmer & Singer, 1998).…”

Tactile and kinesthetic feedbacks improve distance perception in virtual reality

“…These could be seen as a conflicting standpoint compared to that of Cooper et al [7], who claims that auditory and tactile cues enhance the sense of immersion in virtual reality. However, it is worth pointing out that Hecht and Reiner's methodology implemented a non-virtual environment setup with at least decade-old technology, which could be behind the discrepancy.…”

“…Haptics, or tactile interaction, as a modality in virtual environments is under-researched in comparison to others [15], [7], [3]. Azmandian et al note that "studies have shown that using haptics can lead to significantly increased presence and spatial knowledge training transfer" [3].…”