Visual Appearance Modulates Prediction Error in Virtual Reality

Singh, Avinash Kumar; Chen, Hsiang-Ting; Cheng, Yu‐Fan; King, Jung-Tai; Ko, Li-Wei; Gramann, Klaus; Lin, Chin‐Teng

doi:10.1109/access.2018.2832089

Cited by 50 publications

(56 citation statements)

References 43 publications

Supporting

Mentioning

Contrasting

Unclassified

Order By: Relevance

“…ACC is also known to interact with motor controls in a bottom-up fashion (Rauss & Pourtois, 2013). That cognitive conflict originates in ACC is also aligned with other related tasks, such as action monitoring (Botvinick, Braver, Barch, Carter, & Cohen, 2001), observational errors (van Schie et al, 2004), and prediction errors (Gehrke et al, 2019;Ozkan & Pezzetta, 2018;Singh et al, 2018).…”

Section: Origin Of Cognitive Conflict Event-related Spectral Perturbmentioning

confidence: 95%

“…Our previous work with a 3D object selection task demonstrated that changing the selection radius of a virtual cube (leading to premature feedback of touching the object) could lead to a mismatch between the visual and the proprioceptive feedback, invoking cognitive conflict (Gehrke et al, 2019;A. K. Singh, H.-T. Chen, K. Gramann, & C.-T. Lin, 2020;Singh et al, 2018). We found the conflict reflected a form of prediction error negativity (PEN) that seems to belong to the same class of ERP as the ERN and FRN found in the studies mentioned above.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

The Impact of Hand Movement Velocity on Cognitive Conflict Processing in a 3D Object Selection Task in Virtual Reality

Singh

Gramann

Chen

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

Detecting and correcting incorrect body movements is an essential part of everyday interaction with one's environment. The human brain has a constant monitoring system that controls and adjusts our actions according to our surroundings. However, when our brain's predictions about a planned action do not match the sensory inputs resulting from that action, cognitive conflict occurs. Much is known about cognitive conflict in 1D/2D environments; however, less is known about the role of movement characteristics on cognitive conflict in 3D environment. Hence, we devised an object selection task in a virtual reality environment to test how the velocity of hand movements impact a number of brain responses. From a series of analyses of EEG recordings synchronized with motion capture, we found that the velocity of the participants' hand movements modulated the brain's proprioception during the task and induced prediction error negativity. Additionally, prediction error negativity originates in the anterior cingulate cortex and is itself modulated by the ballistic phase of the hand's movement. These findings suggest that velocity is an essential component of integrating hand movements with visual and proprioceptive information during interactions with real and virtual objects.

show abstract

Section: Origin Of Cognitive Conflict Event-related Spectral Perturbmentioning

confidence: 95%

Section: Introductionmentioning

confidence: 99%

The Impact of Hand Movement Velocity on Cognitive Conflict Processing in a 3D Object Selection Task in Virtual Reality

Singh

Gramann

Chen

et al. 2020

Preprint

Self Cite

View full text Add to dashboard Cite

show abstract

“…More recently, Singh et al demonstrated that, in an object selection task in VR, the PEN component of the ERP was more pronounced for incorrect feedback when the user's hand was represented by a realistic hand avatar as compared to unrealistic representations [36]. Furthermore, they found the PEN amplitude correlated with the level of realism of the avatar hand, as suggested by the uncanny valley theory [25].…”

Section: The Impact Of Realism In Brain Responsesmentioning

confidence: 99%

“…The objective of our study is to explore whether ERPs have potential in detecting sensory conflicts in VR. As such, we designed a study in which participants perform a 3D object selection task in VR (modeled after [36]). As a participant reaches out to touch an object, they are presented with three sensory feedback modalities (a visual baseline, tactile and tactile with force-feedback).…”

Section: User Studymentioning

confidence: 99%

See 1 more Smart Citation

Detecting Visuo-Haptic Mismatches in Virtual Reality using the Prediction Error Negativity of Event-Related Brain Potentials

Gehrke

Akman

Lopes

et al. 2019

Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Self Cite

View full text Add to dashboard Cite

Designing immersion is the key challenge in virtual reality; this challenge has driven advancements in displays, rendering and recently, haptics. To increase our sense of physical immersion, for instance, vibrotactile gloves render the sense of touching, while electrical muscle stimulation (EMS) renders forces. Unfortunately, the established metric to assess the effectiveness of haptic devices relies on the user's subjective interpretation of unspecific, yet standardized, questions. Here, we explore a new approach to detect a conflict in visuohaptic integration (e.g., inadequate haptic feedback based on poorly configured collision detection) using electroencephalography (EEG). We propose analyzing event-related potentials (ERPs) during interaction with virtual objects. In our study, participants touched virtual objects in three conditions and received either no haptic feedback, vibration, or vibration and EMS feedback. To provoke a brain response in unrealistic VR interaction, we also presented the feedback prematurely in 25% of the trials. We found that the early negativity component of the ERP (so called prediction error) was more pronounced in the mismatch trials, indicating we successfully detected haptic conflicts using our technique. The results can be understood as a first step towards ERPs as a potential metric of haptic immersion in VR. CCS CONCEPTS • Human-centered computing → Haptic devices; Virtual reality;

show abstract

Exploring the effects of virtual hand appearance on midair typing efficiency

Cui

Mousas

2023

Computer Animation & Virtual

View full text Add to dashboard Cite

Summary Midair typing has gradually become the potential mainstream of virtual reality (VR) typing, as it eliminates the setup of additional hardware devices and provides convenience for portable head‐mounted displays (HMDs). In midair typing, users type on a virtual keyboard while using their virtual hands. In addition to the lack of a dominant method for VR‐based midair typing, we still do not know the impact that different appearances of virtual hands could have on text input in VR. Thus, we picked a widely used midair typing method (i.e., the VR‐rendered QWERTY keyboard) and conducted a VR study to explore the effects of three hand appearances (i.e., abstract, mannequin, realistic hands) on typing efficiency in terms of typing accuracy, typing speed, and task load. Our within‐group study revealed that (1) the mannequin hand caused significantly higher performance on typing speed than the abstract and realistic hands, (2) the abstract hand caused significantly lower performance (more typing errors) on typing accuracy compared to the mannequin and realistic hands, and (3) participants rated the task load of using the abstract hand significantly higher than the mannequin and realistic hands.

show abstract

Visual Appearance Modulates Prediction Error in Virtual Reality

Cited by 50 publications

References 43 publications

The Impact of Hand Movement Velocity on Cognitive Conflict Processing in a 3D Object Selection Task in Virtual Reality

The Impact of Hand Movement Velocity on Cognitive Conflict Processing in a 3D Object Selection Task in Virtual Reality

Detecting Visuo-Haptic Mismatches in Virtual Reality using the Prediction Error Negativity of Event-Related Brain Potentials

Exploring the effects of virtual hand appearance on midair typing efficiency

Contact Info

Product

Resources

About