Studying Spatial Memory in Augmented and Virtual reality

Abstract

Highlights• We built matching spatial memory tasks in VR and AR• Subjectively, subjects find the AR easier, more immersive and more fun• Objectively, subjects are significantly more accurate in AR compared to VR• Pointing based tasks did not fully show the same advantages • Only AR walking significantly correlated with SBSoD, suggesting mobile AR better captures more natural spatial performance Abstract Spatial memory is a crucial part of our lives. Spatial memory research and rehabilitation in humans is typically performed either in real environments, which is challenging practically, or in Virtual Reality (VR), which has limited realism. Here we explored the use of Augmented Reality (AR) for studying spatial cognition. AR combines the best features of real and VR paradigms by allowing subjects to learn spatial information in a flexible fashion while walking through a real-world environment. To compare these methods, we had subjects perform the same spatial memory task in VR and AR settings. Although subjects showed good performance in both, subjects reported that the AR task version was significantly easier, more immersive, and more fun than VR. Importantly, memory performance was significantly better in AR compared to VR. Our findings validate that integrating AR can lead to improved techniques for spatial memory research and suggest their potential for rehabilitation.

Towards real-world neuroscience using mobile EEG and augmented reality

“…The increasing use of AR for experiments could allow more research to be conducted outside the lab, where participants retain a sense of agency and immersion in the environment. Current research suggests AR is more engaging than VR, with potential improvements in memory encoding 17 , and is sensitive to changes in attentional states 31 . In addition, head-mounted AR is immersive yet keeps the participate in the real world, and compared to VR, is more comfortable for longer sessions and is less likely to induce nausea.…”

“…In contrast to virtual reality, where the whole environment is simulated, AR affords the ability to place 3D virtual objects in the actual environment. Recent research suggests that AR is more engaging than VR, with an indication of improved memory performance 17 . The ability to place a limitless set of virtual items in the real world offers a degree of experimental control that can't be matched through brute force methods.…”

Single-Neuron Representations of Spatial Targets in Humans

“…The subjects in our study were neurosurgical patients who volunteered to perform our spatial-memory task, Treasure Hunt, in free time between clinical procedures. Treasure Hunt is a 3D virtual spatial-memory paradigm developed in Unity3D, which we previously used to study various aspects of human spatial memory and electrophysiology [24,55,56].…”

Towards real-world neuroscience using mobile EEG and augmented reality

“…The increasing use of AR for experiments could allow more research to be conducted outside the lab, where participants retain a sense of agency and immersion in the environment. Current research suggests AR is more engaging than VR, with potential improvements in memory encoding (Maidenbaum et al, 2019), and is sensitive to changes in attentional states (Vortmann et al, 2019). The expanded use of AR and EEG presents a new tool for psychologists to meld virtual and real environments in naturalistic paradigms.…”

“…In contrast to virtual reality, where the whole environment is simulated, AR affords the ability to place 3D virtual objects in the actual environment. Recent research suggests that AR is more engaging than VR, with an indication of improved memory performance (Maidenbaum et al, 2019). The ability to place a limitless set of virtual items in the real world offers a degree of experimental control that can't be matched through brute force methods.…”