Perspective: Assessing the Flexible Acquisition, Integration, and Deployment of Human Spatial Representations and Information

Abstract

Studying human spatial navigation in the lab can be challenging, particularly when including non-invasive neural measures like functional magnetic resonance imaging (fMRI) and scalp encephalography (EEG). While there is broad consensus that human spatial navigation involves both egocentric (self-referenced) and allocentric (world-referenced) coding schemes, exactly how these can be measured in ecologically meaningful situations remains controversial. Here, we explore these two forms of representation and how we might better measure them by reviewing commonly used spatial memory tasks and proposing a new task: the relative vector discrimination (RVD) task. Additionally, we explore how different encoding modalities (desktop virtual reality, immersive virtual reality, maps, and real-world navigation) might alter how egocentric and allocentric representations manifest. Specifically, we discuss desktop virtual reality vs. more immersive forms of navigation that better approximate real-world situations, and the extent to which less immersive encoding modalities alter neural and cognitive codes engaged during navigation more generally. We conclude that while encoding modality likely alters navigation-related codes to some degree, including egocentric and allocentric representations, it does not fundamentally change the underlying representations. Considering these arguments together, we suggest that tools to study human navigation in the lab, such as desktop virtual reality, provide overall a reasonable approximation of in vivo navigation, with some caveats.

Studying Spatial Memory in Augmented and Virtual reality

“…S70 Demonstrating the measured memory performance is elevated in AR and that subjects subjectively prefer AR over VR in terms of difficulty, immersion and enjoyment is the main result of this project, indicating that assessing spatial memory in AR is at least as valid, if not significantly better, then VR. Whereas previous VR memory paradigms had been criticized because memory performance was lower compared to real-world spatial memory [30], our results show that AR helps address this gap. Subjects showed significant spatial memory in all 4 conditions.…”

“…The use of these setups has been compared to traditional screen-based desktop VR. This line of work has shown general equivalence between the methods, with various advantages to walking over stationary conditions [23,26,27,28,30]. From the neuroscience perspective, it has been suggested that naturalistic head movements may sufficient to elicit natural spatial signals in VR that might be missing in fully virtual paradigms [2].…”

“…While AR and VR have not been directly compared for spatial memory, two other types of spatial memory comparisons involving VR are relevant to our question. First, performance in VR has been compared extensively to real-world performance, demonstrating the potential for similar levels of accuracy and for transfer between training in one to the other [21,30,33]. From the neuroscience perspective, the extent to which the neural signals underlying behavioral performance are similar between virtual and real-world environments it is currently debated, with some studies showing that signals are maintained while others finding significant differences [1,3,34].…”

“…These technical difficulties have led to a popular use of virtual-reality (VR) based paradigms for studying spatial memory (e.g. [16,25,30]). On the one hand, these VR paradigms are much more flexible and easy to run.…”

“…On the other hand, spatial memory and navigation in VR is inherently different compared to the real world. VR lacks the physical motion, level of immersion, and idiothetic (internal self motion) cues of real-world navigation, which lead to differences in performance (e.g., [15,26,30]). Beyond impairing the environment's perceived realism, the missing cue signals in VR may lead to changes or disruptions in the underlying neural processes in the brain related to spatial memory (e.g.…”