Spatial coding for memory-guided reaching in visual and pictorial spaces

Abstract

An essential difference between pictorial space displayed as paintings, photographs, or computer screens, and the visual space experienced in the real world is that the observer has a defined location, and thus valid information about distance and direction of objects, in the latter but not in the former. Thus egocentric information should be more reliable in visual space, whereas allocentric information should be more reliable in pictorial space. The majority of studies relied on pictorial representations (images on a computer screen), leaving it unclear whether the same coding mechanisms apply in visual space. Using a memory-guided reaching task in virtual reality, we investigated allocentric coding in both visual space (on a table in virtual reality) and pictorial space (on a monitor that is on the table in virtual reality). Our results suggest that the brain uses allocentric information to represent objects in both pictorial and visual space. Contrary to our hypothesis, the influence of allocentric cues was stronger in visual space than in pictorial space, also after controlling for retinal stimulus size, confounding allocentric cues, and differences in presentation depth. We discuss possible reasons for stronger allocentric coding in visual than in pictorial space.

Perspective taking and systematic biases in object location memory

“…It is possible that the experimental setup that we used, and, in particular, the presentation of a three-dimensional virtual world using two-dimensional images (pictorial space), contributes to difficulties in understanding the position of the object in the environment across different viewpoints. As recently pointed out in other studies (Karimpur et al, 2020 ; Troje, 2019 ), the location of the observer in pictorial space is ill defined, because the observer is not actually present in the depicted space (Avraamides & Kelly, 2008 ). Observers may adopt the location of the (virtual) camera, which is what we asked our participants to do, but this process is challenging for a number of reasons: First, when viewing images, observers have access to monoscopic, but not stereoscopic, depth cues; second, the sensorimotor contingencies that link movements in the world to changes in the retinal image are different for images and for real-world visual space; finally, when viewing pictures, we generally accept distortions of the geometry of the displayed space (Troje, 2019 ).…”

“…Observers may adopt the location of the (virtual) camera, which is what we asked our participants to do, but this process is challenging for a number of reasons: First, when viewing images, observers have access to monoscopic, but not stereoscopic, depth cues; second, the sensorimotor contingencies that link movements in the world to changes in the retinal image are different for images and for real-world visual space; finally, when viewing pictures, we generally accept distortions of the geometry of the displayed space (Troje, 2019 ). Together, these factors are likely to contribute to a less reliable understanding of the exact nature of the perspective shift as well as less reliable estimates of the distances and directions to the objects in the stimuli (Karimpur et al, 2020 ; Troje, 2019 ). As noted in discussion of Experiment 2 , difficulties in depth perception can prevent participants from formulating a correct representation of object location and from understanding the perspective shifts correctly, which in turn may give rise to the camera-induced object motion.…”

Can ongoing movements be guided by allocentric visual information when the target is visible?

“…Previous research has shown that allocentric information can play an important role in guiding movements (e.g., Fiehler, Wolf, Klinghammer, & Blohm, 2014 ; Karimpur, Eftekharifar, Troje, & Fiehler, 2020 ; Lu & Fiehler, 2020 ). An important difference between such research and the present study is that participants could see both the target and the cursor throughout the entire movement in our experiments.…”

PTVR – A software in Python to make virtual reality experiments easier to build and more reproducible

“…It has become quite clear, especially in the last decade, that virtual reality (VR) is a valid and high-potential technology to build very sophisticated experiments in behavioral vision science and psychological research in general ( Cipresso, Giglioli, Raya, & Riva, 2018 ; de Gelder, Kätsyri, & de Borst, 2018 ; Huygelier, Schraepen, van Ee, Vanden Abeele, & Gillebert, 2019 ; Kourtesis, Collina, Doumas, & MacPherson, 2019 ; Parsons, 2015 ; Rizzo, Goodwin, De Vito, & Bell, 2021 ; Scarfe & Glennerster, 2015 ; Wilson & Soranzo, 2015 ). To cite without comprehensiveness a few recent examples related to vision science, experiments have been performed in fields as diverse as binocular vision and stereopsis ( Bankó et al, 2022 ; Levi, 2023 ), neuropsychological assessment of visual attention ( Foerster, Poth, Behler, Botsch, & Schneider, 2019 ), visual search ( David, Beitner, & Võ, 2021 ), visual perception with body/head-movements or with freely moving observers ( Bai, Bao, Zhang, & Jiang, 2019 ; Scarfe & Glennerster, 2015 ), self-location, and self-motion perception ( Luu, Zangerl, Kalloniatis, Palmisano, & Kim, 2021 ; Nakul, Orlando-Dessaints, Lenggenhager, & Lopez, 2020 ), visuo-motor control of reaching and pointing movements ( Karimpur, Eftekharifar, Troje, & Fiehler, 2020 ; Wiesing, Kartashova, & Zimmermann, 2021 ), and testing and training of different sensorimotor functions, such as mobility ( Bowman & Liu, 2017 ) or visuo-motor functions in low vision and ophthalmology ( Crossland, Starke, Imielski, Wolffsohn, & Webster, 2019 ; Soans et al, 2021 ).…”