The role of perception and action on the use of allocentric information in a large-scale virtual environment

Abstract

In everyday life, our brain constantly builds spatial representations of the objects surrounding us. Many studies have investigated the nature of these spatial representations. It is well established that we use allocentric information in real-time and memory-guided movements. Most studies relied on small-scale and static experiments, leaving it unclear whether similar paradigms yield the same results on a larger scale using dynamic objects. We created a virtual reality task that required participants to encode the landing position of a virtual ball thrown by an avatar. Encoding differed in the nature of the task in that it was either purely perceptual (“view where the ball landed while standing still”—Experiment 1) or involved an action (“intercept the ball with the foot just before it lands”—Experiment 2). After encoding, participants were asked to place a real ball at the remembered landing position in the virtual scene. In some trials, we subtly shifted either the thrower or the midfield line on a soccer field to manipulate allocentric coding of the ball’s landing position. In both experiments, we were able to replicate classic findings from small-scale experiments and to generalize these results to different encoding tasks (perception vs. action) and response modes (reaching vs. walking-and-placing). Moreover, we found that participants preferably encoded the ball relative to the thrower when they had to intercept the ball, suggesting that the use of allocentric information is determined by the encoding task by enhancing task-relevant allocentric information. Our findings indicate that results previously obtained from memory-guided reaching are not restricted to small-scale movements, but generalize to whole-body movements in large-scale dynamic scenes.

Spatiotemporal Coding in the Macaque Supplementary Eye Fields: Landmark Influence in the Target-to-Gaze Transformation

“…In both the FEF and SEF, there was good agreement between the degree of landmark influence on the neural signals (measured either as a T-T' shift or the final T-T'/T-G slope) and the actual gaze behavior, suggesting that these structures participate in the optimal integration described in previous studies, where 'optimal' is defined as the best estimate of target direction based on statistical weighing of signal uncertainty (Byrne and Crawford, 2010;Karimpur et al, 2020;Körding and Wolpert, 2004). This is consistent with the general notion that the brain employs Bayesian methods of statistical learning, using probabilistic strategy from the task/target distribution and feedback uncertainty, to optimize performance (Aagten-Murphy and Bays, 2019; Byrne and Crawford, 2010;Karimpur et al, 2020;Körding and Wolpert, 2004;Mutluturk and Boduroglu, 2014).…”

“…Previous studies have suggested that egocentric and allocentric visual cues are optimally integrated for goal directed action (Byrne and Crawford, 2010;Karimpur et al, 2020). Here, we emulated this behavior in the gaze system, and found that the SEF (like the FEF) is involved in an eye-centered transformation of target signals into gaze signals while incorporating landmark-centered information (Bharmauria et al, 2020).…”

“…The brain integrates egocentric (eye-centered) and allocentric (landmark-centered) visual cues to guide goal-directed behavior (Ball et al, 2009;Chen et al, 2011;Goodale and Haffenden, 1998;Karimpur et al, 2020). For example, to score a goal, a soccer forward must derive allocentric relationships (e.g.…”

Landmark-Centered Coding in Frontal Cortex Visual Responses

“…Visual landmarks influence spatial cognition [1][2][3], navigation [4,5] and goal-directed behavior [6][7][8], but their influence on visual coding in sensorimotor systems is poorly understood [6,[9][10][11]. We hypothesized that visual responses in frontal cortex control gaze areas encode potential targets in an intermediate gaze-centered / landmark-centered reference frame that might depend on specific target-landmark configurations rather than a global mechanism.…”

Landmark-Centered Coding in Frontal Cortex Visual Responses

“…Various high-level theories have been postulated for the integration of landmarks with the egocentric codes for action 2,5,[9][10][11][12] . It has been speculated that the visual system codes this influence as target-landmark configurations 2,13 , but the neural mechanisms for this are unknown.…”