Perceptual-Cognitive Integration for Goal-Directed Action in Naturalistic Environments

Abstract

Real-world actions require one to simultaneously perceive, think, and act on the surrounding world, requiring the integration of (bottom-up) sensory information and (top-down) cognitive and motor signals. Studying these processes involves the intellectual challenge of cutting across traditional neuroscience silos, and the technical challenge of recording data in uncontrolled natural environments. However, recent advances in techniques, such as neuroimaging, virtual reality, and motion tracking, allow one to address these issues in naturalistic environments for both healthy participants and clinical populations. In this review, we survey six topics in which naturalistic approaches have advanced both our fundamental understanding of brain function and how neurologic deficits influence goal-directed, coordinated action in naturalistic environments. The first part conveys fundamental neuroscience mechanisms related to visuospatial coding for action, adaptive eye-hand coordination, and visuomotor integration for manual interception. The second part discusses applications of such knowledge to neurologic deficits, specifically, steering in the presence of cortical blindness, impact of stroke on visual-proprioceptive integration, and impact of visual search and working memory deficits. This translational approach—extending knowledge from lab to rehab—provides new insights into the complex interplay between perceptual, motor, and cognitive control in naturalistic tasks that are relevant for both basic and clinical research.

Predictive posture stabilization before contact with moving objects: equivalence of smooth pursuit tracking and peripheral vision

“…This behavior is prominently observed in scenarios such as tennis spectators, who actively turn their heads to track the ball’s motion across the court. The development of remote eye-tracking technology, which allows for the measurement of eye movements without restricting head motion, has greatly facilitated the study of more naturalistic eye and head movements (74). Utilizing these advancements, researchers have shifted towards analyzing gaze dynamics through the lens of gaze gain—a metric defined as the ratio of gaze velocity to object velocity, providing a more comprehensive understanding of tracking behavior beyond SPEM gains alone (32, 75, 76).…”

Instruction alters the influence of allocentric landmarks in a reach task

“…Before discussing our data, we are obliged to consider how our experimental design relates to natural reaching behavior ( Fooken et al., 2023 ). First, the normal visual feedback of the goal and hand was removed throughout the memory delay and reach.…”

Impairments in proprioceptively-referenced limb and eye movements in chronic stroke

“…Specifically, previous studies have noted that the posterior parietal cortex (PPC) plays an important role of integrating hand and visually-derived target information, and that damage to the posterior parietal cortex (PPC) can significantly disrupt this integration of information for action execution 43,44 . This suggests two possible explanations of these results, the first being that increases in eye-based proprioceptive error are due to impaired proprioceptive signaling due to damage to primary areas of proprioceptive processing, and/or damage that results in poor integration of proprioception with eye-based information 19,20,[45][46][47][48][49] . To more fully understand the mechanisms underlying this phenomenon, future neuroimaging studies are necessary to confirm lesion-behavior relationships as well as functional contributions of these brain areas.…”