Neuronal Representations Supporting Three-Dimensional Vision in Nonhuman Primates

Abstract

The visual system must reconstruct the dynamic, three-dimensional (3D) world from ambiguous two-dimensional (2D) retinal images. In this review, we synthesize current literature on how the visual system of nonhuman primates performs this transformation through multiple channels within the classically defined dorsal (where) and ventral (what) pathways. Each of these channels is specialized for processing different 3D features (e.g., the shape, orientation, or motion of objects, or the larger scene structure). Despite the common goal of 3D reconstruction, neurocomputational differences between the channels impose distinct information-limiting constraints on perception. Convergent evidence further points to the little-studied area V3A as a potential branchpoint from which multiple 3D-fugal processing channels diverge. We speculate that the expansion of V3A in humans may have supported the emergence of advanced 3D spatial reasoning skills. Lastly, we discuss future directions for exploring 3D information transmission across brain areas and experimental approaches that can further advance the understanding of 3D vision. Expected final online publication date for the Annual Review of Vision Science, Volume 9 is September 2023. Please see http://www.annualreviews.org/page/journal/pubdates for revised estimates.

Optimal motion-in-depth estimation with natural stimuli

“…The neurophysiological underpinnings of 3D motion perception is a topic of recurring interest (Rosenberg et al, 2023). The results here indicate that binocular receptive fields that jointly encode retinal speed and disparity information optimally support the estimation of 3D speed ( Figure 3A , bottom), especially at slow speeds (see Figure 3E ).…”

“…Clearly, humans can estimate 3D motion. And neurons that unambiguously select for 3D motion have recently been reported in area FST (Rosenberg et al, 2023; Thompson et al, 2023).…”

“…A spate of recent research has investigated the psychophysical limits and neurophysiological underpinnings of 3D motion estimation (Cormack et al, 2017; Rosenberg et al, 2023). The development of ideal observers for 3D motion estimation–that are grounded in natural scene statistics–can help interpret existing results by providing principled benchmarks against which performance patterns can be evaluated.…”

“…Animals with binocular vision combine the two two-dimensional (2D) images formed by the eyes to estimate 3D motion. However, many questions remain about how the brain does, and should, estimate 3D motion from binocular images (Bonnen et al, 2020; Cormack et al, 2017; Czuba et al, 2014; Rosenberg et al, 2023; Sanada & DeAngelis, 2014). Ideal observer analysis is a useful tool for providing normative accounts of how perceptual tasks should be solved, and for establishing theoretical bounds on sensory-perceptual performance.…”

Hierarchical computation of 3D motion across macaque areas MT and FST

“…Although 3D visual processing shaped the evolution of the primate brain, 1 current understanding of 3D motion processing remains surprisingly limited compared with that of two-dimensional (2D) retinal-motion processing. 2 – 4 In part, this reflects that distinct cues signal 2D versus 3D motion, with 3D motion computations depending critically on the relationship between retinal-motion signals sensed by the two eyes. 5 – 8 …”