Joint Representation of Depth from Motion Parallax and Binocular Disparity Cues in Macaque Area MT

Abstract

Perception of depth is based on a variety of cues, with binocular disparity and motion parallax generally providing more precise depth information than pictorial cues. Much is known about how neurons in visual cortex represent depth from binocular disparity or motion parallax, but little is known about the joint neural representation of these depth cues. We recently described neurons in the middle temporal (MT) area that signal depth sign (near vs far) from motion parallax; here, we examine whether and how these neurons also signal depth from binocular disparity. We find that most MT neurons in rhesus monkeys (Macaca Mulatta) are selective for depth sign based on both disparity and motion parallax cues. However, the depth-sign preferences (near or far) are not always aligned: 56% of MT neurons have matched depth-sign preferences ("congruent" cells) whereas the remaining 44% of neurons prefer near depth from motion parallax and far depth from disparity, or vice versa ("opposite" cells). For congruent cells, depth-sign selectivity increases when disparity cues are added to motion parallax, but this enhancement does not occur for opposite cells. This suggests that congruent cells might contribute to perceptual integration of depth cues. We also found that neurons are clustered in MT according to their depth tuning based on motion parallax, similar to the known clustering of MT neurons for binocular disparity. Together, these findings suggest that area MT is involved in constructing a representation of 3D scene structure that takes advantage of multiple depth cues available to mobile observers.

The neural basis of depth perception from motion parallax

“…For other neurons, such as the opposite cell illustrated in figure 7b, depth-tuning in the combined condition (orange) appears to be dominated by motion parallax tuning (black), and selectivity in the combined condition is slightly reduced by the incongruent disparity selectivity. These findings suggest that congruent cells might contribute to perceptual cue integration of depth cues, whereas opposite cells would not, but the study of Nadler et al [70] was not designed to test these issues directly. Additional studies, in which both psychophysical and neuronal performance is tested with both congruent and conflicting combinations of disparity and motion parallax cues, will be needed to evaluate whether activity of MT neurons can account for perceptual cue integration and cue weighting (as explored previously for multisensory neurons involved in heading perception, [71 -73]).…”

“…In animals, a few studies have examined how neurons signal three-dimensional surface orientation based on combinations of motion and disparity gradients [67] or perspective gradients and disparity gradients [68,69]. Recently, Nadler et al [70] measured the depth-sign selectivity of macaque MT neurons based on both binocular disparity and motion parallax cues. One might expect neurons to prefer the same depth-sign (near or far) for each cue.…”

“…Nadler et al [70] also measured responses of MT neurons to stimuli involving congruent combinations of binocular disparity and motion parallax cues (combined condition). In general, responses in the combined condition reflected a mixture of selectivity for both depth cues.…”

“…This local discrepancy between disparity and retinal image motion might be sensed by the relative activity of congruent and opposite cells in area MT. (Adapted from Nadler et al [70].) rstb.royalsocietypublishing.org Phil.…”

Adaptation to binocular anticorrelation results in increased neural excitability

“…The current results have implications beyond stereopsis. There is theoretical and empirical evidence supporting the existence of neurons tuned to mismatches from studies of stereopsis (DeAngelis, Ohzawa, & Freeman, 1991;Prince, Cumming, & Parker, 2002;Tsao et al, 2003), binocular rivalry (Katyal et al, 2018;Kingdom et al, 2018;Said & Heeger, 2013), and integration of cues within (Kim, Angelaki, & Deangelis, 2015;Nadler et al, 2013;Rideaux & Welchman, 2018) and between sensory modalities (Gu, Angelaki, & DeAngelis, 2008;Kim, Pitkow, Angelaki, & DeAngelis, 2016;Morgan, DeAngelis, & Angelaki, 2008).…”