A Computational Model for the Neural Representation and Estimation of the Binocular Vector Disparity from Convergent Stereo Image Pairs

Abstract

The depth cue is a fundamental piece of information for artificial and living beings who interact with the surrounding environment in order to handle objects and to avoid obstacles: in such situations, the disparity patterns, which arise when agents fixate objects, are vector fields. We propose a biologically-inspired computational model to estimate dense horizontal and vertical disparity maps by exploiting the cortical paradigms of the primate visual system: in particular, we aim to model the disparity sensitivity of the V1-MT visual pathway. The proposed model is based on a first processing stage composed of a bank of spatial band-pass filters and a static nonlinearity, mimicking complex binocular cells. Then, subsequent pooling stages and decoding strategies allow the model to estimate the vector disparity, after having represented it as a population of MT-like units. We assess the proposed model by using standard benchmarking stereo images, the Middlebury dataset, and specific stereo images that have horizontal and vertical disparities, which characterize the stimuli produced by active vision systems. Moreover, we systemically analyze how the different processing stages affect the model performance, and we discuss their implications for the neural modeling.

Near-optimal combination of disparity across a log-polar scaled visual field

“…For disparity estimation we employ a feed-forward neural model that computes 255 vector disparity [42]. This model can be directly applied on cortical images, since 2D 256 vector disparity is computed without explicitly searching for image correspondences 257 along epipolar lines.…”

“…MT cells response Orientation-independent disparity tuning is obtained at the MT 456 level of the model by pooling afferent V1 responses in the spatial and orientation 457 domains, followed by a non-linearity [42,69].…”

“…where G σ pool denotes a Gaussian kernel (standard deviation σ pool ) for the spatial 461 pooling, F (s) = exp(s) is a static non-linearity, specifically an exponential 462 function [42,70], λ is the gain of the non-linearity, and w φ represents the MT linear 463 weights that give origin to the MT tuning. Similarly to what occurs at the V1 layer, we 464 model neural noise at the MT level as:…”

“…479 Since a neural population tuned to two directions (at an angular difference of φ = π/2) 480 can encode the full vector disparity, a neural population of MT units tuned in a range 481 slightly larger than [−π/4, π/4] is able to recover the full vector disparity and shows 482 also a larger selectivity for the horizontal disparity directions (i.e. cells tuned to 483 directions around the horizontal one, in the retinal domain) [42].…”

“…Multi-scale analysis A standard approach to handle multi-scale analysis is to adopt 485 the following steps [42]: (i) a pyramidal decomposition with L levels [74] and (ii) a 486 coarse-to-fine refinement [75]. This is a computationally efficient way to take into account the presence of different spatial frequency channels in the visual cortex and of 488 large range of disparities and spatial frequencies in the real visual signal.…”