Possible functions of contextual modulations and receptive field nonlinearities: Pop-out and texture segmentation

Abstract

When analyzing a visual image, the brain has to achieve several goals quickly. One crucial goal is to rapidly detect parts of the visual scene that might be behaviorally relevant, while another one is to segment the image into objects, to enable an internal representation of the world. Both of these processes can be driven by local variations in any of several image attributes such as luminance, color, and texture. Here, focusing on texture defined by local orientation, we propose that the two processes are mediated by separate mechanisms that function in parallel. More specifically, differences in orientation can cause an object to “pop out” and attract visual attention, if its orientation differs from that of the surrounding objects. Differences in orientation can also signal a boundary between objects and therefore provide useful information for image segmentation. We propose that contextual response modulations in primary visual cortex (V1) are responsible for orientation pop-out, while a different kind of receptive field nonlinearity in secondary visual cortex (V2) is responsible for orientation-based texture segmentation. We review a recent experiment that led us to put forward this hypothesis along with other research literature relevant to this notion.

Cross-orientation suppression in visual area V2

“…The biphasic pooling was prominent in the data set (34% of neurons), with uniform pooling observed in the remaining cases. Overall, the observed patterns of selectivity based on locally orthogonal excitatory and suppressive features that are repeated across a range of spatial position could mediate the observed selectivity of V2 responses to textures1718 and texture boundaries132021.…”

On texture, form, and fixational eye movements

“…Because textures are defined as patterns with position-invariant statistical properties [10], the responses of neurons tuned to textures are often analyzed using multi-stage models that combine position-invariance with selective tuning to conjunctions of edges of different angles [1**, 2–5,11,12] (Figure 2). Analyses of V2 responses to natural stimuli using such models have yielded three organizing principles for their feature-selectivity [11].…”

“…Within each channel, the stimulus patch is projected onto a set of relevant features (same for all patches and shown here as heat maps) to which we refer as first-order features. The output of a projection onto a given feature is passed through a quadratic function (with a potentially non-zero linear term) [1**]. These outputs are summed and passed through a compressive nonlinearity.…”

“…The output of each quadratic block within each patch is summed, with weights that could be either positive or negative, and the result passed through a soft threshold function to yield a prediction for the firing rate. The block output filtering allows one to connect with filter-rectify-filter (FRF) models [1**,3,4,41–44]. ( B ) Left: Prototypical arrangement of features in the FRF model.…”

“…Impressionist paintings serve to illustrate the point that visual forms can be defined by changes in textures as well as by changes in brightness [1**, 2–5] (see Fig. 1 for an example).…”