Figure-Ground Organization in Visual Cortex for Natural Scenes

Abstract

Figure-ground organization and border-ownership assignment are essential for understanding natural scenes. It has been shown that many neurons in the macaque visual cortex signal border-ownership in displays of simple geometric shapes such as squares, but how well these neurons resolve border-ownership in natural scenes is not known. We studied area V2 neurons in behaving macaques with static images of complex natural scenes. We found that about half of the neurons were border-ownership selective for contours in natural scenes, and this selectivity originated from the image context. The border-ownership signals emerged within 70 ms after stimulus onset, only ∼30 ms after response onset. A substantial fraction of neurons were highly consistent across scenes. Thus, the cortical mechanisms of figure-ground organization are fast and efficient even in images of complex natural scenes. Understanding how the brain performs this task so fast remains a challenge.

Object segmentation controls image reconstruction from natural scenes

“…These apparently conflicting results owe their distinct patterns to the different stages/mechanisms probed by the 2 different sets of experiments. Image inversion has no impact on the top-down effects that represent the focus of this investigation (red symbols in Fig 2F ), and overall performance was no different between upright and inverted trials ( p = 0.64), just as in the earlier work [ 9 , 10 ], and in line with related single-unit measurements [ 53 ]. It is clear that the representational stage targeted by manipulating probe location is distinct from the stage interrogated by scene inversion [ 52 ].…”

“…We carried out an extensive series of additional experiments to determine whether the top-down effect is robust and how far it generalizes across manipulations of cognitive factors and scene characteristics. We found that it does not require semantic labelling of scene content (it is unaffected by image inversion [ 52 , 53 ] or contrast reversal [ 54 ], see red symbols in Fig 2F and S1 Text ), operates independently of spatial attention (it is unaffected by spatial cueing, see Fig 2G , S1 Text and S1 Video ) but does depend on specific manipulations of various image properties such as spatial frequency (it is retained with highpass but lost with lowpass scenes, see blue/red symbols/confidence intervals in Fig 3C ), orientation (it is reduced by image warping, see Fig 3D–3F ), object-boundary definition (it is retained with cut-out scenes but lost when object boundaries are defined solely by lines, see blue/red symbols/confidence intervals in Fig 3I ) and others (see S1 Text ). For example, although the top-down effect is retained with highpass-filtered images, its magnitude is smaller than observed with undistorted scenes (green symbols in inset to Fig 3C fall above diagonal equality line at p < 0.01).…”

“…which specific electrodes, filtering regime and others). The relevant region within Fig 7E (indicated by black rectangular outline) presents substantial modulations (reflected by red tint) between 30 ms and 70 ms. More accurate estimates of the timescale involved will require further EEG investigations combined with relevant single-unit measurements [ 53 , 68 ].…”

“…It is clear that the representational stage targeted by manipulating probe location is distinct from the stage interrogated by scene inversion [ 52 ]. We speculate that the current experiments probe a stage corresponding to the segmented image where object boundaries are delineated and objects possibly segregated [ 53 , 68 , 85 ], without necessarily assigning semantic content to the segmentation. This concept builds on the distinction between tracing out an object from a scene on the one hand and knowing what that object is on the other hand [ 2 , 21 , 68 , 86 ].…”

Behavioral and neural constraints on hierarchical representations

“…Freeman et al (2013) found via neurophysiology and fMRI that neurons in visual area V2 but not V1 were selective to texture stimuli synthesized according to Portilla and Simoncelli (2000), suggesting that V2 represents texture structure in scenes. Other neurophysiology (Williford & von der Heydt, 2016;Zhou, Friedman, & Von Der Heydt, 2000) and modeling (Zhaoping, 2005) studies have suggested that single units in area V2 contribute to border ownership.…”