3-D vision and figure-ground separation by visual cortex

Abstract

A neural network theory of three-dimensional (3-D)vision, called FACADE theory, is described. The theory proposes a solution of the classical figure-ground problem for biological vision. It does so by suggesting how boundary representations and surface representations are formed within a boundary contour system (BCS) and a feature contour system (FCS). The BCS and FCS interact reciprocally to form 3-D boundary and surface representations that are mutually consistent. Their interactions generate 3-D percepts wherein occluding and occluded object parts are separated, completed, and grouped. The theory clarifies how preattentive processes of 3-D perception and figure-ground separation interact reciprocally with attentive processes of spatial localization, object recognition, and visual search. A new theory of stereopsis is proposed that predicts how cells sensitive to multiple spatial frequencies, disparities, and orientations are combined by contextsensitive filtering, competition, and cooperation to form coherent BCS boundary segmentations. Several factors contribute to figure-ground pop-out, including: boundary contrast between spatially contiguous boundaries, whether due to scenic differences in luminance, color, spatial frequency, or disparity; partially ordered interactions from larger spatial scales and disparities to smaller scales and disparities; and surface filling-in restricted to regions surrounded by a connected boundary. Phenomena such as 3-D pop-out from a 2-D picture, Da Vinci stereopsis, 3-D neon color spreading, completion of partially occluded objects, and figure-ground reversals are analyzed. The BCS and FCS subsystems model aspects of how the two parvocellular cortical processing streams that join the lateral geniculate nucleus to prestriate cortical area V4 interact to generate a multiplexed representation of Form-And-Color-And-DEpth, or FACADE, within area V4. Area V4 is suggested to support figure-ground separation and to interact with cortical mechanisms of spatial attention, attentive object learning, and visual search. Adaptive resonance theory (ART) mechanisms model aspects of how prestriate visual cortex interacts reciprocally with a visual object recognition system in inferotemporal (IT)

The complementary brain: unifying brain dynamics and modularity

“…LGN-Blob-Thin Stripe-V4 stream while perceptual boundaries are formed in the LGN-InterblobInterstripe-V4 stream 7 . Many experiments have supported this prediction [8][9][10] .…”

“…Even in the seemingly simple case of the Kanizsa square, one often perceives a square hovering in front of four partially occluded circular disks, which seem to be completed behind the square. FACADE theory predicts how surface filling-in is organized to help such figure-ground percepts to occur, in response to both two-dimensional pictures and three-dimensional scenes 7,18 .…”

“…Hierarchical resolution of uncertainty is illustrated by surface filling-in: Discounting the illuminant creates uncertainty by suppressing surface color and brightness signals except near surface discontinuities. Higher stages of filling-in complete the surface representation using properties that are complementary to those whereby boundaries are formed, guided by signals from these boundaries 7,[15][16][17] . Figure 3.…”

“…In this example of neon color spreading, the color in the blue contours spreads in all directions until it fills the square illusory contour. An explanation of this percept is given in reference (7). Three complementary computational properties of visual boundaries and surfaces are also described.…”

“…Such emergent boundary and surface representations, rather than just the energy impinging on our retinas, define the form percepts of which we are consciously aware. Precise orientationally tuned comparisons of left eye and right eye inputs are used to compute sharp estimates of the relative depth of an object from its observer 25,26 , and thereby to form three-dimensional boundary and surface representations of objects separated from their backgrounds 7 .…”