A recurrent neural model for proto-object based contour integration and figure-ground segregation

Abstract

Visual processing of objects makes use of both feedforward and feedback streams of information. However, the nature of feedback signals is largely unknown, as is the identity of the neuronal populations in lower visual areas that receive them. Here, we develop a recurrent neural model to address these questions in the context of contour integration and figure-ground segregation. A key feature of our model is the use of grouping neurons whose activity represents tentative objects (“proto-objects”) based on the integration of local feature information. Grouping neurons receive input from an organized set of local feature neurons, and project modulatory feedback to those same neurons. Additionally, inhibition at both the local feature level and the object representation level biases the interpretation of the visual scene in agreement with principles from Gestalt psychology. Our model explains several sets of neurophysiological results (Zhou et al, 2000; Qiu et al, 2007; Chen et al, 2014), and makes testable predictions about the influence of neuronal feedback and attentional selection on neural responses across different visual areas. Our model also provides a framework for understanding how object-based attention is able to select both objects and the features associated with them.

Correspondence between Monkey Visual Cortices and Layers of a Saliency Map Model Based on a Deep Convolutional Neural Network for Representations of Natural Images

“…Our results showed that the characteristics of V1-like representations in early layers of the DCNN saliency map model were obtained during early training epochs, whereas the and that rapid feedback signals from higher-level visual areas play crucial roles in the neural representation of the figural region, which might correspond to suggestions from the computational models for understanding the neural mechanism underlying figureground segregation (Li, 1999a;Zhaoping, 2003;Sakai and Nishimura, 2006;Sakai et al, 2012;Craft et al, 2007;Mihalas et al, 2011;Zhaoping, 2014;Wagatsuma et al, 2016;Hu and Niebur, 2017).…”

“…If model neurons in intermediate and higher-intermediate layers are selective to the figural region for computing visual saliency as discussed in the previous section, the selectivity of figure–ground segregation in these layers might develop after the early layers obtain orientation selectivity and the function of edge detection, similar to V1 neurons. These results suggest that feedforward processing based on edge detection in early vision underlies the selectivity of figure–ground segregation in intermediate-level visual areas, and that rapid feedback signals from higher-level visual areas play crucial roles in the neural representation of the figural region, which might correspond to suggestions from the computational models for understanding the neural mechanism underlying figure–ground segregation ( Li, 1999a ; Zhaoping, 2003 , 2014 ; Sakai and Nishimura, 2006 ; Craft et al, 2007 ; Mihalas et al, 2011 ; Sakai et al, 2012 ; Wagatsuma et al, 2016 ; Hu and Niebur, 2017 ).…”

Figure-Ground Organization in Natural Scenes: Performance of a Recurrent Neural Model Compared with Neurons of Area V2

“…Furthermore, although we do not make use of the population activity in this study, in practice we find that the combination of scale invariance and recurrent processing allows the model to accurately predict figure-ground relationships in natural scenes. We also do not rule out the possibility that other types of grouping neurons may also exist, including those that respond to straight contours (Hu and Niebur, 2017), gratings (Hegdé and Van Essen, 2007), illusory surfaces (Cox et al, 2013), or 3D surfaces (He and Nakayama, 1995; Hu et al, 2015). For the sake of simplicity in this proof-of-concept study, we do not attempt to model the whole array of grouping neurons that may exist.…”

A Neurally Inspired Model of Figure Ground Organization with Local and Global Cues

“…But, it is a feedback model tested only on simple geometric shapes, not real-world natural images. Several models [16,[63][64][65] with similar computational mechanisms have been proposed to explain various phenomena related to FGO, saliency, spatial attention, and so forth. A model akin to Reference [13] was proposed in Reference [66], where in addition to G cells the model consists of region cells at multiple scales.…”