Proto-Object Based Saliency Model With Texture Detection Channel

Abstract

The amount of visual information projected from the retina to the brain exceeds the information processing capacity of the latter. Attention, therefore, functions as a filter to highlight important information at multiple stages of the visual pathway that requires further and more detailed analysis. Among other functions, this determines where to fixate since only the fovea allows for high resolution imaging. Visual saliency modeling, i.e. understanding how the brain selects important information to analyze further and to determine where to fixate next, is an important research topic in computational neuroscience and computer vision. Most existing bottom-up saliency models use low-level features such as intensity and color, while some models employ high-level features, like faces. However, little consideration has been given to mid-level features, such as texture, for visual saliency models. In this paper, we extend a biologically plausible proto-object based saliency model by adding simple texture channels which employ nonlinear operations that mimic the processing performed by primate visual cortex. The extended model shows statistically significant improved performance in predicting human fixations compared to the previous model. Comparing the performance of our model with others on publicly available benchmarking datasets, we find that our biologically plausible model matches the performance of other models, even though those were designed entirely for maximal performance with little regard to biological realism.

Correspondence between Monkey Visual Cortices and Layers of a Saliency Map Model Based on a Deep Convolutional Neural Network for Representations of Natural Images

“… Poort et al (2012 , 2016 ) performed neurophysiological experiments that indicated that the neural responses in V1 for representing the figure occurred according to the process of edge detection. Furthermore, biologically plausible saliency map models have implied that the neural mechanism of figure–ground segregation plays an important role in predicting the locations of attentional selection and in improving the prediction accuracy of the human gaze ( Li, 1999a ; Zhaoping, 2003 , 2014 ; Russell et al, 2014 ; Wagatsuma, 2019 ; Uejima et al, 2020 ). Our analyses demonstrated that the responses of intermediate and higher-intermediate layers (layer 4 ReLU, layer 5 ReLU, and layer 6; see Fig.…”

“…In this study, we used RDMs ( Kriegeskorte et al, 2008 ) to compare the characteristics of the responses of the DCNN saliency map model with those of the neural representation in visual cortices. Our analysis methods and metrics used in this study are applicable to various other saliency map models ( Itti and Koch, 2000 ; Kümmerer et al, 2014 , 2017 ; Russell et al, 2014 ; Pan et al, 2017 ; Liu and Han, 2018 ; Wagatsuma, 2019 ; Uejima et al, 2020 ). Our analysis results and the V1 saliency hypothesis implied that the activities of model neurons with similar characteristics to V1 responses were the basis for better gaze prediction accuracy for the saliency map models.…”

“…In this model, the most salient location in a visual scene induces attentional selection. From the original model ( Itti et al, 1998 ), various saliency map models based on the visual system have been proposed ( Russell et al, 2014 ; Wagatsuma, 2019 ; Uejima et al, 2020 ) in which the activities of model neurons in early vision are the first, and necessary, process for organizing the saliency map. The crucial role of responses in the primary visual cortex (V1) used for computing visual saliency has been demonstrated by various studies, including physiological, psychophysical, and computational works (V1 saliency hypothesis; Li, 1999a , 2002 ; Jingling and Zhaoping, 2008 ; Zhaoping, 2014 , 2019 ; Yan et al, 2018 ).…”

A Neuromorphic Proto-Object Based Dynamic Visual Saliency Model With a Hybrid FPGA Implementation

“…It further performed better than all models except for Fang et al [34] for the KLD metric. This is most likely due to the additional texture feature computed in the Fang et al model, which is not considered in this model, but can easily be integrated [19]. Studies have shown texture plays an important role in early vision and perception [73].…”

Characteristics of Visual Saliency Caused by Character Feature for Reconstruction of Saliency Map Model

“…A graph-based saliency map (GBVS) is constructed by introducing a fully connected graph into Itti’s model [ 2 ]. The proto-object saliency map model employs a biologically plausible feature [ 3 , 4 ]. The hybrid model of visual saliency was developed by using low, middle, and high-level image features [ 5 ].…”