Model Performance for Visual Attention in Real 3D Color Scenes

Abstract

Abstract. Visual attention is the ability of a vision system, be it biological or artificial, to rapidly detect potentially relevant parts of a visual scene. The saliency-based model of visual attention is widely used to simulate this visual mechanism on computers. Though biologically inspired, this model has been only partially assessed in comparison with human behavior. The research described in this paper aims at assessing its performance in the case of natural scenes, i.e. real 3D color scenes. The evaluation is based on the comparison of computer saliency maps with human visual attention derived from fixation patterns while subjects are looking at the scenes. The paper presents a number of experiments involving natural scenes and computer models differing by their capacity to deal with color and depth. The results point on the large range of scene specific performance variations and provide typical quantitative performance values for models of different complexity.

A proto-object based saliency model in three-dimensional space

“…In the second type of depth-saliency models, depth information is treated as an additional feature channel, on the same footing as intensity, color, orientation etc. The final saliency map is then a function of depth as well as of these other features (Ouerhani and Hügli, 2000; Jost et al, 2004; Hügli et al, 2005). …”

“…This agrees with anatomical and neurophysiological data that show that disparity selective cells, which are important for encoding stereoscopic depth information, are found in the same early cortical areas as neurons representing other features used in typical saliency models, like color and orientation (Hubel and Wiesel, 1962; Poggio et al, 1988). Different from previous models (Ouerhani and Hügli, 2000; Jost et al, 2004; Hügli et al, 2005), our model is not only based on basic image features (like color, intensity etc. ) but it includes elements of perceptual organization, in particular proto-objects.…”

“…Integrating depth information into the representation of an object by its contours is a critical difference between our model and other depth saliency models which directly combine depth feature information with 2D information, without taking into account perceptual grouping effects ( e.g. Ouerhani and Hügli, 2000; Jost et al, 2004; Hügli et al, 2005). By modeling the response of border ownership cells to depth edges, we enforce an additional constraint on how depth information is to be combined with 2D information in order to produce proto-objects and the resulting salient points of the image.…”

Model-based control architecture for attentive robots in rescue scenarios

“…Attention based-control have been considered also in Asada (2000, 2002) where the exploration is visuallyguided by the direction which has largest expected information gain. Other approaches, can be found in Winters and Santos-Victor (2001) and Ouerhani et al (2004), Hgli et al (2005), based on feature extraction for self-localization and topological navigation, and Goncalves et al (2000). A long term memory for feature integration during attentive exploration is proposed in Soyer et al (2006).…”

Saliency prediction in the coherence theory of attention

“…In general, approaches have exploited the simulation of saccades either by active cameras, as in Butko, Zhang, Cottrell, and Movellan (2008), Mancas, Pirri, and Pizzoli (2011), or via biologically founded prior models of saliency as in Pichon and Itti (2002), Ackerman and Itti (2005), Hügli, Jost, and Ouerhani (2005), Cerf, Harel, Einhäuser, and Koch (2007), Sala, Sim, Shokoufandeh, and Dickinson (2006), Mahadevan and Vasconcelos (2010), to cite some of the works from the wide literature on saliency prediction.…”