Fusion of Multiple Visual Cues for Visual Saliency Extraction from Wearable Camera Settings with Strong Motion

Abstract

Abstract. In this paper we are interested in the saliency of visual content from wearable cameras. The subjective saliency in wearable video is studied first due to the psycho-visual experience on this content. Then the method for objective saliency map computation with a specific contribution based on geometrical saliency is proposed. Fusion of spatial, temporal and geometric cues in an objective saliency map is realized by the multiplicative operator. Resulting objective saliency maps are evaluated against the subjective maps with promising results, highlighting interesting performance of proposed geometric saliency model.

Geometrical cues in visual saliency models for active object recognition in egocentric videos

“…The overall model of visual attention predictor proposed in the present work is a followup of the model [5]. The latter itself was based on the previous research of O. Brouard et al [6].…”

“…Prediction of visual attention maps from observer perspective in egocentric video content, has been first proposed in the work by Boujut at al. [5]. In this work three cues: spatial (contrast), temporal (residual motion) and geometric (use of camera position) were fused in an ad hoc fusion framework to produce resultant spatio-temporal saliency maps for this content.…”

“…In the literature, predicted saliency maps derived from image properties are usually compared with human fixation maps in order to assess their validity [5,26,32,34]. However, as the authors of [22] point out, humans do not always focus on objects of interest.…”

“…The locus of important visual information depends on the camera position on a body. In [5] the authors propose a geometrical model which is trained on the basis of a psychovisual experiment. Subjects observed egocentric video content recorded with a shoulder-mounted cameras and their gaze fixations were recorded.…”

“…Hence, in this paper we further study saliency models with non central bias geometrical cues develping the idea of [5]. We assess them in terms of pattern recognition performances using the MAP metric.…”