Discovering salient objects from videos using spatiotemporal salient region detection

Abstract

Abstract. Detecting salient objects from images and videos has many useful applications in computer vision. In this paper, a novel spatiotemporal salient region detection approach is proposed. The proposed approach computes spatiotemporal saliency by estimating spatial and temporal saliencies separately. The spatial saliency is computed estimating of color contrast cue and color distribution cue by exploiting patch level and region level image abstractions in a unified way. The aforementioned cues are fused to compute an initial spatial saliency map, which is further refined to emphasize saliencies of objects uniformly, and to suppress saliencies of background noises. The final spatial saliency map is computed by integrating the refined saliency map with center prior map. Temporal saliency is computed based on local and global temporal saliencies estimations using patch level optical flow abstractions. Both local and global temporal saliencies are fused to compute the temporal saliency. Finally, spatial and temporal saliencies are integrated to generate a spatiotemporal saliency map. The proposed temporal and spatiotemporal salient region detection approaches are extensively experimented on challenging salient object detection video datasets. The experimental results show that the proposed approaches achieve an improved performance than several state-of-the-art saliency detection approaches. In order to compensate different needs in respect of the speed/accuracy tradeoff, faster variants of the spatial, temporal and spatiotemporal salient region detection approaches are also presented in this paper.

Salient object detection in video using deep non-local neural networks

“…Saliency detection has been an active research area in computer vision for a long time but most of the previous research has been focused on still-image saliency detection [27,49,11,53,36,41,10]. During the last couple of years, saliency detection in videos has gained a lot of interest as well [69,63,35,32,59,30,34,4,73,14,21,25,9,15,74,35,64]. Most of these methods try to incorporate motion cues into the previously designed saliency detection models and use them together with appearance features to predict video saliency.…”