Dynamic Whitening Saliency

Abstract

General dynamic scenes involve multiple rigid and flexible objects, with relative and common motion, camera induced or not. The complexity of the motion events together with their strong spatio-temporal correlations make the estimation of dynamic visual saliency a big computational challenge. In this work, we propose a computational model of saliency based on the assumption that perceptual relevant information is carried by high-order statistical structures. Through whitening, we completely remove the second-order information (correlations and variances) of the data, gaining access to the relevant information. The proposed approach is an analytically tractable and computationally simple framework which we call Dynamic Adaptive Whitening Saliency (AWS-D). For model assessment, the provided saliency maps were used to predict the fixations of human observers over six public video datasets, and also to reproduce the human behavior under certain psychophysical experiments (dynamic pop-out). The results demonstrate that AWS-D beats state-of-the-art dynamic saliency models, and suggest that the model might contain the basis to understand the key mechanisms of visual saliency. Experimental evaluation was performed using an extension to video of the well-known methodology for static images, together with a bootstrap permutation test (random label hypothesis) which yields additional information about temporal evolution of the metrics statistical significance.

A Comparison Study of Saliency Models for Fixation Prediction on Infants and Adults

“…Zhu et al [24] proposed a robust background measure to characterize the spatial layout of the image, and then proposed an optimized framework to integrate low-level cues. Leboran et al [25] developed a dynamic adaptive whitening saliency, which is based on high level statistical structures. Recently, Wang et al [26] developed a saliency model by combining 13 existing state-of-the-art saliency models.…”

Psychophysical evaluation of individual low-level feature influences on visual attention

“…In that respect, to present the same stimuli with several observations for each feature contrast and distinct cueing would reveal absolute influences from endogenous guidance. Our study could be extended by analyzing the influence of dynamic scenes on saliency modeling [Leboran et al, 2017] [Riche and Mancas, 2016b] using synthetic videos with both static or dynamic camera. In that direction, it would be able to see the interaction between low-level visual features and temporally-variant features.…”

Surgical Phase Recognition of Short Video Shots based on Temporal Modeling of Deep Features

“…Integration of visual saliency in CNN-based content-based image representation is a major trend nowadays. The main idea is to generate a saliency map that represents the most salient regions of the input image, without any prior assumption, based on various criteria such as color-and texture-contrast [17], [18]. Recent works have shown that using as input to the CNN a salient, instead of a center/resized crop, image provides better classification results [19].…”

“…Recent works have shown that using as input to the CNN a salient, instead of a center/resized crop, image provides better classification results [19]. In this work we have employed the static version of the adaptive whitening saliency (AWS) methodology, which has shown superior performance in predicting human attention [17]. Recently, this method was applied for keyframe extraction from video shots of LC operations [20].…”