A probabilistic topic approach for context-aware visual attention modeling

Abstract

The modeling of visual attention has gained much interest during the last few years since it allows to efficiently drive complex visual processes to particular areas of images or video frames. Although the literature concerning bottom-up saliency models is vast, we still lack of generic approaches modeling topdown task and context-driven visual attention. Indeed, many top-down models simply modulate the weights associated to low-level descriptors to learn more accurate representations of visual attention than those ones of the generic fusion schemes in bottom-up techniques. In this paper we propose a hierarchical generic probabilistic framework that decomposes the complex process of context-driven visual attention into a mixture of latent subtasks, each of them being in turn modeled as a combination of specific distributions of low-level descriptors. The inclusion of this intermediate level bridges the gap between low-level features and visual attention and enables more comprehensive representations of the later. Our experiments on a dataset in which videos are organized by genre demonstrate that, by learning specific distributions for each video category, we can notably enhance the system performance.Index Terms-Top-down visual attention, hierarchical probabilistic framework, context-aware model.

Probabilistic Topic Model for Context-Driven Visual Attention Understanding

“…With the aim of assessing the performance of our approach in comparison with other methods available in the state-of-the-art, we have selected 17 static and dynamic visual attention models, which are representative of the existing diversity for visual attention prediction: we have included both BU and TD or learnable models, a model that uses CNNs to predict, etc., as well as the three reference 3 contain all the results obtained for the assessed methods in CRCNS-ORIG [15] and DIEM [16] databases, respectively, together with those reached by the system proposed in Chapter 3 (ATOM). We also include on the list the first approach we presented in [125], which make use of a linear regressor to estimate visual attention instead of the logistic regressor currently employed, as well as other features. Features and number of topics (K = 40) taken for this previous configuration are those reported in [125].…”

“…We also include on the list the first approach we presented in [125], which make use of a linear regressor to estimate visual attention instead of the logistic regressor currently employed, as well as other features. Features and number of topics (K = 40) taken for this previous configuration are those reported in [125].…”

“…Our LTM-based design was first described in [125]. It introduced an intermediate level formed by latent sub-tasks, which bridges the gap between features and visual attention, and enables to obtain more comprehensive interpretations of attention guidance.…”

“…1. We generated a categorical binary response for each spatial location to model visual attention, in contrast to the continuous variable used in our previous approach [125]. The new system now allows to automatically align the sub-tasks discovered to a binary response by means of a logistic regressor, which fully corresponds to the definition of human fixations.…”

“…2. We extended the initial set of basic and novelty spatio-temporal low-level features presented in [125], including and modeling some new mid-and high-level features related to camera motion estimation and object detection, and taking advantage of powerful paradigms such as Convolutional Neural Networks (CNNs). To do the latter, we make use of the features derived from a recently released deep contrast network for salient object detection with pixel-level accuracy [2].…”