Probabilistic Topic Model for Context-Driven Visual Attention Understanding

Fernández‐Torres, Miguel‐Ángel; González-Díaz, Iván; Díaz-de-María, Fernando

doi:10.1109/tcsvt.2019.2909427

Cited by 4 publications

(4 citation statements)

References 118 publications

(225 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In addition, since the autonomous driving task takes place in diverse scenarios, it is required a hierarchical model that first understands the context in which attention will be guided. Fernández-Torres et al present in [32] their hypothesis about visual attention, similar to the one stated in this work: when looking at particular contexts, visual attention may be attracted by different events or elements in the scene. They try to discover latent sub-tasks that guide the later processing to the areas where those occur, which are modeled as a combination of spatio-temporal features.…”

Section: Related Work a Visual Attentionsupporting

confidence: 70%

“…In this work, we propose a hierarchical multi-task approach which is able to estimate eye fixations based on several contextual conditions of the scene. Following the assumption made by Fernández-Torres et al in [32], our system begins with the following hypothesis: Context-driven visual attention in video can be modeled by considering several conditions that define scenarios which, in turn, are represented as combinations of several spatio-temporal features. For instance, an autonomous driving context can be defined as a cloudy morning across downtown, and each of these conditions can be represented as combinations of features (e.g.…”

Section: Global-local Dynamics Of Visual Attention a Model Overmentioning

confidence: 99%

See 1 more Smart Citation

Interpretable Global-Local Dynamics for the Prediction of Eye Fixations in Autonomous Driving Scenarios

2020

Self Cite

View full text Add to dashboard Cite

Human eye movements while driving reveal that visual attention largely depends on the context in which it occurs. Furthermore, an autonomous vehicle which performs this function would be more reliable if its outputs were understandable. Capsule Networks have been presented as a great opportunity to explore new horizons in the Computer Vision field, due to their capability to structure and relate latent information. In this paper, we present a hierarchical approach for the prediction of eye fixations in autonomous driving scenarios. Context-driven visual attention can be modeled by considering different conditions which, in turn, are represented as combinations of several spatio-temporal features. With the aim of learning these conditions, we have built an encoder-decoder network which merges visual features' information using a global-local definition of capsules. Two types of capsules are distinguished: representational capsules for features and discriminative capsules for conditions. The latter and the use of eye fixations recorded with wearable eye tracking glasses allow the model to learn both to predict contextual conditions and to estimate visual attention, by means of a multi-task loss function. Experiments show how our approach is able to express either frame-level (global) or pixel-wise (local) relationships between features and contextual conditions, allowing for interpretability while maintaining or improving the performance of black-box related systems in the literature. Indeed, our proposal offers an improvement of 29% in terms of information gain with respect to the best performance reported in the literature. INDEX TERMS Top-down visual attention, eye fixation prediction, context-based learning, interpretability, capsule networks, convolutional neural networks, autonomous driving.

show abstract

Section: Related Work a Visual Attentionsupporting

confidence: 70%

Section: Global-local Dynamics Of Visual Attention a Model Overmentioning

confidence: 99%

Interpretable Global-Local Dynamics for the Prediction of Eye Fixations in Autonomous Driving Scenarios

2020

Self Cite

View full text Add to dashboard Cite

show abstract

“…In Chapter 3, we have presented our first approach, which is called visual Attention TOpic Model (ATOM) [126]. Our proposal involves a hierarchical generative probabilistic model for spatio-temporal visual attention prediction and understanding.…”

Section: Discussionmentioning

confidence: 99%

“…For the sake of simplicity, we will only describe the current extended version of the model in this thesis [126], providing the results on visual attention estimation of the first approach as part of the comparison with the state-of-the-art methods carried out in Section 4.4.…”

Section: Related Work and Main Contributionsmentioning

confidence: 99%

Lateral Inhibition-Inspired Convolutional Neural Network for Visual Attention and Saliency Detection

Cao

Huang

Wang

et al. 2018

AAAI

View full text Add to dashboard Cite

Lateral inhibition in top-down feedback is widely existing in visual neurobiology, but such an important mechanism has not be well explored yet in computer vision. In our recent research, we find that modeling lateral inhibition in convolutional neural network (LICNN) is very useful for visual attention and saliency detection. In this paper, we propose to formulate lateral inhibition inspired by the related studies from neurobiology, and embed it into the top-down gradient computation of a general CNN for classification, i.e. only category-level information is used. After this operation (only conducted once), the network has the ability to generate accurate category-specific attention maps. Further, we apply LICNN for weakly-supervised salient object detection.Extensive experimental studies on a set of databases, e.g., ECSSD, HKU-IS, PASCAL-S and DUT-OMRON, demonstrate the great advantage of LICNN which achieves the state-of-the-art performance. It is especially impressive that LICNN with only category-level supervised information even outperforms some recent methods with segmentation-level supervised learning.

show abstract

Generative Augmentation-Driven Prediction of Diverse Visual Scanpaths in Images

Verma

Sen

2024

IEEE Trans. Artif. Intell.

View full text Add to dashboard Cite

Probabilistic Topic Model for Context-Driven Visual Attention Understanding

Abstract: Presented by miguel ángel fernández torres in partial fulfillment of the requirements for the Degree of Doctor in Multimedia and Communications universidad carlos iii de madrid Advisors: dr. iván gonzález díaz dr. fernando díaz de maría

Cited by 4 publications

References 118 publications

Interpretable Global-Local Dynamics for the Prediction of Eye Fixations in Autonomous Driving Scenarios

Interpretable Global-Local Dynamics for the Prediction of Eye Fixations in Autonomous Driving Scenarios

Lateral Inhibition-Inspired Convolutional Neural Network for Visual Attention and Saliency Detection

Generative Augmentation-Driven Prediction of Diverse Visual Scanpaths in Images

Contact Info

Product

Resources

About