Forward-backward visual saliency propagation in Deep NNs vs internal attentional mechanisms

Obeso, Abraham Montoya; Benois‐Pineau, Jenny; Vázquez, Mireya Saraí García; Acosta, Alejandro

doi:10.1109/ipta.2019.8936125

Cited by 8 publications

(3 citation statements)

References 8 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Attention mechanisms, in classification problem from rgb images, are used to determine which part of information is useful and/or needed to classify an image. Such attention can be obtained by recording the gaze fixations of individuals when looking at the image and performing the same classification task [OBGR19] as a CNN classifier, to create saliency map on image and then propagate it through the layers of the CNN. In DNNs, internal attention mechanisms have become popular.…”

Section: D Attention -What Could It Bring?mentioning

confidence: 99%

3D Convolutional Networks for Action Recognition: Application to Sport Gesture Recognition

Martin,

Benois-Pineau,

Péteri

et al. 2022

Preprint

View full text Add to dashboard Cite

3D convolutional networks is a good means to perform tasks such as video segmentation into coherent spatio-temporal chunks and classification of them with regard to a target taxonomy. In the chapter we are interested in the classification of continuous video takes with repeatable actions, such as strokes of table tennis. Filmed in a free marker less ecological environment, these videos represent a challenge from both segmentation and classification point of view. The 3D convnets are an efficient tool for solving these problems with window-based approaches.

show abstract

Section: D Attention -What Could It Bring?mentioning

confidence: 99%

3D Convolutional Networks for Action Recognition: Application to Sport Gesture Recognition

Martin,

Benois-Pineau,

Péteri

et al. 2022

Preprint

View full text Add to dashboard Cite

show abstract

“…MexCulture: MexCulture dataset [18] contains a total of 20000 images of architectural structures for classification of cultural heritage buildings. The taxonomy ranges into four different classes, i.e., three classes of architectural structures Colonial, Prehispanic, Modern, and structures that cannot be classified as in any of the three mentioned classes are classified as Other.…”

Section: Datasetmentioning

confidence: 99%

“…In this dataset, 284 images are complemented by Gaze Fixation Density Maps (GFDMs), which are computed using the gaze fixations and available at 1 . Gaze fixations were recorded in a psycho-visual experiment where subjects performed a visual task of recognition of an architectural style of historical buildings [18].…”

Section: Datasetmentioning

confidence: 99%

I Saw: A Self-Attention Weighted Method for Explanation of Visual Transformers

Mallick

Benois‐Pineau

Zemmari

2022

2022 IEEE International Conference on Image Processing (ICIP)

View full text Add to dashboard Cite

Recently, visual transformers have shown promising results in tasks such as image classification, segmentation, object detection, etc. The explanation of their decision remains a challenge. This paper focuses on exploiting self-attention for an explanation. We propose a generalized interpretation of the transformers i.e model agnostic but class-specific explanations. The main principle is in the use and weighting self-attention maps of a visual transformer. To evaluate it, we use the popular hypothesis that an explanation is good if it correlates with human perception of a visual scene. Thus, the method has been evaluated against the Gaze Fixation Density Maps obtained in a psycho-visual experiment on a public database. It has been compared with other popular explainers such as Grad-Cam, LRP, Rollout, and Adaptive Relevance methods. The proposed method outperforms the best baseline by 2% in a standard Pearson Correlation Coefficient (PCC) metric.

show abstract

Case Study for Digital Cultural Content Mining

Zemmari

Benois‐Pineau

2020

Deep Learning in Mining of Visual Content

View full text Add to dashboard Cite

Forward-backward visual saliency propagation in Deep NNs vs internal attentional mechanisms

Cited by 8 publications

References 8 publications

3D Convolutional Networks for Action Recognition: Application to Sport Gesture Recognition

3D Convolutional Networks for Action Recognition: Application to Sport Gesture Recognition

I Saw: A Self-Attention Weighted Method for Explanation of Visual Transformers

Case Study for Digital Cultural Content Mining

Contact Info

Product

Resources

About