An Element Sensitive Saliency Model with Position Prior Learning for Web Pages

Gu, Yujun; Chang, J. Morris; Zhang, Ya; Wang, Yanfeng

doi:10.1145/3319921.3319932

Cited by 4 publications

(5 citation statements)

References 24 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Such models perform best over this kind of visual scenes whereas their performances are significantly reduced when the input stimulus does not belong to such a category, such as webpages, UAV (Unmanned Aerial Vehicles) imagery [19], comics [20] to name a few. To cope with the lack of generalisation of visual attention models, it is common to fine-tune deep saliency models with eye-tracking data collected over the target visual scenes, such as comics [20] or webpages [21].…”

Section: Introductionmentioning

confidence: 99%

Can we accurately predict where we look at paintings?

Meur¹,

Pen²,

Cozot³

2020

PLoS ONE

View full text Add to dashboard Cite

The objective of this study is to investigate and to simulate the gaze deployment of observers on paintings. For that purpose, we built a large eye tracking dataset composed of 150 paintings belonging to 5 art movements. We observed that the gaze deployment over the proposed paintings was very similar to the gaze deployment over natural scenes. Therefore, we evaluate existing saliency models and propose a new one which significantly outperforms the most recent deep-based saliency models. Thanks to this new saliency model, we can predict very accurately what are the salient areas of a painting. This opens new avenues for many image-based applications such as animation of paintings or transformation of a still painting into a video clip.

show abstract

Section: Introductionmentioning

confidence: 99%

Can we accurately predict where we look at paintings?

Meur¹,

Pen²,

Cozot³

2020

PLoS ONE

View full text Add to dashboard Cite

show abstract

“…Zheng et al [33] collected human fixations under various tasks such as shopping and formfilling, and predicted saliency based on these task labels. Gu et al [4] captured page biases using a variational autoencoder and used feature detectors to predict webpage saliency. Xia et al's [5] saccadic model predicts scanpaths while viewing webpages, using hand-crafted features for training and is not interpretable.…”

Section: B Attention Prediction On Graphic Designsmentioning

confidence: 99%

“…We compare our AGD-F model with natural saliency models: Itti [16], Deep Gaze II [38], SalGAN [56], DVA [57], SAM-ResNet [12], UAVDSM [37], DI Net [39], EML-NET [58], webpage saliency models: MKL [2], MMF [3] and SPPL [4], TaskWebSal (FreeView) [33] and the graphic design saliency model, UMSI [10]. We quantitatively compare over the metrics: Normalized Scanpath Saliency (NSS), Cross Correlation (CC), KL divergence (KL), the Judd AUC (AUC-J) [59] and the Shuffled AUC (sAUC).…”

Section: A Evaluation Of the Saliency Prediction Modelmentioning

confidence: 99%

“…Previous methods to predict attention allocation during document viewing [2]- [4] have significant limitations. For one, they mainly adopt black-box solutions that lack interpretability, making it difficult to understand how individual document S. Chakraborty (Email: souchakrabor@cs.stonybrook.edu), Z. Wei, C. Kelton, A. Balasubramanian, G.J.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Predicting Visual Attention in Graphic Design Documents

Chakraborty

Wei

Kelton

et al. 2023

IEEE Trans. Multimedia

View full text Add to dashboard Cite

We present a model for predicting visual attention during the free viewing of graphic design documents. While existing works on this topic have aimed at predicting static saliency of graphic designs, our work is the first attempt to predict both spatial attention and dynamic temporal order in which the document regions are fixated by gaze using a deep learning based model. We propose a two-stage model for predicting dynamic attention on such documents, with webpages being our primary choice of document design for demonstration. In the first stage, we predict the saliency maps for each of the document components (e.g. logos, banners, texts, etc. for webpages) conditioned on the type of document layout. These component saliency maps are then jointly used to predict the overall document saliency. In the second stage, we use these layout-specific component saliency maps as the state representation for an inverse reinforcement learning model of fixation scanpath prediction during document viewing. To test our model, we collected a new dataset consisting of eye movements from 41 people freely viewing 450 webpages (the largest dataset of its kind). Experimental results show that our model outperforms existing models in both saliency and scanpath prediction for webpages, and also generalizes very well to other graphic design documents such as comics, posters, mobile UIs, etc. and natural images.

show abstract

“…Metadata saliency describes the confidence of a recommendation in the transferred domain, which reflects the graph-based relationships among items. It is generally known that the visual saliency represents the position-based visual preference [20]. As such, methods for predicting eye fixation maps on websites or mobile interfaces [21][22][23] have been studied, and the relation between the probability of the movement of a mouse pointer and the eye fixation has been discussed [24].…”

Section: Introductionmentioning

confidence: 99%

Re-Enrichment Learning: Metadata Saliency for the Evolutive Personalization of a Recommender System

Ban

Lee

2021

Applied Sciences

View full text Add to dashboard Cite

Many studies have been conducted on recommender systems in both the academic and industrial fields, as they are currently broadly used in various digital platforms to make personalized suggestions. Despite the improvement in the accuracy of recommenders, the diversity of interest areas recommended to a user tends to be reduced, and the sparsity of explicit feedback from users has been an important issue for making progress in recommender systems. In this paper, we introduce a novel approach, namely re-enrichment learning, which effectively leverages the implicit logged feedback from users to enhance user retention in a platform by enriching their interest areas. The approach consists of (i) graph-based domain transfer and (ii) metadata saliency, which (i) find an adaptive and collaborative domain representing the relations among many users’ metadata and (ii) extract attentional features from a user’s implicit logged feedback, respectively. The experimental results show that our proposed approach has a better capacity to enrich the diversity of interests of a user by means of implicit feedback and to help recommender systems achieve more balanced personalization. Our approach, finally, helps recommenders improve user retention, i.e., encouraging users to click more items or dwell longer on the platform.

show abstract

An Element Sensitive Saliency Model with Position Prior Learning for Web Pages

Cited by 4 publications

References 24 publications

Can we accurately predict where we look at paintings?

Can we accurately predict where we look at paintings?

Predicting Visual Attention in Graphic Design Documents

Re-Enrichment Learning: Metadata Saliency for the Evolutive Personalization of a Recommender System

Contact Info

Product

Resources

About