Predicting Interestingness of Visual Content

Demarty, Claire-Hélène; Sjöberg, Mats; Constantin, Mihai Gabriel; Duong, Ngoc Q. K.; Ionescu, Bogdan; Wang, Hanli

doi:10.1007/978-3-319-57687-9_10

Cited by 14 publications

(11 citation statements)

References 56 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Thus the ability to understand the content plays a key role in such media systems to help them optimize their processing. Different concepts such as visual saliency and aesthetics [1,2], emotion [3], social popularity [4], interestingness [5][6][7], and memorability [8,9] may intervene in the understanding of content. This paper focuses on memorability of images, an emerging and less investigated concept in multimedia and computer vision.…”

Section: Introductionmentioning

confidence: 99%

Deep Learning for Predicting Image Memorability

Squalli-Houssaini

Duong

Marquant

et al. 2018

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

Memorability of media content such as images and videos has recently become an important research subject in computer vision. This paper presents our computation model for predicting image memorability, which is based on a deep learning architecture designed for a classification task. We exploit the use of both convolutional neural network (CNN) -based visual features and semantic features related to image captioning for the task. We train and test our model on the large-scale benchmarking memorability dataset: LaMem. Experiment result shows that the proposed computational model obtains better prediction performance than the state of the art, and even outperforms human consistency. We further investigate the genericity of our model on other memorability datasets. Finally, by validating the model on interestingness datasets, we reconfirm the uncorrelation between memorability and interestingness of images.Index Terms-Image memorability, computational model, deep learning, interestingness, image captioning

show abstract

Section: Introductionmentioning

confidence: 99%

Deep Learning for Predicting Image Memorability

Squalli-Houssaini

Duong

Marquant

et al. 2018

2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)

Self Cite

View full text Add to dashboard Cite

show abstract

“…and 2017 MediaEval benchmark campaigns; Demarty et al [31] book chapter presenting and analyzing the results of the 2016 MediaEval benchmark.…”

Section: Previous Workmentioning

confidence: 99%

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

et al. 2021

Self Cite

View full text Add to dashboard Cite

In this paper, we report on the creation of a publicly available, common evaluation framework for image and video visual interestingness prediction. We propose a robust data set, the Interestingness10k, with 9,831 images and more than 4 hours of video, interestigness scores determined based on more than 1M pair-wise annotations of 800 trusted annotators, some pre-computed multi-modal descriptors, and 192 system output results as baselines. The data were validated extensively during the 2016-2017 MediaEval benchmark campaigns. We provide an in-depth analysis of the crucial components of visual interestingness prediction algorithms by reviewing the capabilities and the evolution of the MediaEval benchmark systems, as well as of prominent systems from the literature. We discuss overall trends, influence of the employed features and techniques, generalization capabilities and the reliability of results. We also discuss the possibility of going beyond state-of-the-art performance via an automatic, ad-hoc system fusion, and propose a deep MLP-based architecture that outperforms the current state-of-theart systems by a large margin. Finally, we provide the most important lessons learned and insights gained.

show abstract

“…The term of memorability is linked with the likelihood that a user will recognize the same photograph after a certain time delay [16,19]. The term of interestingness is linked with the ability of a certain image or video to draw attention of a user to its content and keep this attention for an extent of time [9,13]. Such methods are primarily concerned with evaluating the effect a new, unseen image might have to a user, and hence, they might be not directly applicable to the image selection task.…”

Section: Related Workmentioning

confidence: 99%

Image Selection in Photo Albums

Kuzovkin

Pouli

Cozot

et al. 2018

Proceedings of the 2018 ACM on International Conference on Multimedia Retrieval

View full text Add to dashboard Cite

The selection of the best photos in personal albums is a task that is often faced by photographers. This task can become laborious when the photo collection is large and it contains multiple similar photos. Recent advances on image aesthetics and photo importance evaluation has led to the creation of different metrics for automatically assessing a given image. However, these metrics are intended for the independent assessment of an image, without considering the possible context implicitly present within photo albums. In this work, we perform a user study for assessing how users select photos when provided with a complete photo album-a task that better reflects how users may review their personal photos and collections. Using the data provided by our study, we evaluate how existing state-of-the-art photo assessment methods perform relative to user selection, focusing in particular on deep learning based approaches. Finally, we explore a recent framework for adapting independent image scores to collections and evaluate in which scenarios such an adaptation can prove beneficial.

show abstract

Predicting Interestingness of Visual Content

Cited by 14 publications

References 56 publications

Deep Learning for Predicting Image Memorability

Deep Learning for Predicting Image Memorability

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

Image Selection in Photo Albums

Contact Info

Product

Resources

About