Deep Pairwise Classification and Ranking for Predicting Media Interestingness

Parekh, Jayneel; Tibrewal, Harshvardhan; Parekh, Sanjeel

doi:10.1145/3206025.3206078

Cited by 5 publications

(8 citation statements)

References 10 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…There is then an obvious trend of increasing performance from 2016 to 2017. The best mAP performance for image prediction increases by 25.75%, from 0.2485 on 2016.Image data, Constantin and Ionescu [51], to 0.3125 on 2017.Image data, Parekh et al [52].…”

Section: Analysis Of the Overall Performancementioning

confidence: 98%

“…The authors also performed a mean image subtraction for normalization. Another example is the approach of Parekh et al [52], which achieved the best overall mAP@10 on 2017.Image data, i.e., 0.156. The authors use the fc7 layer of AlexNet as input for their DNN ranking.…”

Section: Per Feature Type Analysismentioning

confidence: 99%

“…However, there are state-of-the-art approaches which outperform the best results. For instance, Parekh et al [52] provides the best overall results on 2017.Image data, achieving a mAP@10 of 0.156. The authors train a DNN network that takes as input pairs of CNN representations of images, to predict which one is more interesting from the pair.…”

Section: Approachmentioning

confidence: 99%

“…Overall system performance. Over the analyzed systems, without taking into account our system fusion experiments, the highest precision for image visual interestingness prediction is obtained via a learning-to-rank DNN using both deep features and a deep ranking approach, Parekh et al [52], mAP of 0.3125. For video, the highest precision is achieved via a late fusion between a learning-to-rank Siamese network and a reinforcement ranking based on a Markov decision process, using both visual and audio descriptors, Wang et al [54], mAP of 0.2228.…”

Section: What Are the Current Capabilities?mentioning

confidence: 99%

“…Data, annotations and techniques should still adapt and come with new improvements to address this subjective task. Nevertheless, one should notice an ascending trend, as performance significantly increased over the years, e.g., in 2016 best mAP is 0.2336 for images, Liem [64], and 0.1815 for videos, Alemida [53]; in 2017 it reaches 0.3075 for images, Permadi et al [73], and 0.2094 for videos, Ben-Ahmed et al [65]; and in 2018 0.3125 for images, Parekh et al [52], and 0.2228 for videos, Wang et al [54]. Therefore, progress is continuously made.…”

Section: What Are the Current Capabilities?mentioning

confidence: 99%

See 4 more Smart Citations

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

et al. 2021

View full text Add to dashboard Cite

In this paper, we report on the creation of a publicly available, common evaluation framework for image and video visual interestingness prediction. We propose a robust data set, the Interestingness10k, with 9,831 images and more than 4 hours of video, interestigness scores determined based on more than 1M pair-wise annotations of 800 trusted annotators, some pre-computed multi-modal descriptors, and 192 system output results as baselines. The data were validated extensively during the 2016-2017 MediaEval benchmark campaigns. We provide an in-depth analysis of the crucial components of visual interestingness prediction algorithms by reviewing the capabilities and the evolution of the MediaEval benchmark systems, as well as of prominent systems from the literature. We discuss overall trends, influence of the employed features and techniques, generalization capabilities and the reliability of results. We also discuss the possibility of going beyond state-of-the-art performance via an automatic, ad-hoc system fusion, and propose a deep MLP-based architecture that outperforms the current state-of-theart systems by a large margin. Finally, we provide the most important lessons learned and insights gained.

show abstract

Section: Analysis Of the Overall Performancementioning

confidence: 98%

Section: Per Feature Type Analysismentioning

confidence: 99%

Section: Approachmentioning

confidence: 99%

Section: What Are the Current Capabilities?mentioning

confidence: 99%

Section: What Are the Current Capabilities?mentioning

confidence: 99%

See 3 more Smart Citations

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

et al. 2021

View full text Add to dashboard Cite

show abstract

Exploring Deep Fusion Ensembling for Automatic Visual Interestingness Prediction

Constantin

Ştefan

Ionescu

2021

Human Perception of Visual Information

View full text Add to dashboard Cite

In the context of the ever growing quantity of multimedia content from social, news and educational platforms, generating meaningful recommendations and ratings now requires a more advanced understanding of their impact on the user, such as their subjective perception. One of the important subjective concepts explored by researchers is visual interestingness. While several definitions of this concept are given in the current literature, in a broader sense, this property attempts to measure the ability of audio-visual data to capture and keep the viewer's attention for longer periods of time. While many computer vision and machine learning methods have been tested for predicting media interestingness, overall, due to the heavily subjective nature of interestingness, the precision of the results is relatively low. In this chapter, we investigate several methods that address this problem from a different angle. We first review the literature on interestingness prediction and present an overview of the traditional fusion mechanisms, such as statistical fusion, weighted approaches, boosting, random forests or randomized trees. Further, we explore the possibility of employing a stronger, novel deep learning-based, system fusion for enhancing the performance. We investigate several types of deep networks for creating the fusion systems, including dense, attention, convolutional and cross-space-fusion networks, while also proposing some input decoration methods that help these networks achieve optimal performance. We present the results, as well as an analysis of the correlation between network structure and overall system performance. Experimental validation is carried out on a publicly available data set and on the systems benchmarked during the 2017 MediaEval Predicting Media Interestingness task.

show abstract

Anomaly and Interestingness Detection in Timed Hierarchical Business Processes

Deng

Zhou

2024

IEEE Trans. Eng. Manage.

View full text Add to dashboard Cite

Deep Pairwise Classification and Ranking for Predicting Media Interestingness

Cited by 5 publications

References 10 publications

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

Visual Interestingness Prediction: A Benchmark Framework and Literature Review

Exploring Deep Fusion Ensembling for Automatic Visual Interestingness Prediction

Anomaly and Interestingness Detection in Timed Hierarchical Business Processes

Contact Info

Product

Resources

About