Computational Understanding of Visual Interestingness Beyond Semantics

Abstract

Understanding visual interestingness is a challenging task addressed by researchers in various disciplines ranging from humanities and psychology to, more recently, computer vision and multimedia. The rise of infographics and the visual information overload that we are facing today have given this task a crucial importance. Automatic systems are increasingly needed to help users navigate through the growing amount of visual information available, either on the web or our personal devices, for instance by selecting relevant and interesting content. Previous studies indicate that visual interest is highly related to concepts like arousal, unusualness, or complexity, where these connections are found based on psychological theories, user studies, or computational approaches. However, the link between visual interestingness and other related concepts has been only partially explored so far, for example, by considering only a limited subset of covariates at a time. In this article, we present a comprehensive survey on visual interestingness and related concepts, aiming to bring together works based on different approaches, highlighting controversies, and identifying links that have not been fully investigated yet. Finally, we present some open questions that may be addressed in future works. Our work aims to support researchers interested in visual interestingness and related subjective or abstract concepts, providing an in-depth overlook at state-of-the-art theories in humanities and methods in computational approaches, as well as providing an extended list of datasets.

Chromatic-Luminance and True-Texture Analysis for Image Mining in the Social Sciences

“…architecture and the built environment) -there is a shared interest in streamlining the process of feature extraction from visual data, and importantly, in building deeper understanding around visual communication and aesthetics in general (see for example : Lyons, 2017;Ibarra et al, 2017;Joo and Steinert-Threlkeld, 2018;Steinert-Threlkeld, 2019;Al-Halah and Grauman, 2020;Xi et al, 2020). This divides broadly into understanding individual perceptions and preferences (Stamps, 2002;Witzel et al, 2017), as well as building generalizations about desirable and compelling image composition -a theme currently explored across the social science and computer science literatures (Badea et al, 2018;Constantin et al, 2019;Matz et al, 2019). This research note aims to highlight that scalability should not be an overwhelming factor in methodology design in image mining in the social sciences, especially at the expense of detail and nuance.…”

Computational Creativity and Music Generation Systems: An Introduction to the State of the Art

“…Galanter ( 2012 ) made a review on metrics and methods to evaluate aesthetic value of computer generated artifacts, that is a vital part of many CC systems. Within the field of computer vision, the use of fuzzy logic applied to visual features was suggested for the automatic evaluation of complexity, as well as interestingness and aesthetic value (Cardaci et al, 2009 ; Tabacchi and Termini, 2011 ; Constantin et al, 2019 ). Shaker et al ( 2016 ) focus on procedural content generation, and describe how it is possible to give a visual indication of the capabilities of a system in terms of the variety of products it can generate.…”

The Transition From Intelligent to Affective Tutoring System: A Review and Open Issues

“…However, it is not possible to show all the methods for emotion recognition channels, and in this current study, we are describing the most commonly used emotion recognition channels found in the literature. To learn more about other emotion recognition channels, one can refer to [128]- [130]. • Pre-processing Pre-processing is carried out to ensure the precision of the feature extraction process because the performance of the facial expression recognition (FER) system mostly depends upon it.…”