Towards Unified Aesthetics and Emotion Prediction in Images

Yu, Jun; Cui, Chaoran; Geng, LeiLei; Ma, Yuling; Yin, Yilong

doi:10.1109/icip.2019.8803388

Cited by 9 publications

(8 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is the case, for example, for colorfulness [122] or dynamic range [32]. Other datasets provide additional attributes such as the emotional response, which are not directly related to aesthetics, but can participate in image preference formation [119]. Finally, aesthetic scores can be augmented with unique identifiers of voters, to facilitate personalized aesthetics applications.…”

Section: Additional Labels and Attributesmentioning

confidence: 99%

Advances and Challenges in Computational Image Aesthetics

Valenzise

Chen

Dufaux

2021

Human Perception of Visual Information

View full text Add to dashboard Cite

HAL is a multi-disciplinary open access archive for the deposit and dissemination of scientific research documents, whether they are published or not. The documents may come from teaching and research institutions in France or abroad, or from public or private research centers. L'archive ouverte pluridisciplinaire HAL, est destinée au dépôt et à la diffusion de documents scientifiques de niveau recherche, publiés ou non, émanant des établissements d'enseignement et de recherche français ou étrangers, des laboratoires publics ou privés.

show abstract

Section: Additional Labels and Attributesmentioning

confidence: 99%

Advances and Challenges in Computational Image Aesthetics

Valenzise

Chen

Dufaux

2021

Human Perception of Visual Information

View full text Add to dashboard Cite

show abstract

“…Studying emotion recognition can also utilize knowledge from other fields. Considering the correlation between aesthetics and emotion of images, Yu et al [134] designed a novel unified aesthetics-emotion hybrid network (AEN) to simultaneously conduct image aesthetic assessment and emotion recognition. Inspired by the emotion of the generation process in brain, Zhang et al [135] developed a multi-subnet neural network to simulate the generation of specific emotional signals and the process of signal suppression in brain neurons.…”

Section: Learning-based Methodsmentioning

confidence: 99%

Affective Image Content Analysis: Two Decades Review and New Perspectives

Zhao

Yao

Yang

et al. 2022

IEEE Trans. Pattern Anal. Mach. Intell.

View full text Add to dashboard Cite

Images can convey rich semantics and induce various emotions in viewers. Recently, with the rapid advancement of emotional intelligence and the explosive growth of visual data, extensive research efforts have been dedicated to affective image content analysis (AICA). In this survey, we will comprehensively review the development of AICA in the recent two decades, especially focusing on the state-of-the-art methods with respect to three main challenges -the affective gap, perception subjectivity, and label noise and absence. We begin with an introduction to the key emotion representation models that have been widely employed in AICA and description of available datasets for performing evaluation with quantitative comparison of label noise and dataset bias. We then summarize and compare the representative approaches on (1) emotion feature extraction, including both handcrafted and deep features, (2) learning methods on dominant emotion recognition, personalized emotion prediction, emotion distribution learning, and learning from noisy data or few labels, and (3) AICA based applications. Finally, we discuss some challenges and promising research directions in the future, such as image content and context understanding, group emotion clustering, and viewer-image interaction.

show abstract

“…Joshi et al [21] begin with background discussions in philosophy, photography, painting, visual arts, and psychology, and introduce a set of aesthetic and affective computational inference computational problems that the research community has been grappling with and the computational frameworks needed to solve them. Yu et al [22] propose a new aesthetic emotion hybrid network for multi-task learning of aesthetic evaluation and emotion recognition. Research confirms that emotion and aesthetics are closely related.…”

Section: Emotion and Aestheticsmentioning

confidence: 99%

“…To incorporate image aesthetic information, we adopt off-the-shelf ResNet with appropriate modifications since its performance on image aesthetic evaluation has been verified by most methods [22,25] . However, ResNet was initially designed for object-centric image recognition.…”

Section: Emotion and Aesthetics Collaborative Learningmentioning

confidence: 99%

“…To enable the model to query more detailed visual features for aesthetics, we modify its initial two 3 × 3 convolutional layers into one 7 × 7 convolution with 𝑠𝑡𝑒𝑝 = 2. Yu et al [22] designed and produced an IAE dataset for image aesthetics and sentiment analysis. This facilitates us to unify image aesthetics and sentiment.…”

Section: Emotion and Aesthetics Collaborative Learningmentioning

confidence: 99%

See 1 more Smart Citation

Exploring affective image representation with visual attention and aesthetic fusion

JiXiong,

Hao,

KangJian

et al. 2023

Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022)

View full text Add to dashboard Cite

Affective image analysis aims to understand the sentiment of different images. The challenge is to develop a discriminative representation that bridges the affective gap between low-level features and high-level emotions. Most existing studies bridge the gap by designing deep models carefully to learn global representations in one shot directly or identify image emotion by extracting features at different levels in the model. They ignore that both local regions of an image and relationships between them impact emotional representation learning. This paper develops an affective image analysis method based on the aesthetic fusion hybrid attention network (AFHA). A modular hybrid attention block is designed to extract image emotion features and model long-range dependencies of images. By stacking hybrid attention blocks in ResNet-style, we obtain an affective representation backbone. Furthermore, considering that image emotion is inseparable from aesthetics, we employ a modified ResNet to extract image aesthetics. Finally, through a fusion strategy, the image's emotion is considered with the aesthetics conveyed. Experiments demonstrate the close relationship between emotion and aesthetics, and our plan has an excellent competitive effect compared with existing methods on the image sentiment analysis dataset.

show abstract

Towards Unified Aesthetics and Emotion Prediction in Images

Cited by 9 publications

References 16 publications

Advances and Challenges in Computational Image Aesthetics

Advances and Challenges in Computational Image Aesthetics

Affective Image Content Analysis: Two Decades Review and New Perspectives

Exploring affective image representation with visual attention and aesthetic fusion

Contact Info

Product

Resources

About