CycleEmotionGAN: Emotional Semantic Consistency Preserved CycleGAN for Adapting Image Emotions

Zhao, Sicheng; Chen, Lin; Xu, Pengfei; Zhao, Sendong; Guo, Yuchen; Krishna, Ravi; Ding, Guiguang; Keutzer, Kurt

doi:10.1609/aaai.v33i01.33012620

Cited by 57 publications

(54 citation statements)

References 29 publications

(55 reference statements)

Supporting

Mentioning

Contrasting

Order By: Relevance

“…However, training a CNN requires massive labeled data and many emotion-image domains lack them. Although some methods were designed to ease the problem, such as generating images similar the target domains [33,36], the generating procedure is fussy and the qualities of generated images can not be guaranteed.…”

Section: Related Workmentioning

confidence: 99%

Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Ding

2019

Neural Process Lett

Self Cite

View full text Add to dashboard Cite

A lot of research attentions have been paid to image emotion analysis in recent years. Meanwhile, as convolutional neural networks (CNNs) have made great successful in computer vision, many researchers start to employ CNN to discriminate image emotions. However, the training procedure of CNNs depends on sufficient labeled data. Therefore, a CNN is hard to perform well in an image domain with scant labeled information. In this paper, we propose a deep transfer learning method for image emotion analysis. The method can leverage rich emotion knowledge from a source domain to the target domain. Our method reduces both marginal and joint domain distribution discrepancies at fully-connected layers. Through this way, we can effectively extract more transferable features and advance the performance of CNNs on poor-label emotion-image domains.

show abstract

Section: Related Workmentioning

confidence: 99%

Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Ding

2019

Neural Process Lett

Self Cite

View full text Add to dashboard Cite

show abstract

“…The Source-only methods are trained on source images and directly test their classification performance on the target images. The Single-source DA methods include CycleGAN (Zhu et al 2017) and CycleEmo-tionGAN (Zhao et al 2019b). For CycleGAN, we extend the original transfer network, i.e.…”

Section: Baselinesmentioning

confidence: 99%

“…(2) Both adaptation methods, CycleGAN (Zhu et al 2017) and CycleEmotionGAN (Zhao et al 2019b), are superior to the source-only methods, while CycleEmotionGAN performs better. This result demonstrates the effectiveness of CycleEmotionGAN for unsupervised domain adaptation in classifying image emotions.…”

Section: Comparison With State-of-the-artmentioning

confidence: 99%

Multi-Source Domain Adaptation for Visual Sentiment Classification

Chen

Zhao

Meng

et al. 2020

AAAI

Self Cite

View full text Add to dashboard Cite

Existing domain adaptation methods on visual sentiment classification typically are investigated under the single-source scenario, where the knowledge learned from a source domain of sufficient labeled data is transferred to the target domain of loosely labeled or unlabeled data. However, in practice, data from a single source domain usually have a limited volume and can hardly cover the characteristics of the target domain. In this paper, we propose a novel multi-source domain adaptation (MDA) method, termed Multi-source Sentiment Generative Adversarial Network (MSGAN), for visual sentiment classification. To handle data from multiple source domains, it learns to find a unified sentiment latent space where data from both the source and target domains share a similar distribution. This is achieved via cycle consistent adversarial learning in an end-to-end manner. Extensive experiments conducted on four benchmark datasets demonstrate that MSGAN significantly outperforms the state-of-the-art MDA approaches for visual sentiment classification.

show abstract

“…In the field of visual emotion analysis, most existing methods focus on emotion prediction [73,35,70,57,62,69,40,23]. Early work uses a variety of hand-crafted features [30,60] including shape features [29] and principlesof-art features [68] to represent the emotions evoked by images.…”

Section: Visual Emotion Analysismentioning

confidence: 99%

Attention-Aware Polarity Sensitive Embedding for Affective Image Retrieval

Yao

She

Zhao

et al. 2019

2019 IEEE/CVF International Conference on Computer Vision (ICCV)

Self Cite

View full text Add to dashboard Cite

Images play a crucial role for people to express their opinions online due to the increasing popularity of social networks. While an affective image retrieval system is useful for obtaining visual contents with desired emotions from a massive repository, the abstract and subjective characteristics make the task challenging. To address the problem, this paper introduces an Attention-aware Polarity Sensitive Embedding (APSE) network to learn affective representations in an end-to-end manner. First, to automatically discover and model the informative regions of interest, we develop a hierarchical attention mechanism, in which both polarity-and emotion-specific attended representations are aggregated for discriminative feature embedding. Second, we present a weighted emotion-pair loss to take the interand intra-polarity relationships of the emotional labels into consideration. Guided by attention module, we weight the sample pairs adaptively which further improves the performance of feature embedding. Extensive experiments on four popular benchmark datasets show that the proposed method performs favorably against the state-of-the-art approaches.

show abstract

CycleEmotionGAN: Emotional Semantic Consistency Preserved CycleGAN for Adapting Image Emotions

Cited by 57 publications

References 29 publications

Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Deep Transfer Learning for Image Emotion Analysis: Reducing Marginal and Joint Distribution Discrepancies Together

Multi-Source Domain Adaptation for Visual Sentiment Classification

Attention-Aware Polarity Sensitive Embedding for Affective Image Retrieval

Contact Info

Product

Resources

About