A Synaesthetic Approach for Image Slideshow Generation

Xiang, Yangyang; Kankanhalli, Mohan

doi:10.1109/icme.2012.75

Cited by 2 publications

(2 citation statements)

References 15 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…They separately extracted hand-crafted features, learned emotion classifiers, and composited images and music based on the predicted emotions. Many methods follow this pipeline [34,48,53,61,86]. They (1) extracted more discriminative emotion features, such as low-level color [9,34,48,53,61] and mid-level principlesof-art [86] for image; (2) employed different emotion representation models, from categorical states [9,34,53,61] to dimensional space [48,86]; (3) correspondingly learned different classifiers, from Support Vector Machine [9], Naive Bayes, and Decision Tree [53] to Support Vector Regression [86]; and (4) used different composition strategies to match image and music, from emotion category comparison [9,34,53,61] to Euclidean distance [48,86].…”

Section: Related Workmentioning

confidence: 99%

“…Such emotion-based matching is essential for various applications [57], such as affective cross-modal retrieval, emotion-based multimedia slideshow, and emotion-aware recommendation systems. The early emotion-based matching methods mainly employ a shallow pipeline [9,34,48,53,61,86], i.e. extracting hand-crafted features and training matching classifiers (or training emotion classifiers for both modalities and then learning matching similarities).…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

Emotion-Based End-to-End Matching Between Image and Music in Valence-Arousal Space

Zhao

Yao

et al. 2020

Proceedings of the 28th ACM International Conference on Multimedia

View full text Add to dashboard Cite

Both images and music can convey rich semantics and are widely used to induce specific emotions. Matching images and music with similar emotions might help to make emotion perceptions more vivid and stronger. Existing emotion-based image and music matching methods either employ limited categorical emotion states which cannot well reflect the complexity and subtlety of emotions, or train the matching model using an impractical multi-stage pipeline.In this paper, we study end-to-end matching between image and music based on emotions in the continuous valence-arousal (VA) space. First, we construct a large-scale dataset, termed Image-Music-Emotion-Matching-Net (IMEMNet), with over 140K image-music pairs. Second, we propose cross-modal deep continuous metric learning (CDCML) to learn a shared latent embedding space which preserves the cross-modal similarity relationship in the continuous matching space. Finally, we refine the embedding space by further preserving the single-modal emotion relationship in the VA spaces of both images and music. The metric learning in the embedding space and task regression in the label space are jointly optimized for both cross-modal matching and single-modal VA prediction. The extensive experiments conducted on IMEMNet demonstrate the superiority of CDCML for emotion-based image and music matching as compared to the state-of-the-art approaches. CCS CONCEPTS• Information systems → Sentiment analysis; Multimedia information systems.

show abstract