TransIEA: Transformer-Baseartd Image Emotion Analysis

Liu, Chang; Zhao, Shuang; Luo, Yutong; Liu, Guangyuan

doi:10.1109/icccs55155.2022.9846146

Cited by 1 publication

(1 citation statement)

References 11 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…It is of great value in the imitation of human perception to identify different situations in the application of multimedia products. As the main carrier of human perception, visual basic human emotions [4,5]; however, the impressed affects that the audience can feel from film and TV scenes are not simple, and thus, the polarity classification of only one identified category cannot describe the feelings of the audience. We investigate a finegrained affective prediction method to capture the perceived details of affects for film and TV scenes.…”

Section: Introductionmentioning

confidence: 99%

Multitask Learning-Based Affective Prediction for Videos of Films and TV Scenes

Su,

Lin,

Zhang

et al. 2024

Applied Sciences

View full text Add to dashboard Cite

Film and TV video scenes contain rich art and design elements such as light and shadow, color, composition, and complex affects. To recognize the fine-grained affects of the art carrier, this paper proposes a multitask affective value prediction model based on an attention mechanism. After comparing the characteristics of different models, a multitask prediction framework based on the improved progressive layered extraction (PLE) architecture (multi-headed attention and factor correlation-based PLE), incorporating a multi-headed self-attention mechanism and correlation analysis of affective factors, is constructed. Both the dynamic and static features of a video are chosen as fusion input, while the regression of fine-grained affects and classification of whether a character exists in a video are designed as different training tasks. Considering the correlation between different affects, we propose a loss function based on association constraints, which effectively solves the problem of training balance within tasks. Experimental results on a self-built video dataset show that the algorithm can give full play to the complementary advantages of different features and improve the accuracy of prediction, which is more suitable for fine-grained affect mining of film and TV scenes.

show abstract

Section: Introductionmentioning

confidence: 99%