Towards Multimodal Search and Visualization of Movies Based on Emotions

Abstract

Movies are one of the most important and impactful forms of entertainment and a powerful vehicle for culture and education, due to the cognitive and emotional impact on the viewers, and technology has been making them more accessible in pervasive services and devices. As such, the huge amount of movies we can access, and the important role emotions play in our lives, make more pertinent the ability to access, visualize and search movies based on their emotional impact. In this paper, we characterize the challenges and approaches in this scenario, and present interactive means to visualize and search movies based on their dominant and actual emotional impact along the movie, with different models and modalities. In particular through emotional highlights and trajectories, the user's emotional state, or a music being played. Music contributes greatly to the emotional impact of movies and it can also be a trigger to get us into one of them in serendipitous moments.

Multi-Modal Affective Computing: An Application in Teaching Evaluation Based on Combined Processing of Texts and Images

“…Figure 6 gives changes in the affection classification accuracy corresponding to different key area numbers, showing the relationship between key area number and affection classification accuracy. For the given six key area numbers (1,2,3,4,5,6), the corresponding affection classification accuracy is 77%, 77.5%, 81.9%, 82.7%, 83.1%, and 83.4%, respectively. Analysis of table data shows that with the increase of key area number, the affection classification accuracy shows an upward trend, indicating more key areas can help the model better capture affection information and improve affection classification accuracy.…”

“…As AI and deep learning are both developing fast these days, significant progresses have been made in fields of computer vision and natural language processing [1][2][3][4][5], which has created a background for the emergence of MAC for combined processing of texts and images [6][7][8][9][10][11][12]. Conventional teaching evaluation emphasizes students' knowledge mastery over their affections [13][14][15], however, the state of affection has a very important influence on students' learning process and outcome.…”