Video emotion analysis enhanced by recognizing emotion in video comments

Cao, Wei; Zhang, Kun; Wu, Han; Xu, Tong; Chen, Enhong; Lv, Guangyi; He, Ming Xuan

doi:10.1007/s41060-022-00317-0

Cited by 9 publications

(4 citation statements)

References 46 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…Current researchers use TSCs to extract video highlights [4][5][6]. Moreover, current approaches are beginning to apply TSCs to the emotional analysis of videos [7,8]. Bonifazi et al [9] take into account the similarity between patterns and put forth a content semantic network called CS-Net to handle reviews.…”

Section: Related Work 21 Time-sync Commentsmentioning

confidence: 99%

Sentiment Analysis on Online Videos by Time-Sync Comments

et al. 2023

Entropy

View full text Add to dashboard Cite

Video highlights are welcomed by audiences, and are composed of interesting or meaningful shots, such as funny shots. However, video shots of highlights are currently edited manually by video editors, which is inconvenient and consumes an enormous amount of time. A way to help video editors locate video highlights more efficiently is essential. Since interesting or meaningful highlights in videos usually imply strong sentiments, a sentiment analysis model is proposed to automatically recognize sentiments of video highlights by time-sync comments. As the comments are synchronized with video playback time, the model detects sentiment information in time series of user comments. Moreover, in the model, a sentimental intensity calculation method is designed to compute sentiments of shots quantitatively. The experiments show that our approach improves the F1 score by 12.8% and overlapped number by 8.0% compared with the best existing method in extracting sentiments of highlights and obtaining sentimental intensities, which provides assistance for video editors in editing video highlights efficiently.

show abstract

Section: Related Work 21 Time-sync Commentsmentioning

confidence: 99%

Sentiment Analysis on Online Videos by Time-Sync Comments

et al. 2023

Entropy

View full text Add to dashboard Cite

show abstract

“…Deep learning (DL) has gradually penetrated the field of affective computing and promoted the development of video recommendations. Deep learning-based models applied in AVRS in recent years include reinforcement learning (RL) (Leite et al, 2022), convolutional neural network (CNN) (Zhu et al, 2019), long short-term memory (LSTM) (Cao et al, 2022), multilayer perception (Ðor dević Čegar et al, 2020) (MLP), deep hybrid models (DHM) (Mishra et al, 2020), etc. The evolution of AVRS with different algorithms and databases is illustrated in Figure 2.…”

Section: The State-of-the-art Affective Video Recommendation Algorith...mentioning

confidence: 99%

“…Traditional methods SVM/SVR Arapakis et al, 2009a,b;Soleymani and Pantic, 2012;Srivastava and Roy, 2014;Sivakumar et al, 2015;Niu et al, 2017;Dabas et al, 2018;Bohlin et al, 2019Clustering Niu et al, 2013Niu et al, 2016AdaBoost Zhao et al, 2013Shu et al, 2018aMA Tkalčič et al, 2013bDnodxvndv et al, 2018;Kaklauskas et al, 2018CF Soleymani et al, 2009Winoto and Tang, 2010;Tkalčič et al, 2013b;Choi et al, 2016;Diaz et al, 2018CBF Shi et al, 2013Tkalčič et al, 2013b;Deldjoo et al, 2018KG Breitfuss et al, 2021Qi et al, 2021GA Yadati et al, 2014HRS Mugellini et al, 2014Wakil et al, 2015 Deep learning-based methods RL Tripathi et al, 2018;Leite et al, 2022 CNN Hewitt andGunes, 2018;Kwon et al, 2018;Yang et al, 2019;Zhu et al, 2019LSTM Alhagry, 2017Zhang and Zhang, 2017;Ogawa et al, 2018;Nie et al, 2020;Cao et al, 2022MLP Boughrara et al, 2016Ðor dević Čegar et al, 2020;Krishnamurthy, 2020…”

Section: Categories Algorithm/model Publicationsmentioning

confidence: 99%

See 1 more Smart Citation

Affective video recommender systems: A survey

Wang

Zhao

2022

Front. Neurosci.

View full text Add to dashboard Cite

Traditional video recommendation provides the viewers with customized media content according to their historical records (e.g., ratings, reviews). However, such systems tend to generate terrible results if the data is insufficient, which leads to a cold-start problem. An affective video recommender system (AVRS) is a multidiscipline and multimodal human-robot interaction (HRI) system, and it incorporates physical, physiological, neuroscience, and computer science subjects and multimedia resources, including text, audio, and video. As a promising research domain, AVRS employs advanced affective analysis technologies in video resources; therefore, it can solve the cold-start problem. In AVRS, the viewers’ emotional responses can be obtained from various techniques, including physical signals (e.g., facial expression, gestures, and speech) and internal signals (e.g., physiological signals). The changes in these signals can be detected when the viewers face specific situations. The physiological signals are a response to central and autonomic nervous systems and are mostly involuntarily activated, which cannot be easily controlled. Therefore, it is suitable for reliable emotion analysis. The physical signals can be recorded by a webcam or recorder. In contrast, the physiological signals can be collected by various equipment, e.g., psychophysiological heart rate (HR) signals calculated by echocardiogram (ECG), electro-dermal activity (EDA), and brain activity (GA) from electroencephalography (EEG) signals, skin conductance response (SCR) by a galvanic skin response (GSR), and photoplethysmography (PPG) estimating users’ pulse. This survey aims to provide a comprehensive overview of the AVRS domain. To analyze the recent efforts in the field of affective video recommendation, we collected 92 relevant published articles from Google Scholar and summarized the articles and their key findings. In this survey, we feature these articles concerning AVRS from different perspectives, including various traditional recommendation algorithms and advanced deep learning-based algorithms, the commonly used affective video recommendation databases, audience response categories, and evaluation methods. Finally, we conclude the challenge of AVRS and provide the potential future research directions.

show abstract

Exploring contactless techniques in multimodal emotion recognition: insights into diverse applications, challenges, solutions, and prospects

Khan,

Xu,

Liu

et al. 2024

Multimedia Systems

View full text Add to dashboard Cite

In recent years, emotion recognition has received significant attention, presenting a plethora of opportunities for application in diverse fields such as human–computer interaction, psychology, and neuroscience, to name a few. Although unimodal emotion recognition methods offer certain benefits, they have limited ability to encompass the full spectrum of human emotional expression. In contrast, Multimodal Emotion Recognition (MER) delivers a more holistic and detailed insight into an individual's emotional state. However, existing multimodal data collection approaches utilizing contact-based devices hinder the effective deployment of this technology. We address this issue by examining the potential of contactless data collection techniques for MER. In our tertiary review study, we highlight the unaddressed gaps in the existing body of literature on MER. Through our rigorous analysis of MER studies, we identify the modalities, specific cues, open datasets with contactless cues, and unique modality combinations. This further leads us to the formulation of a comparative schema for mapping the MER requirements of a given scenario to a specific modality combination. Subsequently, we discuss the implementation of Contactless Multimodal Emotion Recognition (CMER) systems in diverse use cases with the help of the comparative schema which serves as an evaluation blueprint. Furthermore, this paper also explores ethical and privacy considerations concerning the employment of contactless MER and proposes the key principles for addressing ethical and privacy concerns. The paper further investigates the current challenges and future prospects in the field, offering recommendations for future research and development in CMER. Our study serves as a resource for researchers and practitioners in the field of emotion recognition, as well as those intrigued by the broader outcomes of this rapidly progressing technology.

show abstract

Video emotion analysis enhanced by recognizing emotion in video comments

Cited by 9 publications

References 46 publications

Sentiment Analysis on Online Videos by Time-Sync Comments

Sentiment Analysis on Online Videos by Time-Sync Comments

Affective video recommender systems: A survey

Exploring contactless techniques in multimodal emotion recognition: insights into diverse applications, challenges, solutions, and prospects

Contact Info

Product

Resources

About