On the benefits of using Hidden Markov Models to predict emotions

Abstract

The availability of low-cost wireless physiological sensors has allowed the use of emotion recognition technologies in various applications. In this work, we describe a technique to predict emotional states in Russell's two-dimensional emotion space (valence and arousal), using electroencephalography (EEG), electrocardiography (ECG), and electromyography (EMG) signals. For each of the two dimensions, the proposed method uses a classification scheme based on two Hidden Markov Models (HMMs), with the first one trained using positive samples, and the second one using negative samples. The class of new unseen samples is then decided based on which model returns the highest score. The proposed approach was validated on a recently published dataset that contained physiological signals recordings (EEG, ECG, EMG) acquired during a human-horse interaction experiment. The experimental results demonstrate that this approach achieves a better performance than the published baseline methods, achieving an F1-score of 0.940 for valence and 0.783 for arousal, an improvement of more than +0.12 in both cases. CCS CONCEPTS• Information systems → Sentiment analysis; • Mathematics of computing → Kalman filters and hidden Markov models; • Computing methodologies → Supervised learning by classification.

Analyzing Differences in Subjective Annotations by Participants and Third-party Annotators in Multimodal Dialogue Corpus

“…In particular, adapting to the user's emotions is essential for social interaction (Barros et al, 2021). Moreover, different types of information, including multimodal information (e.g., vision and prosody), can be utilized to recognize the user's emotions, as can physiological signals (Katada et al, 2022;Wu et al, 2022). In this paper, emotion is treated as sentiments per exchange.…”