WT Feature Based Emotion Recognition from Multi-channel Physiological Signals with Decision Fusion

Xie, Jinyan; Xu, Xiangmin; Shu, Lin

doi:10.1109/aciiasia.2018.8470381

Cited by 11 publications

(10 citation statements)

References 9 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…The literature has shown that the classification performance improves with the simultaneous exploitation of different signal modalities [21], [130]. Modality fusion can be performed at two main levels: feature fusion [23], [131], [132] and classifier fusion [21], [71], [130], [133]. In the former, features are extracted from each modality and latter concatenated to form a single feature vector space, to be used as input for the ML model.…”

Section: C: Feature Fusionmentioning

confidence: 99%

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

Wang²,

et al. 2019

View full text Add to dashboard Cite

The seminal work on Affective Computing in 1995 by Picard set the base for computing that relates to, arises from, or influences emotions. Affective computing is a multidisciplinary field of research spanning the areas of computer science, psychology, and cognitive science. Potential applications include automated driver assistance, healthcare, human-computer interaction, entertainment, marketing, teaching and many others. Thus, quickly, the field acquired high interest, with an enormous growth of the number of papers published on the topic since its inception. This paper aims to (1) Present an introduction to the field of affective computing though the description of key theoretical concepts; (2) Describe the current state-of-theart of emotion recognition, tracing the developments that helped foster the growth of the field; and lastly, (3) point the literature take-home messages and conclusions, evidencing the main challenges and future opportunities that lie ahead, in particular for the development of novel machine learning (ML) algorithms in the context of emotion recognition using physiological signals.

show abstract

Section: C: Feature Fusionmentioning

confidence: 99%

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

Wang²,

et al. 2019

View full text Add to dashboard Cite

show abstract

“…proposed a new emotion recognition framework based on multi-channel physiological signals, including electrocardiogram (ECG), electromyogram (EMG), and serial clock line (SCL), using the dataset of Bio Vid Emo DB, and evaluated a series of feature selection methods and fusion methods. Finally, they achieved 94.81% accuracy in their dataset [9]. In a study using the MAHNOB-human computer interface (MAHNOB-HCI) dataset, Sander Koelstra et al performed binary classification based on the valence-arousal-dominance emotion model using a fusion of EEG and facial expression and found that the accuracies of valence, arousal, and control were 68.5%, 73%, and 68.5%, respectively [10].…”

Section: Introductionmentioning

confidence: 99%

Combining Facial Expressions and Electroencephalography to Enhance Emotion Recognition

et al. 2019

View full text Add to dashboard Cite

Emotion recognition plays an essential role in human–computer interaction. Previous studies have investigated the use of facial expression and electroencephalogram (EEG) signals from single modal for emotion recognition separately, but few have paid attention to a fusion between them. In this paper, we adopted a multimodal emotion recognition framework by combining facial expression and EEG, based on a valence-arousal emotional model. For facial expression detection, we followed a transfer learning approach for multi-task convolutional neural network (CNN) architectures to detect the state of valence and arousal. For EEG detection, two learning targets (valence and arousal) were detected by different support vector machine (SVM) classifiers, separately. Finally, two decision-level fusion methods based on the enumerate weight rule or an adaptive boosting technique were used to combine facial expression and EEG. In the experiment, the subjects were instructed to watch clips designed to elicit an emotional response and then reported their emotional state. We used two emotion datasets—a Database for Emotion Analysis using Physiological Signals (DEAP) and MAHNOB-human computer interface (MAHNOB-HCI)—to evaluate our method. In addition, we also performed an online experiment to make our method more robust. We experimentally demonstrated that our method produces state-of-the-art results in terms of binary valence/arousal classification, based on DEAP and MAHNOB-HCI data sets. Besides this, for the online experiment, we achieved 69.75% accuracy for the valence space and 70.00% accuracy for the arousal space after fusion, each of which has surpassed the highest performing single modality (69.28% for the valence space and 64.00% for the arousal space). The results suggest that the combination of facial expressions and EEG information for emotion recognition compensates for their defects as single information sources. The novelty of this work is as follows. To begin with, we combined facial expression and EEG to improve the performance of emotion recognition. Furthermore, we used transfer learning techniques to tackle the problem of lacking data and achieve higher accuracy for facial expression. Finally, in addition to implementing the widely used fusion method based on enumerating different weights between two models, we also explored a novel fusion method, applying boosting technique.

show abstract

“…Authors in [100] presented a new emotion recognition framework based on Decision fusion, where three separate classifiers were trained on ECG, EMG and skin conductance level(SCL). The majority voting principle was used to determine a final classification result on the three outputs of the separate classifiers.…”

Section: Multimodal Fusionmentioning

confidence: 99%

A Survey on Physiological Signal Based Emotion Recognition

Ahmad¹,

Khan²

2022

Preprint

View full text Add to dashboard Cite

Physiological Signals are the most reliable form of signals for emotion recognition, as they cannot be controlled deliberately by the subject. Existing review papers on emotion recognition based on physiological signals surveyed only the regular steps involved in the workflow of emotion recognition such as preprocessing, feature extraction, and classification. While these are important steps, such steps are required for any signal processing application. Emotion recognition poses its own set of challenges that are very important to address for a robust system. Thus, to bridge the gap in the existing literature, in this paper, we review the effect of inter-subject data variance on emotion recognition, important data annotation techniques for emotion recognition and their comparison, data preprocessing techniques for each physiological signal, data splitting techniques for improving the generalization of emotion recognition models and different multimodal fusion techniques and their comparison. Finally we discuss key challenges and future directions in this field.

show abstract

WT Feature Based Emotion Recognition from Multi-channel Physiological Signals with Decision Fusion

Cited by 11 publications

References 9 publications

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

A Review, Current Challenges, and Future Possibilities on Emotion Recognition Using Machine Learning and Physiological Signals

Combining Facial Expressions and Electroencephalography to Enhance Emotion Recognition

A Survey on Physiological Signal Based Emotion Recognition

Contact Info

Product

Resources

About