Fusion of facial expressions and EEG for implicit affective tagging

Koelstra, Sander; Patras, Ioannis

doi:10.1016/j.imavis.2012.10.002

Cited by 197 publications

(152 citation statements)

References 21 publications

Supporting

Mentioning

144

Contrasting

Order By: Relevance

“…[104] Employing images, sounds, and videos for emotion elicitation is also motivated by affective tagging applications, which consist in automatically assigning tags to multimedia contents. [105,106] In a psychophysiological study of emotion induced by music and film stimuli, Stephens et al [107] replicated the finding of autonomic specific basic emotions and demonstrated that the phenomenon of autonomic nervous system (ANS) specificity of emotion was not a function of the emotion induction technique. In [108], the authors showed that the emotion assessment performance obtained using visual and auditory stimuli is similar.…”

Section: Emotion Elicitationmentioning

confidence: 97%

“…[113] Focusing on valence and arousal, Soleymani et al [105] argue that the arousal dimension is better discriminated by brain activity than the valence dimension. When looking at the studies which analyzed both the classification of valence and arousal on two classes [105,106,110,114,115] the valence accuracy is only marginally higher than the arousal accuracy (valence mean accuracy is 65.6%, arousal mean accuracy is 68.2%), and it is difficult to conclude any potential advantage of neurophysiological signals for arousal assessment. It is unfortunately difficult to compare valence-arousal results with those obtained with basic emotions due to the difference in the number of classes employed.…”

Section: Assessed Emotionsmentioning

confidence: 99%

“…An alternative would be to increase the number of studies working on common standardized datasets, as mentioned in Section 7. Among the studies in Table 1, there is only one that tries to assess emotions continuously in the valence-arousal space using regression, [106] while other studies define classes as areas in this space. Furthermore, there are no studies that have performed continuous emotion recognition over time.…”

Section: Classification / Regressionmentioning

confidence: 99%

See 2 more Smart Citations

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

Mühl

Allison

Nijholt

et al. 2014

Brain-Computer Interfaces

233

159

View full text Add to dashboard Cite

Affective states, moods and emotions, are an integral part of human nature: they shape our thoughts, govern the behavior of the individual, and influence our interpersonal relationships. The last decades have seen a growing interest in the automatic detection of such states from voice, facial expression, and physiological signals, primarily with the goal of enhancing human-computer interaction with an affective component. With the advent of brain-computer interface research, the idea of affective brain-computer interfaces (aBCI), enabling affect detection from brain signals, arose. In this article, we set out to survey the field of neurophysiology-based affect detection. We outline possible applications of aBCI in a general taxonomy of brain-computer interface approaches and introduce the core concepts of affect and their neurophysiological fundamentals. We show that there is a growing body of literature that evidences the capabilities, but also the limitations and challenges of affect detection from neurophysiological activity

show abstract

Section: Emotion Elicitationmentioning

confidence: 97%

Section: Assessed Emotionsmentioning

confidence: 99%

Section: Classification / Regressionmentioning

confidence: 99%

See 1 more Smart Citation

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

Mühl

Allison

Nijholt

et al. 2014

Brain-Computer Interfaces

233

159

View full text Add to dashboard Cite

show abstract

“…To the best of our knowledge this is the first study with classification results using EEG signals. We demonstrate how EEG signals can be used for tagging on one of the publicly available databases [12,13,14].…”

Section: Introductionmentioning

confidence: 99%

“…Koelstra et al [11] used EEG signals to detect emotional tags for music videos. In a recent study, Koelstra and Patras [14] fused facial expressions analysis and EEG signals to detect two classes of arousal, valence and dominance on MAHNOB-HCI database [12].…”

Section: Introductionmentioning

confidence: 99%

Multimedia implicit tagging using EEG signals

Soleymani

Pantić

2013

2013 IEEE International Conference on Multimedia and Expo (ICME)

View full text Add to dashboard Cite

Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.

show abstract

A Proposed LSTM‐Based Neuromarketing Model for Consumer Emotional State Evaluation Using EEG

Gill

Singh

2022

Advanced Analytics and Deep Learning Models

View full text Add to dashboard Cite

Fusion of facial expressions and EEG for implicit affective tagging

Cited by 197 publications

References 21 publications

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

A survey of affective brain computer interfaces: principles, state-of-the-art, and challenges

Multimedia implicit tagging using EEG signals

A Proposed LSTM‐Based Neuromarketing Model for Consumer Emotional State Evaluation Using EEG

Contact Info

Product

Resources

About