Single Trial Classification of EEG and Peripheral Physiological Signals for Recognition of Emotions Induced by Music Videos

Abstract

Abstract. Recently, the field of automatic recognition of users' affective states has gained a great deal of attention. Automatic, implicit recognition of affective states has many applications, ranging from personalized content recommendation to automatic tutoring systems. In this work, we present some promising results of our research in classification of emotions induced by watching music videos. We show robust correlations between users' self-assessments of arousal and valence and the frequency powers of their EEG activity. We present methods for single trial classification using both EEG and peripheral physiological signals. For EEG, an average (maximum) classification rate of 55.7% (67.0%) for arousal and 58.8% (76.0%) for valence was obtained. For peripheral physiological signals, the results were 58.9% (85.5%) for arousal and 54.2% (78.5%) for valence.

EEG Correlates of Different Emotional States Elicited during Watching Music Videos

“…These music video clips were carefully selected using a subjective test. More information about the selection procedure can be found in [10]. Before displaying each video a 5-second long baseline was recorded.…”

Multimedia implicit tagging using EEG signals

“…The users' behavior and spontaneous reactions to multimedia data can provide useful information for multimedia indexing with the following scenarios: (i) direct assessment of tags: users spontaneous reactions will be translated into emotional keywords, e.g., funny, disgusting, scary [18,13,16,19]; (ii) assessing the relevance of explicit tags or topic relevance, e.g., agreement or disagreement over a displayed tag or the relevance of the retrieved result [8,12,20,21]; (iii) user profiling: a user's personal preferences can be detected based on her reactions to retrieved data and be used for re-ranking the results; (iv) content summarization: highlight detection is also possible using implicit feedbacks from the users [22,23].…”

Human-centered implicit tagging: Overview and perspectives

“…The existing literature can be divided into two categories, one dealing with using emotional reactions to tag the content with the expressed emotion, e.g., laughter detection for hilarity [5], and the second group of studies using the spontaneous reactions for information retrieval or search results, e.g., eye gaze for relevance feedback [22]. A summary of the recent relevant literature on this topic is given in Table I. There have been also studies using unnimodal or multimodal approaches for detecting the behavioral or emotional responses to multimedia [21], [23], [24], [20], [6]. There is currently a research trend towards estimating emotions from multimedia content automatically [13], [15], [14].…”

“…Koelstra et al [6] recorded EEG and peripheral physiological signals of six participants in response to music videos. Participants rated their felt emotions by means of arousal, valence and like/dislike rating rating.…”

“…The users' behavior and spontaneous reactions to multimedia data can provide useful information for multimedia indexing with the following scenarios: (i) direct assessment of tags: users spontaneous reactions will be translated into emotional keywords, e.g., funny, disgusting, scary [3], [4], [5], [6]; (ii) assessing the correctness of explicit tags or topic relevance, e.g., agreement or disagreement over a displayed tag or the relevance of the retrieved result [7], [8], [9], [10]; (iii) user profiling: a user's personal preferences can be detected based on her reactions to retrieved data and be used for re-ranking the results; (iv) content summarization: highlight detection is also possible using implicit feedbacks from the users [11], [12].…”