DECAF: MEG-Based Multimodal Database for Decoding Affective Physiological Responses

Abstract

In this work, we present DECAF-a multimodal dataset for decoding user physiological responses to affective multimedia content. Different from datasets such as DEAP [15] and MAHNOB-HCI [31], DECAF contains (1) Brain signals acquired using the Magnetoencephalogram (MEG) sensor, which requires little physical contact with the user's scalp and consequently facilitates naturalistic affective response, and (2) Explicit and implicit emotional responses of 30 participants to 40 one-minute music video segments used in [15] and 36 movie clips, thereby enabling comparisons between the EEG vs MEG modalities as well as movie vs music stimuli for affect recognition. In addition to MEG data, DECAF comprises synchronously recorded near-infra-red (NIR) facial videos, horizontal Electrooculogram (hEOG), Electrocardiogram (ECG), and trapeziusElectromyogram (tEMG) peripheral physiological responses. To demonstrate DECAF's utility, we present (i) a detailed analysis of the correlations between participants' self-assessments and their physiological responses and (ii) single-trial classification results for valence, arousal and dominance, with performance evaluation against existing datasets. DECAF also contains timecontinuous emotion annotations for movie clips from seven users, which we use to demonstrate dynamic emotion prediction.

Psychophysiology-Based QoE Assessment: A Survey

“…We argue that such evidence needs to be created collectively and made openly accessible to the research community. An overview of some already available databases is provided in Table IV. True multimodal data sets are provided by Soleymani et al [96] in their database MAHNOB-HCI and Abadi et al [97] in their database for decoding user physiological responses to estimate affective multimedia content (DECAF). Besides some of the measures discussed in our survey, several other peripheral measures are included in these databases such as magnetoencephalogram (MEG), electrooculogram (EOG), electromyogram (EMG), respiratory measurements (RM), facial tracking (FT), and skin temperature (ST).…”

Emotion Recognition Using Neighborhood Components Analysis and ECG/HRV-Based Features

“…Other affect recognition databases which include ECG signals are RECOLA [10], Decaf [11], and Augsburg [12]. DEAP also provide signals from the heart activities but they were quantified as Heart Rate Variability (HRV) measured using Blood Volume Pulse (BVP) on finger [13].…”

The use of data science for education: The case of social-emotional learning

“…There are also open datasets that can be used for research on social and emotional analytics, such as PhysioBank, which includes digital recordings of physiological signals and related data for use by the biomedical research community (Goldberger et al 2000); DEAP, a database for emotion analysis using physiological signals (Koelstra et al 2012); and DECAF, a multimodal dataset for decoding user physiological responses to affective multimedia content (Abadi et al 2015). Verbert et al (2012) further review the availability of such open educational datasets, including dataTEL (http://www.teleurope.eu/ pg/pages/view/50630/), DataShop (https://pslcdatashop.web.cmu.edu/) and Mulce (http:// mulce.univ-bpclermont.fr:8080/PlateFormeMulce/).…”