Multimedia implicit tagging using EEG signals

Abstract

Electroencephalogram (EEG) signals reflect brain activities associated with emotional and cognitive processes. In this paper, we demonstrate how they can be used to find tags for multimedia content without users' direct input. Alternative methods for multimedia tagging is attracting increasing interest from multimedia community. The new portable EEG helmets are paving the way for employing brain waves in human computer interaction. In this paper, we demonstrate the performance of EEG for tagging purposes using two different scenarios on MAHNOB-HCI database. First, an emotional tagging and classification using a reduced set of electrodes is presented. The emotional responses of 24 participants to short video clips are classified into three classes on arousal and valence. We show how a reduced set of electrodes based on previous studies can preserve and even enhance the emotional classification rate. We then demonstrate the feasibility of using EEG signals for tag relevance tasks. A set of images were shown to participants first, without any tag and then with a relevant or irrelevant tag. The relevance of the tag was assessed based on the EEG responses of the participants in the first second after the tag was depicted. Finally, we demonstrate that by aggregating multiple participants' responses we can significantly improve the tagging accuracy.

Human behavior sensing for tag relevance assessment

“…The best subject independent classification rate was obtained using only eye gaze whose classification rate is 59.5% with the average F1 score of 0.59. This is superior to the average subject dependent results reported in [5,12,14] and in the same line with the subject dependent results reported in [3] using eye gaze, the best F1 score reported was 0.6. It is worth noting that a subject independent approach is more useful in practice due to the higher chance of generalizing over a population.…”

DREAMER: A Database for Emotion Recognition Through EEG and ECG Signals From Wireless Low-cost Off-the-Shelf Devices

“…Welch's overlapped segment averaging estimator is then used to estimate the PSD of each EEG band, using a 256 samples window with an overlap of 128 samples. The logarithms of the PSD from each of the aforementioned bands are extracted from the signal of each of the 14 electrodes in order to be used as features, as also proposed in [1], [11], [12], [37], leading to a total of 42 features (3 for each of the 14 electrodes). Finally, all the features are concatenated into the final feature vector F EEG as follows: Let F iθ , F iα , and F iβ be the logarithm of the PSD for the signal of the i-th electrode, i = 1, 2, ..., 14, for the theta, alpha and beta bands respectively.…”

C2SP-Net: Joint Compression and Classification Network for Epilepsy Seizure Prediction

“…In-sensor signal compression could vastly reduce the transmission bandwidth. Conventional compression methods such as average downsampling [7] and uniform downsampling [8] are substantially limited by the Nyquist sampling rate, and the compressed data can not be reconstructed afterward. Compressive sensing (CS) has been proposed to compress signals at a sub-Nyquist sampling rate in recent years.…”