Real-time affect detection in virtual reality: a technique based on a three-dimensional model of affect and EEG signals

Abstract

This manuscript explores the development of a technique for detecting the affective states of Virtual Reality (VR) users in real-time. The technique was tested with data from an experiment where 18 participants observed 16 videos with emotional content inside a VR home theater, while their electroencephalography (EEG) signals were recorded. Participants evaluated their affective response toward the videos in terms of a three-dimensional model of affect. Two variants of the technique were analyzed. The difference between both variants was the method used for feature selection. In the first variant, features extracted from the EEG signals were selected using Linear Mixed-Effects (LME) models. In the second variant, features were selected using Recursive Feature Elimination with Cross Validation (RFECV). Random forest was used in both variants to build the classification models. Accuracy, precision, recall and F1 scores were obtained by cross-validation. An ANOVA was conducted to compare the accuracy of the models built in each variant. The results indicate that the feature selection method does not have a significant effect on the accuracy of the classification models. Therefore, both variations (LME and RFECV) seem equally reliable for detecting affective states of VR users. The mean accuracy of the classification models was between 87% and 93%.

A Comprehensive Survey of Brain-Computer Interface Technology in Healthcare: Research Perspectives

“…In the context of EEG based-emotion classification, authors proposed a novel approach that combines electroencephalography (EEG) and galvanic skin response (GSR) to capture and classify emotional states. The authors also performed feature selection using a Recursive Feature Elimination (RFE) algorithm to identify the most relevant features for emotion recognition [100]. The results of the study suggest that the proposed BCI system using EEG and GSR data can achieve high accuracy in recognizing emotions in people with visual disabilities.…”

Em direção à um checklist para garantia da qualidade em atributos de acessibilidade das tecnologias emergentes de RV e RA

“…Como já tratado, são inúmeros os benefícios que a RA e a RV trazem. Em [Pinilla et al 2023], é explorado o desenvolvimento de uma tecnologia para detectar estados afetivos. A partir de sinais de eletroencefalografia (EEG) com experimentos realizados em realidade virtual em tempo real.…”

The Impact of Light Conditions on Neural Affect Classification: A Deep Learning Approach

“…Affect-eliciting pictures like those in the International Affective Picture System (IAPS) have already been used to classify affect in terms of valence and arousal with machine learning approaches. In these studies, binary classification was performed on one of the two parameters rather than classification on a two-dimensional valence-arousal plane [11,12].…”