A machine learning approach to predict emotional arousal and valence from gaze extracted features

Skaramagkas, Vasileios; Ktistakis, Emmanouil; Manousos, Dimitris; Tachos, Nikolaos S.; Kazantzaki, Eleni; Tripoliti, Evanthia E.; Fotiadis, Dimitrios I.; Tsiknakis, Manolis

doi:10.1109/bibe52308.2021.9635346

Cited by 6 publications

(3 citation statements)

References 16 publications

Supporting

Mentioning

Contrasting

Order By: Relevance

“…In agreement with prior research, our findings reveal the effect of emotional charge on eye movements and pupillary responses. Specifically and in line with the ideas of [ 11 , 12 , 16 ], high arousal and positive valence levels can be successfully identified based solely on eye-tracking features. Furthermore, when comparing our work to those of [ 13 , 14 , 15 ], our results demonstrate significant improvements in terms of accuracy which can be attributed in part to the significant larger size of the training database.…”

Section: Discussionmentioning

confidence: 68%

“…Few researchers, however, have utilized ocular features as the only predictor of emotional arousal and valence levels. These studies attempt to solve either binary [ 11 , 12 ] or multi-class classification problems [ 13 , 14 , 15 , 16 ] with success rates for multi-class cases remaining below 80%, whereas binary classification approaches have proven to be more effective, with success rates reaching as high as 93%. Furthermore, neural network-based approaches have been used for the discrimination of various emotional levels.…”

Section: Introductionmentioning

confidence: 99%

See 1 more Smart Citation

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

et al. 2023

Self Cite

View full text Add to dashboard Cite

Affective state estimation is a research field that has gained increased attention from the research community in the last decade. Two of the main catalysts for this are the advancement in the data analysis using artificial intelligence and the availability of high-quality video. Unfortunately, benchmarks and public datasets are limited, thus making the development of new methodologies and the implementation of comparative studies essential. The current work presents the eSEE-d database, which is a resource to be used for emotional State Estimation based on Eye-tracking data. Eye movements of 48 participants were recorded as they watched 10 emotion-evoking videos, each of them followed by a neutral video. Participants rated four emotions (tenderness, anger, disgust, sadness) on a scale from 0 to 10, which was later translated in terms of emotional arousal and valence levels. Furthermore, each participant filled three self-assessment questionnaires. An extensive analysis of the participants’ answers to the questionnaires’ self-assessment scores as well as their ratings during the experiments is presented. Moreover, eye and gaze features were extracted from the low-level eye-recorded metrics, and their correlations with the participants’ ratings are investigated. Finally, we take on the challenge to classify arousal and valence levels based solely on eye and gaze features, leading to promising results. In particular, the Deep Multilayer Perceptron (DMLP) network we developed achieved an accuracy of 92% in distinguishing positive valence from non-positive and 81% in distinguishing low arousal from medium arousal. The dataset is made publicly available.

show abstract

Section: Discussionmentioning

confidence: 68%

Section: Introductionmentioning

confidence: 99%

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

et al. 2023

Self Cite

View full text Add to dashboard Cite

show abstract

“…But their proposed approach has lower accuracy for multimodal methods using VGAF and AFEW datasets. For predicting and understanding the affective attitude of students, it is very important to predict and analyze the valence and arousal [27].…”

Section: Related Workmentioning

confidence: 99%

Multimodal Engagement Recognition From Image Traits Using Deep Learning Techniques

Sukumaran,

Manoharan

2024

IEEE Access

View full text Add to dashboard Cite

Learner engagement is a significant factor determining the success of implementing an intelligent educational network. Currently the use of Massive Open Online Courses has increased because of the flexibility offered by such online learning systems. The COVID period has encouraged practitioners to continue to engage in new ways of online and hybrid teaching. However, monitoring student engagement and keeping the right level of interaction in an online classroom is challenging for teachers. In this paper we propose an engagement recognition model by combining the image traits obtained from a camera, such as facial emotions, gaze tracking with head pose estimation and eye blinking rate. In the first step, a face recognition model was implemented. The next stage involved training the facial emotion recognition model using deep learning convolutional neural network with the datasets FER 2013.The classified emotions were assigned weights corresponding to the academic affective states. Subsequently, by using the Dlib's face detector and shape predicting algorithm, the gaze direction with head pose estimation, eyes blinking rate and status of the eye (closed or open) were identified. Combining all these modalities obtained from the image traits, we propose an engagement recognition system. The experimental results of the proposed system were validated by the quiz score obtained at the end of each session. This model can be used for real time video processing of the student's affective state. The teacher can obtain a detailed analytics of engagement statics on a spreadsheet at the end of the session thus facilitating the necessary follow-up actions.

show abstract

DLEmotion: Deep learning-based emotion classification using visual attention location information

Eslamian Koupaei,

Daliri

2024

Biomedical Signal Processing and Control

View full text Add to dashboard Cite

A machine learning approach to predict emotional arousal and valence from gaze extracted features

Cited by 6 publications

References 16 publications

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

eSEE-d: Emotional State Estimation Based on Eye-Tracking Dataset

Multimodal Engagement Recognition From Image Traits Using Deep Learning Techniques

DLEmotion: Deep learning-based emotion classification using visual attention location information

Contact Info

Product

Resources

About